[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:15:29.504280 27116 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.123.62:44811
I20260709 12:15:29.506102 27116 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:15:29.507325 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:29.527143 27123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:29.527285 27122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:29.527148 27125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:30.685497 27124 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1156 milliseconds
I20260709 12:15:30.685638 27116 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 12:15:30.687044 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:30.687279 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:30.687443 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599330687421 us; error 0 us; skew 500 ppm
I20260709 12:15:30.688249 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:30.695279 27116 webserver.cc:533] Webserver started at http://127.26.123.62:35165/ using document root <none> and password file <none>
I20260709 12:15:30.696252 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:30.696466 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:30.696908 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:30.703161 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/master-0-root/instance:
uuid: "967bd590434a476eb193ea471ecd8707"
format_stamp: "Formatted at 2026-07-09 12:15:30 on dist-test-slave-9q7n"
I20260709 12:15:30.712292 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:15:30.718952 27131 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:30.720316 27116 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 12:15:30.720700 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/master-0-root
uuid: "967bd590434a476eb193ea471ecd8707"
format_stamp: "Formatted at 2026-07-09 12:15:30 on dist-test-slave-9q7n"
I20260709 12:15:30.721072 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:30.776721 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:30.778537 27116 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:15:30.779047 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:30.862514 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.62:44811
I20260709 12:15:30.862593 27182 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.62:44811 every 8 connection(s)
I20260709 12:15:30.871240 27183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:30.897948 27183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707: Bootstrap starting.
I20260709 12:15:30.904737 27183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:30.906913 27183 log.cc:826] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707: Log is configured to *not* fsync() on all Append() calls
I20260709 12:15:30.912237 27183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707: No bootstrap required, opened a new log
I20260709 12:15:30.935365 27183 raft_consensus.cc:359] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967bd590434a476eb193ea471ecd8707" member_type: VOTER }
I20260709 12:15:30.935981 27183 raft_consensus.cc:385] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:30.936247 27183 raft_consensus.cc:740] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 967bd590434a476eb193ea471ecd8707, State: Initialized, Role: FOLLOWER
I20260709 12:15:30.937259 27183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967bd590434a476eb193ea471ecd8707" member_type: VOTER }
I20260709 12:15:30.937885 27183 raft_consensus.cc:399] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:15:30.938179 27183 raft_consensus.cc:493] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:15:30.938515 27183 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:30.943593 27183 raft_consensus.cc:515] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967bd590434a476eb193ea471ecd8707" member_type: VOTER }
I20260709 12:15:30.944496 27183 leader_election.cc:304] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 967bd590434a476eb193ea471ecd8707; no voters: 
I20260709 12:15:30.946579 27183 leader_election.cc:290] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:15:30.946941 27186 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:30.949834 27186 raft_consensus.cc:697] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 1 LEADER]: Becoming Leader. State: Replica: 967bd590434a476eb193ea471ecd8707, State: Running, Role: LEADER
I20260709 12:15:30.950841 27186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967bd590434a476eb193ea471ecd8707" member_type: VOTER }
I20260709 12:15:30.952069 27183 sys_catalog.cc:565] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:15:30.962678 27187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "967bd590434a476eb193ea471ecd8707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967bd590434a476eb193ea471ecd8707" member_type: VOTER } }
I20260709 12:15:30.963711 27187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:30.963593 27188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 967bd590434a476eb193ea471ecd8707. Latest consensus state: current_term: 1 leader_uuid: "967bd590434a476eb193ea471ecd8707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967bd590434a476eb193ea471ecd8707" member_type: VOTER } }
I20260709 12:15:30.964329 27188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:30.967578 27194 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:15:30.979347 27194 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:15:30.983543 27116 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:15:30.999250 27194 catalog_manager.cc:1383] Generated new cluster ID: 49226e4153374f1eaab8c27512d77a21
I20260709 12:15:30.999600 27194 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:15:31.019821 27194 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:15:31.021546 27194 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:15:31.039551 27194 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707: Generated new TSK 0
I20260709 12:15:31.040659 27194 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:15:31.051033 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:31.060627 27205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:31.061878 27206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:31.066664 27208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:31.066886 27116 server_base.cc:1061] running on GCE node
I20260709 12:15:31.067781 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:31.067988 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:31.068179 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599331068158 us; error 0 us; skew 500 ppm
I20260709 12:15:31.068819 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:31.073853 27116 webserver.cc:533] Webserver started at http://127.26.123.1:34199/ using document root <none> and password file <none>
I20260709 12:15:31.074488 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:31.074688 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:31.074991 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:31.076457 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-0-root/instance:
uuid: "05d1aaa742b64c7986de8e602f23333c"
format_stamp: "Formatted at 2026-07-09 12:15:31 on dist-test-slave-9q7n"
I20260709 12:15:31.082858 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.003s
I20260709 12:15:31.087730 27213 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:31.088682 27116 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:15:31.089133 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "05d1aaa742b64c7986de8e602f23333c"
format_stamp: "Formatted at 2026-07-09 12:15:31 on dist-test-slave-9q7n"
I20260709 12:15:31.089576 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:31.119324 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:31.121299 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:31.124435 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:31.130543 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:31.130909 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:31.131255 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:31.131490 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:31.237485 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:46627
I20260709 12:15:31.237617 27275 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:46627 every 8 connection(s)
I20260709 12:15:31.245532 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:31.260098 27280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:31.262362 27281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:31.273415 27283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:31.273907 27116 server_base.cc:1061] running on GCE node
I20260709 12:15:31.275061 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:31.275362 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:31.275596 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599331275573 us; error 0 us; skew 500 ppm
I20260709 12:15:31.276476 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:31.281155 27116 webserver.cc:533] Webserver started at http://127.26.123.2:41683/ using document root <none> and password file <none>
I20260709 12:15:31.281934 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:31.282220 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:31.282580 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:31.284045 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-1-root/instance:
uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
format_stamp: "Formatted at 2026-07-09 12:15:31 on dist-test-slave-9q7n"
I20260709 12:15:31.286482 27276 heartbeater.cc:344] Connected to a master server at 127.26.123.62:44811
I20260709 12:15:31.287038 27276 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:31.288435 27276 heartbeater.cc:507] Master 127.26.123.62:44811 requested a full tablet report, sending...
I20260709 12:15:31.292279 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.007s	sys 0.003s
I20260709 12:15:31.292805 27148 ts_manager.cc:194] Registered new tserver with Master: 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627)
I20260709 12:15:31.296144 27148 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52468
I20260709 12:15:31.298609 27288 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:31.299731 27116 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:15:31.300202 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-1-root
uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
format_stamp: "Formatted at 2026-07-09 12:15:31 on dist-test-slave-9q7n"
I20260709 12:15:31.300688 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:31.319922 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:31.321429 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:31.323983 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:31.327395 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:31.327701 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:31.328034 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:31.328253 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:31.435973 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.2:37647
I20260709 12:15:31.436285 27350 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.2:37647 every 8 connection(s)
I20260709 12:15:31.443465 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:31.455257 27354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:31.459105 27355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:31.466784 27351 heartbeater.cc:344] Connected to a master server at 127.26.123.62:44811
I20260709 12:15:31.467541 27351 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:31.468433 27351 heartbeater.cc:507] Master 127.26.123.62:44811 requested a full tablet report, sending...
I20260709 12:15:31.470781 27148 ts_manager.cc:194] Registered new tserver with Master: fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:31.472215 27148 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52470
W20260709 12:15:31.476444 27357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:31.478329 27116 server_base.cc:1061] running on GCE node
I20260709 12:15:31.479377 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:31.479579 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:31.479770 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599331479748 us; error 0 us; skew 500 ppm
I20260709 12:15:31.480370 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:31.483372 27116 webserver.cc:533] Webserver started at http://127.26.123.3:32981/ using document root <none> and password file <none>
I20260709 12:15:31.484095 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:31.484350 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:31.484639 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:31.485772 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-2-root/instance:
uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
format_stamp: "Formatted at 2026-07-09 12:15:31 on dist-test-slave-9q7n"
I20260709 12:15:31.490679 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:15:31.494495 27362 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:31.495465 27116 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:15:31.495872 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-2-root
uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
format_stamp: "Formatted at 2026-07-09 12:15:31 on dist-test-slave-9q7n"
I20260709 12:15:31.496289 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599329322122-27116-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:31.510226 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:31.512074 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:31.514333 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:31.518632 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:31.518956 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:31.519282 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:31.519513 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:31.620820 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.3:34435
I20260709 12:15:31.620893 27424 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.3:34435 every 8 connection(s)
I20260709 12:15:31.639820 27425 heartbeater.cc:344] Connected to a master server at 127.26.123.62:44811
I20260709 12:15:31.640221 27425 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:31.641058 27425 heartbeater.cc:507] Master 127.26.123.62:44811 requested a full tablet report, sending...
I20260709 12:15:31.643275 27148 ts_manager.cc:194] Registered new tserver with Master: ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:31.644024 27116 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016929629s
I20260709 12:15:31.645004 27148 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52476
I20260709 12:15:31.702720 27148 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52484:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:15:31.707073 27148 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:15:31.833673 27316 tablet_service.cc:1511] Processing CreateTablet for tablet 3e212ead0131461882a03aed471bb0b7 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:31.835140 27312 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd7e88f8aed47f49bed724c4173b51a (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:31.835767 27316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e212ead0131461882a03aed471bb0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.836938 27312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd7e88f8aed47f49bed724c4173b51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.843772 27313 tablet_service.cc:1511] Processing CreateTablet for tablet 20d96a0ba5324569a4286fd261c93c8c (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:31.833673 27315 tablet_service.cc:1511] Processing CreateTablet for tablet 954e28fb958e467c9d6be24151505db4 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:31.843189 27314 tablet_service.cc:1511] Processing CreateTablet for tablet 61c1a911eff14ce4ad34d1401e44e6bc (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:31.846952 27311 tablet_service.cc:1511] Processing CreateTablet for tablet 6b251acd25d84b029195fd87c744df81 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:31.850917 27315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 954e28fb958e467c9d6be24151505db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.853267 27313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20d96a0ba5324569a4286fd261c93c8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.872151 27311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b251acd25d84b029195fd87c744df81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.876044 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61c1a911eff14ce4ad34d1401e44e6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.897533 27390 tablet_service.cc:1511] Processing CreateTablet for tablet 3e212ead0131461882a03aed471bb0b7 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:31.891852 27310 tablet_service.cc:1511] Processing CreateTablet for tablet 5f632f7ba89546b9a966c6a2ce46f5f3 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:31.900259 27389 tablet_service.cc:1511] Processing CreateTablet for tablet 954e28fb958e467c9d6be24151505db4 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:31.902354 27388 tablet_service.cc:1511] Processing CreateTablet for tablet 61c1a911eff14ce4ad34d1401e44e6bc (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:31.907490 27309 tablet_service.cc:1511] Processing CreateTablet for tablet a6b134736e8042e49a6f57e52e2eee59 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:31.921239 27390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e212ead0131461882a03aed471bb0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.918263 27387 tablet_service.cc:1511] Processing CreateTablet for tablet 20d96a0ba5324569a4286fd261c93c8c (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:31.929929 27310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f632f7ba89546b9a966c6a2ce46f5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.916695 27386 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd7e88f8aed47f49bed724c4173b51a (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:31.936723 27389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 954e28fb958e467c9d6be24151505db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.945554 27240 tablet_service.cc:1511] Processing CreateTablet for tablet 954e28fb958e467c9d6be24151505db4 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:31.950251 27309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b134736e8042e49a6f57e52e2eee59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.947042 27238 tablet_service.cc:1511] Processing CreateTablet for tablet 20d96a0ba5324569a4286fd261c93c8c (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:31.936681 27385 tablet_service.cc:1511] Processing CreateTablet for tablet 6b251acd25d84b029195fd87c744df81 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:31.947060 27384 tablet_service.cc:1511] Processing CreateTablet for tablet 5f632f7ba89546b9a966c6a2ce46f5f3 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:31.945544 27239 tablet_service.cc:1511] Processing CreateTablet for tablet 61c1a911eff14ce4ad34d1401e44e6bc (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:31.951885 27388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61c1a911eff14ce4ad34d1401e44e6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.949678 27237 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd7e88f8aed47f49bed724c4173b51a (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:31.952345 27383 tablet_service.cc:1511] Processing CreateTablet for tablet a6b134736e8042e49a6f57e52e2eee59 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:31.950971 27241 tablet_service.cc:1511] Processing CreateTablet for tablet 3e212ead0131461882a03aed471bb0b7 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:31.968801 27236 tablet_service.cc:1511] Processing CreateTablet for tablet 6b251acd25d84b029195fd87c744df81 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:31.972857 27240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 954e28fb958e467c9d6be24151505db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.975034 27308 tablet_service.cc:1511] Processing CreateTablet for tablet 42d75d4decde4719873000d6a56e4992 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
W20260709 12:15:31.976440 27196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:31.976879 27387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20d96a0ba5324569a4286fd261c93c8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.980480 27308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d75d4decde4719873000d6a56e4992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.993927 27234 tablet_service.cc:1511] Processing CreateTablet for tablet a6b134736e8042e49a6f57e52e2eee59 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:31.988713 27241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e212ead0131461882a03aed471bb0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:31.986155 27235 tablet_service.cc:1511] Processing CreateTablet for tablet 5f632f7ba89546b9a966c6a2ce46f5f3 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:31.996889 27384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f632f7ba89546b9a966c6a2ce46f5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.005991 27382 tablet_service.cc:1511] Processing CreateTablet for tablet 42d75d4decde4719873000d6a56e4992 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:32.002871 27235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f632f7ba89546b9a966c6a2ce46f5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.008971 27233 tablet_service.cc:1511] Processing CreateTablet for tablet 42d75d4decde4719873000d6a56e4992 (DEFAULT_TABLE table=test-workload [id=e4feff7c414941568ff88cc8ba3618b3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:32.020493 27236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b251acd25d84b029195fd87c744df81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.022728 27234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b134736e8042e49a6f57e52e2eee59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.026007 27233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d75d4decde4719873000d6a56e4992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.027832 27383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b134736e8042e49a6f57e52e2eee59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.029258 27237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd7e88f8aed47f49bed724c4173b51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.030336 27382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d75d4decde4719873000d6a56e4992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.032748 27238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20d96a0ba5324569a4286fd261c93c8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.038060 27239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61c1a911eff14ce4ad34d1401e44e6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.039259 27385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b251acd25d84b029195fd87c744df81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.059042 27386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd7e88f8aed47f49bed724c4173b51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:32.119581 27443 tablet_bootstrap.cc:492] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5: Bootstrap starting.
I20260709 12:15:32.175100 27443 tablet_bootstrap.cc:654] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.199491 27444 tablet_bootstrap.cc:492] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9: Bootstrap starting.
I20260709 12:15:32.235026 27445 tablet_bootstrap.cc:492] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c: Bootstrap starting.
I20260709 12:15:32.244738 27444 tablet_bootstrap.cc:654] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.244823 27445 tablet_bootstrap.cc:654] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.256132 27443 tablet_bootstrap.cc:492] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5: No bootstrap required, opened a new log
I20260709 12:15:32.256734 27443 ts_tablet_manager.cc:1403] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent bootstrapping tablet: real 0.141s	user 0.012s	sys 0.006s
I20260709 12:15:32.264104 27443 raft_consensus.cc:359] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.264986 27443 raft_consensus.cc:385] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.265311 27443 raft_consensus.cc:740] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.270334 27443 consensus_queue.cc:260] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.282394 27351 heartbeater.cc:499] Master 127.26.123.62:44811 was elected leader, sending a full tablet report...
I20260709 12:15:32.283353 27445 tablet_bootstrap.cc:492] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c: No bootstrap required, opened a new log
I20260709 12:15:32.283874 27445 ts_tablet_manager.cc:1403] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.025s
I20260709 12:15:32.283917 27444 tablet_bootstrap.cc:492] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9: No bootstrap required, opened a new log
I20260709 12:15:32.284482 27444 ts_tablet_manager.cc:1403] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent bootstrapping tablet: real 0.086s	user 0.027s	sys 0.023s
I20260709 12:15:32.287531 27445 raft_consensus.cc:359] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.288394 27445 raft_consensus.cc:385] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.288705 27443 ts_tablet_manager.cc:1434] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:15:32.288187 27444 raft_consensus.cc:359] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.289356 27444 raft_consensus.cc:385] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.289798 27444 raft_consensus.cc:740] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.288786 27445 raft_consensus.cc:740] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.290743 27443 tablet_bootstrap.cc:492] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5: Bootstrap starting.
I20260709 12:15:32.292654 27444 consensus_queue.cc:260] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.291502 27445 consensus_queue.cc:260] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.299249 27443 tablet_bootstrap.cc:654] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.300938 27276 heartbeater.cc:499] Master 127.26.123.62:44811 was elected leader, sending a full tablet report...
I20260709 12:15:32.334583 27425 heartbeater.cc:499] Master 127.26.123.62:44811 was elected leader, sending a full tablet report...
I20260709 12:15:32.338992 27445 ts_tablet_manager.cc:1434] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c: Time spent starting tablet: real 0.055s	user 0.010s	sys 0.019s
I20260709 12:15:32.340164 27445 tablet_bootstrap.cc:492] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c: Bootstrap starting.
I20260709 12:15:32.344344 27444 ts_tablet_manager.cc:1434] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent starting tablet: real 0.060s	user 0.021s	sys 0.005s
I20260709 12:15:32.345572 27444 tablet_bootstrap.cc:492] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9: Bootstrap starting.
I20260709 12:15:32.356782 27445 tablet_bootstrap.cc:654] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.362001 27444 tablet_bootstrap.cc:654] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.371652 27445 tablet_bootstrap.cc:492] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c: No bootstrap required, opened a new log
I20260709 12:15:32.372210 27445 ts_tablet_manager.cc:1403] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.005s
I20260709 12:15:32.375679 27445 raft_consensus.cc:359] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.377671 27443 tablet_bootstrap.cc:492] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5: No bootstrap required, opened a new log
I20260709 12:15:32.376628 27445 raft_consensus.cc:385] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.378304 27443 ts_tablet_manager.cc:1403] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5: Time spent bootstrapping tablet: real 0.089s	user 0.037s	sys 0.038s
I20260709 12:15:32.378285 27445 raft_consensus.cc:740] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.379256 27445 consensus_queue.cc:260] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
W20260709 12:15:32.380944 27426 tablet.cc:2406] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:15:32.381924 27445 ts_tablet_manager.cc:1434] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 12:15:32.382408 27444 tablet_bootstrap.cc:492] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9: No bootstrap required, opened a new log
I20260709 12:15:32.382913 27444 ts_tablet_manager.cc:1403] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.005s
I20260709 12:15:32.384935 27443 raft_consensus.cc:359] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.385953 27443 raft_consensus.cc:385] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.386286 27443 raft_consensus.cc:740] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.386248 27444 raft_consensus.cc:359] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.387331 27444 raft_consensus.cc:385] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.387630 27445 tablet_bootstrap.cc:492] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c: Bootstrap starting.
I20260709 12:15:32.387259 27443 consensus_queue.cc:260] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.387645 27444 raft_consensus.cc:740] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.392241 27444 consensus_queue.cc:260] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.395741 27443 ts_tablet_manager.cc:1434] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:15:32.394630 27444 ts_tablet_manager.cc:1434] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:15:32.396879 27444 tablet_bootstrap.cc:492] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9: Bootstrap starting.
I20260709 12:15:32.396879 27443 tablet_bootstrap.cc:492] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5: Bootstrap starting.
I20260709 12:15:32.405654 27444 tablet_bootstrap.cc:654] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.406527 27445 tablet_bootstrap.cc:654] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.411998 27443 tablet_bootstrap.cc:654] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.416520 27451 raft_consensus.cc:493] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.417073 27451 raft_consensus.cc:515] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.417879 27444 tablet_bootstrap.cc:492] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9: No bootstrap required, opened a new log
I20260709 12:15:32.417928 27445 tablet_bootstrap.cc:492] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c: No bootstrap required, opened a new log
I20260709 12:15:32.418639 27445 ts_tablet_manager.cc:1403] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.008s
I20260709 12:15:32.418650 27444 ts_tablet_manager.cc:1403] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.004s
I20260709 12:15:32.421554 27451 leader_election.cc:290] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:32.422637 27444 raft_consensus.cc:359] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.423538 27444 raft_consensus.cc:385] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.423889 27444 raft_consensus.cc:740] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.424935 27444 consensus_queue.cc:260] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.427561 27444 ts_tablet_manager.cc:1434] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:15:32.428772 27444 tablet_bootstrap.cc:492] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9: Bootstrap starting.
I20260709 12:15:32.434806 27445 raft_consensus.cc:359] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.436239 27444 tablet_bootstrap.cc:654] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.438776 27445 raft_consensus.cc:385] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.439452 27445 raft_consensus.cc:740] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.440389 27445 consensus_queue.cc:260] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.443471 27445 ts_tablet_manager.cc:1434] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.005s
I20260709 12:15:32.446477 27445 tablet_bootstrap.cc:492] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c: Bootstrap starting.
I20260709 12:15:32.461906 27443 tablet_bootstrap.cc:492] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5: No bootstrap required, opened a new log
I20260709 12:15:32.462476 27443 ts_tablet_manager.cc:1403] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.007s
I20260709 12:15:32.463986 27444 tablet_bootstrap.cc:492] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9: No bootstrap required, opened a new log
I20260709 12:15:32.464604 27444 ts_tablet_manager.cc:1403] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent bootstrapping tablet: real 0.036s	user 0.005s	sys 0.021s
I20260709 12:15:32.465848 27443 raft_consensus.cc:359] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.466666 27443 raft_consensus.cc:385] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.466966 27443 raft_consensus.cc:740] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.467913 27445 tablet_bootstrap.cc:654] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.467787 27443 consensus_queue.cc:260] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.470518 27443 ts_tablet_manager.cc:1434] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:15:32.470760 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b251acd25d84b029195fd87c744df81" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:32.471755 27326 raft_consensus.cc:2468] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 0.
I20260709 12:15:32.471995 27444 raft_consensus.cc:359] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.472590 27443 tablet_bootstrap.cc:492] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5: Bootstrap starting.
I20260709 12:15:32.472793 27444 raft_consensus.cc:385] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.473098 27444 raft_consensus.cc:740] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.473907 27444 consensus_queue.cc:260] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.480124 27443 tablet_bootstrap.cc:654] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.480490 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b251acd25d84b029195fd87c744df81" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c" is_pre_election: true
I20260709 12:15:32.481580 27365 leader_election.cc:304] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:32.486217 27444 ts_tablet_manager.cc:1434] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:15:32.487361 27451 raft_consensus.cc:493] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:32.487794 27364 leader_election.cc:343] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.488148 27451 raft_consensus.cc:515] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.491706 27451 leader_election.cc:290] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627)
I20260709 12:15:32.492318 27451 raft_consensus.cc:2804] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:32.492729 27451 raft_consensus.cc:493] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:32.493103 27451 raft_consensus.cc:3060] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.495945 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c" is_pre_election: true
I20260709 12:15:32.497615 27444 tablet_bootstrap.cc:492] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9: Bootstrap starting.
W20260709 12:15:32.497993 27364 leader_election.cc:343] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.505856 27451 raft_consensus.cc:515] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.509994 27445 tablet_bootstrap.cc:492] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c: No bootstrap required, opened a new log
I20260709 12:15:32.510602 27445 ts_tablet_manager.cc:1403] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.006s
I20260709 12:15:32.510663 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:32.513063 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b251acd25d84b029195fd87c744df81" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
W20260709 12:15:32.514643 27364 leader_election.cc:343] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.515231 27445 raft_consensus.cc:359] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.515724 27451 leader_election.cc:290] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:32.516022 27445 raft_consensus.cc:385] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.516366 27445 raft_consensus.cc:740] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.517643 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b251acd25d84b029195fd87c744df81" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:32.518321 27326 raft_consensus.cc:3060] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.519449 27444 tablet_bootstrap.cc:654] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:32.519572 27365 leader_election.cc:343] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.520208 27365 leader_election.cc:304] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:32.520831 27445 consensus_queue.cc:260] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.522924 27443 tablet_bootstrap.cc:492] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5: No bootstrap required, opened a new log
I20260709 12:15:32.526333 27443 ts_tablet_manager.cc:1403] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent bootstrapping tablet: real 0.054s	user 0.019s	sys 0.002s
I20260709 12:15:32.529224 27326 raft_consensus.cc:2468] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 1.
I20260709 12:15:32.529620 27443 raft_consensus.cc:359] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.530503 27443 raft_consensus.cc:385] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.530828 27443 raft_consensus.cc:740] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.531597 27443 consensus_queue.cc:260] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.534042 27443 ts_tablet_manager.cc:1434] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:15:32.535068 27445 ts_tablet_manager.cc:1434] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:15:32.535288 27443 tablet_bootstrap.cc:492] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5: Bootstrap starting.
I20260709 12:15:32.536123 27445 tablet_bootstrap.cc:492] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c: Bootstrap starting.
I20260709 12:15:32.544204 27443 tablet_bootstrap.cc:654] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.544675 27451 raft_consensus.cc:493] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.545151 27465 raft_consensus.cc:2749] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:32.545188 27451 raft_consensus.cc:515] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.547247 27365 leader_election.cc:304] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 05d1aaa742b64c7986de8e602f23333c
I20260709 12:15:32.547688 27451 leader_election.cc:290] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:32.549561 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d75d4decde4719873000d6a56e4992" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c" is_pre_election: true
W20260709 12:15:32.551225 27364 leader_election.cc:343] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.551988 27445 tablet_bootstrap.cc:654] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.556989 27451 raft_consensus.cc:2804] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:32.557400 27443 tablet_bootstrap.cc:492] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5: No bootstrap required, opened a new log
I20260709 12:15:32.557561 27451 raft_consensus.cc:697] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Becoming Leader. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:32.558099 27443 ts_tablet_manager.cc:1403] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.012s
I20260709 12:15:32.558497 27451 consensus_queue.cc:237] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.561031 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d75d4decde4719873000d6a56e4992" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:32.562222 27443 raft_consensus.cc:359] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
W20260709 12:15:32.562810 27365 leader_election.cc:343] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.563059 27443 raft_consensus.cc:385] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.563488 27443 raft_consensus.cc:740] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.563539 27365 leader_election.cc:304] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:32.564368 27443 consensus_queue.cc:260] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.564843 27465 raft_consensus.cc:2749] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:32.567257 27443 ts_tablet_manager.cc:1434] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:15:32.568392 27443 tablet_bootstrap.cc:492] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5: Bootstrap starting.
I20260709 12:15:32.580215 27443 tablet_bootstrap.cc:654] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.581007 27450 raft_consensus.cc:493] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.581758 27444 tablet_bootstrap.cc:492] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9: No bootstrap required, opened a new log
I20260709 12:15:32.581571 27450 raft_consensus.cc:515] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.582470 27444 ts_tablet_manager.cc:1403] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent bootstrapping tablet: real 0.085s	user 0.025s	sys 0.021s
I20260709 12:15:32.582190 27146 catalog_manager.cc:5697] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:32.594132 27450 leader_election.cc:290] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:32.599591 27445 tablet_bootstrap.cc:492] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c: No bootstrap required, opened a new log
I20260709 12:15:32.607807 27445 ts_tablet_manager.cc:1403] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.002s
I20260709 12:15:32.609133 27449 raft_consensus.cc:493] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.608404 27444 raft_consensus.cc:359] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.613195 27449 raft_consensus.cc:515] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.614336 27444 raft_consensus.cc:385] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.617161 27444 raft_consensus.cc:740] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.624243 27449 leader_election.cc:290] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:32.622382 27444 consensus_queue.cc:260] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.632619 27444 ts_tablet_manager.cc:1434] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.001s
I20260709 12:15:32.636885 27444 tablet_bootstrap.cc:492] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9: Bootstrap starting.
I20260709 12:15:32.644037 27445 raft_consensus.cc:359] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.646385 27445 raft_consensus.cc:385] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.646732 27445 raft_consensus.cc:740] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.647899 27443 tablet_bootstrap.cc:492] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5: No bootstrap required, opened a new log
I20260709 12:15:32.649344 27443 ts_tablet_manager.cc:1403] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.012s
I20260709 12:15:32.660622 27444 tablet_bootstrap.cc:654] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.658025 27443 raft_consensus.cc:359] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.651342 27445 consensus_queue.cc:260] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.662829 27443 raft_consensus.cc:385] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.663196 27443 raft_consensus.cc:740] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.666292 27443 consensus_queue.cc:260] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.670464 27443 ts_tablet_manager.cc:1434] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 12:15:32.676580 27443 tablet_bootstrap.cc:492] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5: Bootstrap starting.
I20260709 12:15:32.673473 27450 raft_consensus.cc:493] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.679939 27450 raft_consensus.cc:515] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.682468 27450 leader_election.cc:290] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:32.678035 27445 ts_tablet_manager.cc:1434] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c: Time spent starting tablet: real 0.070s	user 0.004s	sys 0.007s
I20260709 12:15:32.688318 27445 tablet_bootstrap.cc:492] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c: Bootstrap starting.
I20260709 12:15:32.688257 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954e28fb958e467c9d6be24151505db4" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" is_pre_election: true
I20260709 12:15:32.692741 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" is_pre_election: true
W20260709 12:15:32.693473 27215 leader_election.cc:343] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:32.697170 27215 leader_election.cc:343] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.697846 27450 raft_consensus.cc:493] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.698328 27450 raft_consensus.cc:515] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.704698 27450 leader_election.cc:290] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:32.707274 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" is_pre_election: true
W20260709 12:15:32.711017 27215 leader_election.cc:343] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.715538 27450 raft_consensus.cc:493] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.716822 27450 raft_consensus.cc:515] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.719951 27450 leader_election.cc:290] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:32.720039 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954e28fb958e467c9d6be24151505db4" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:32.722293 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:32.723608 27326 raft_consensus.cc:2468] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 0.
I20260709 12:15:32.724530 27325 raft_consensus.cc:2468] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 0.
I20260709 12:15:32.730506 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:32.731962 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" is_pre_election: true
I20260709 12:15:32.732722 27443 tablet_bootstrap.cc:654] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.732748 27399 raft_consensus.cc:2468] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 0.
I20260709 12:15:32.736094 27215 leader_election.cc:304] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:32.737488 27450 raft_consensus.cc:2804] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:32.737898 27450 raft_consensus.cc:493] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:32.738267 27450 raft_consensus.cc:3060] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.738950 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:32.761691 27326 raft_consensus.cc:2468] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 0.
I20260709 12:15:32.764287 27444 tablet_bootstrap.cc:492] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9: No bootstrap required, opened a new log
I20260709 12:15:32.764834 27444 ts_tablet_manager.cc:1403] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent bootstrapping tablet: real 0.128s	user 0.011s	sys 0.013s
I20260709 12:15:32.766103 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c" is_pre_election: true
I20260709 12:15:32.766479 27216 leader_election.cc:304] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:32.766084 27450 raft_consensus.cc:515] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.778067 27450 leader_election.cc:290] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 election: Requested vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:32.761602 27445 tablet_bootstrap.cc:654] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.780576 27216 leader_election.cc:304] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:32.785905 27451 raft_consensus.cc:493] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.786185 27450 raft_consensus.cc:2804] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:32.786638 27450 raft_consensus.cc:493] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:32.786337 27444 raft_consensus.cc:359] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.786509 27451 raft_consensus.cc:515] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.787099 27450 raft_consensus.cc:3060] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.780180 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:32.788007 27399 raft_consensus.cc:3060] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.789937 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c" is_pre_election: true
I20260709 12:15:32.790684 27251 raft_consensus.cc:2468] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 0.
I20260709 12:15:32.792872 27451 leader_election.cc:290] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
W20260709 12:15:32.792975 27290 leader_election.cc:343] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.794096 27482 raft_consensus.cc:2804] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:32.795307 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:32.796185 27443 tablet_bootstrap.cc:492] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5: No bootstrap required, opened a new log
I20260709 12:15:32.796742 27443 ts_tablet_manager.cc:1403] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent bootstrapping tablet: real 0.123s	user 0.021s	sys 0.015s
I20260709 12:15:32.792836 27364 leader_election.cc:304] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:32.787158 27444 raft_consensus.cc:385] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.798955 27444 raft_consensus.cc:740] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.799089 27451 raft_consensus.cc:2804] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:32.795281 27482 raft_consensus.cc:493] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:32.800015 27451 raft_consensus.cc:493] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:32.800442 27451 raft_consensus.cc:3060] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.801079 27445 tablet_bootstrap.cc:492] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c: No bootstrap required, opened a new log
I20260709 12:15:32.800850 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" is_pre_election: true
I20260709 12:15:32.800946 27444 consensus_queue.cc:260] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.801666 27400 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 0.
I20260709 12:15:32.801666 27445 ts_tablet_manager.cc:1403] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c: Time spent bootstrapping tablet: real 0.115s	user 0.018s	sys 0.018s
I20260709 12:15:32.800258 27482 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:32.799120 27365 leader_election.cc:343] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.807749 27450 raft_consensus.cc:515] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.804060 27290 leader_election.cc:304] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 05d1aaa742b64c7986de8e602f23333c
I20260709 12:15:32.810217 27399 raft_consensus.cc:2468] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 1.
I20260709 12:15:32.810750 27449 raft_consensus.cc:2804] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:32.811118 27449 raft_consensus.cc:493] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:32.811435 27449 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:32.805872 27216 leader_election.cc:343] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.811919 27215 leader_election.cc:304] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:32.813083 27216 leader_election.cc:304] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c; no voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:32.820871 27449 raft_consensus.cc:515] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.821887 27444 ts_tablet_manager.cc:1434] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.004s
I20260709 12:15:32.822211 27451 raft_consensus.cc:515] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.824862 27443 raft_consensus.cc:359] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.817034 27483 raft_consensus.cc:2804] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:32.826947 27444 tablet_bootstrap.cc:492] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9: Bootstrap starting.
I20260709 12:15:32.827805 27451 leader_election.cc:290] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:32.825407 27445 raft_consensus.cc:359] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.831619 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:32.829035 27443 raft_consensus.cc:385] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.832949 27443 raft_consensus.cc:740] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Initialized, Role: FOLLOWER
W20260709 12:15:32.833161 27365 leader_election.cc:343] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.830335 27445 raft_consensus.cc:385] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.821269 27482 raft_consensus.cc:515] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.838740 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:32.840097 27449 leader_election.cc:290] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:32.838039 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:32.841598 27251 raft_consensus.cc:3060] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.834270 27485 raft_consensus.cc:2749] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:32.838390 27445 raft_consensus.cc:740] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Initialized, Role: FOLLOWER
W20260709 12:15:32.842631 27290 leader_election.cc:343] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.844628 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:32.845888 27399 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.847589 27444 tablet_bootstrap.cc:654] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.844679 27443 consensus_queue.cc:260] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.853401 27443 ts_tablet_manager.cc:1434] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.004s
I20260709 12:15:32.854295 27445 consensus_queue.cc:260] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.860930 27251 raft_consensus.cc:2468] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 1.
I20260709 12:15:32.864614 27399 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 1.
I20260709 12:15:32.868963 27482 leader_election.cc:290] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 election: Requested vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:32.869334 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:32.870513 27400 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.878679 27364 leader_election.cc:304] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:32.880475 27445 ts_tablet_manager.cc:1434] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c: Time spent starting tablet: real 0.078s	user 0.005s	sys 0.003s
I20260709 12:15:32.880645 27451 raft_consensus.cc:2804] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:32.881108 27451 raft_consensus.cc:697] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Becoming Leader. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:32.881582 27445 tablet_bootstrap.cc:492] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c: Bootstrap starting.
I20260709 12:15:32.881994 27451 consensus_queue.cc:237] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.871307 27450 leader_election.cc:290] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 election: Requested vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:32.883414 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:32.883986 27326 raft_consensus.cc:3060] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.884114 27443 tablet_bootstrap.cc:492] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5: Bootstrap starting.
I20260709 12:15:32.887678 27290 leader_election.cc:304] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 05d1aaa742b64c7986de8e602f23333c
I20260709 12:15:32.889111 27449 raft_consensus.cc:2804] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:32.903491 27400 raft_consensus.cc:2468] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 1.
I20260709 12:15:32.896229 27483 raft_consensus.cc:697] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Becoming Leader. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:32.906929 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954e28fb958e467c9d6be24151505db4" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:32.907542 27325 raft_consensus.cc:3060] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.905074 27483 consensus_queue.cc:237] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.905001 27215 leader_election.cc:304] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:32.909154 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954e28fb958e467c9d6be24151505db4" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:32.910797 27449 raft_consensus.cc:697] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
W20260709 12:15:32.912734 27215 leader_election.cc:343] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.915190 27486 raft_consensus.cc:2804] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:32.915328 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:32.918166 27324 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:32.918519 27326 raft_consensus.cc:2468] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 1.
I20260709 12:15:32.919909 27486 raft_consensus.cc:697] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Becoming Leader. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:32.916853 27449 consensus_queue.cc:237] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.925118 27444 tablet_bootstrap.cc:492] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9: No bootstrap required, opened a new log
I20260709 12:15:32.925606 27444 ts_tablet_manager.cc:1403] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent bootstrapping tablet: real 0.099s	user 0.018s	sys 0.002s
I20260709 12:15:32.929150 27443 tablet_bootstrap.cc:654] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.927306 27147 catalog_manager.cc:5697] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:32.934158 27325 raft_consensus.cc:2468] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 1.
I20260709 12:15:32.935688 27216 leader_election.cc:304] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:32.938832 27450 raft_consensus.cc:2804] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:32.939322 27450 raft_consensus.cc:697] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Becoming Leader. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:32.940372 27450 consensus_queue.cc:237] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.941856 27449 raft_consensus.cc:493] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.921012 27486 consensus_queue.cc:237] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.944463 27445 tablet_bootstrap.cc:654] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:32.934849 27324 raft_consensus.cc:2468] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 1.
I20260709 12:15:32.942399 27449 raft_consensus.cc:515] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.955089 27146 catalog_manager.cc:5697] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:32.958472 27444 raft_consensus.cc:359] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.959245 27444 raft_consensus.cc:385] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:32.959550 27444 raft_consensus.cc:740] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Initialized, Role: FOLLOWER
I20260709 12:15:32.960326 27444 consensus_queue.cc:260] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:32.972961 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c" is_pre_election: true
W20260709 12:15:32.974524 27290 leader_election.cc:343] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.975008 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" is_pre_election: true
W20260709 12:15:32.976791 27290 leader_election.cc:343] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:32.977247 27290 leader_election.cc:304] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fec66eb4aa88487eb30eb41db2fc28c5; no voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:32.978446 27487 raft_consensus.cc:2749] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:32.970592 27449 leader_election.cc:290] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
W20260709 12:15:32.979795 27196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:32.981700 27147 catalog_manager.cc:5697] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c reported cstate change: term changed from 0 to 1, leader changed from <none> to 05d1aaa742b64c7986de8e602f23333c (127.26.123.1). New cstate: current_term: 1 leader_uuid: "05d1aaa742b64c7986de8e602f23333c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:32.989916 27444 ts_tablet_manager.cc:1434] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.003s
I20260709 12:15:32.991139 27444 tablet_bootstrap.cc:492] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9: Bootstrap starting.
I20260709 12:15:32.999046 27445 tablet_bootstrap.cc:492] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c: No bootstrap required, opened a new log
I20260709 12:15:32.999526 27445 ts_tablet_manager.cc:1403] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c: Time spent bootstrapping tablet: real 0.118s	user 0.014s	sys 0.013s
I20260709 12:15:33.000151 27443 tablet_bootstrap.cc:492] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5: No bootstrap required, opened a new log
I20260709 12:15:33.000496 27443 ts_tablet_manager.cc:1403] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent bootstrapping tablet: real 0.121s	user 0.010s	sys 0.005s
I20260709 12:15:33.001771 27445 raft_consensus.cc:359] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.002556 27445 raft_consensus.cc:385] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:33.002707 27443 raft_consensus.cc:359] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.002948 27445 raft_consensus.cc:740] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Initialized, Role: FOLLOWER
I20260709 12:15:33.003389 27443 raft_consensus.cc:385] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:33.003644 27443 raft_consensus.cc:740] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Initialized, Role: FOLLOWER
I20260709 12:15:33.004266 27443 consensus_queue.cc:260] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.004014 27445 consensus_queue.cc:260] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.006691 27445 ts_tablet_manager.cc:1434] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:15:33.008662 27445 tablet_bootstrap.cc:492] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c: Bootstrap starting.
I20260709 12:15:33.010630 27444 tablet_bootstrap.cc:654] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:33.014369 27443 ts_tablet_manager.cc:1434] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:15:33.015372 27443 tablet_bootstrap.cc:492] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5: Bootstrap starting.
I20260709 12:15:33.013993 27147 catalog_manager.cc:5697] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c reported cstate change: term changed from 0 to 1, leader changed from <none> to 05d1aaa742b64c7986de8e602f23333c (127.26.123.1). New cstate: current_term: 1 leader_uuid: "05d1aaa742b64c7986de8e602f23333c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:33.019336 27147 catalog_manager.cc:5697] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c reported cstate change: term changed from 0 to 1, leader changed from <none> to 05d1aaa742b64c7986de8e602f23333c (127.26.123.1). New cstate: current_term: 1 leader_uuid: "05d1aaa742b64c7986de8e602f23333c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:33.018287 27444 tablet_bootstrap.cc:492] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9: No bootstrap required, opened a new log
I20260709 12:15:33.021653 27443 tablet_bootstrap.cc:654] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:33.022003 27444 ts_tablet_manager.cc:1403] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.008s
I20260709 12:15:33.023841 27445 tablet_bootstrap.cc:654] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:33.024886 27444 raft_consensus.cc:359] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.025614 27444 raft_consensus.cc:385] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:33.025908 27444 raft_consensus.cc:740] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Initialized, Role: FOLLOWER
I20260709 12:15:33.026636 27444 consensus_queue.cc:260] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.028671 27444 ts_tablet_manager.cc:1434] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:15:33.029573 27444 tablet_bootstrap.cc:492] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9: Bootstrap starting.
I20260709 12:15:33.043231 27445 tablet_bootstrap.cc:492] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c: No bootstrap required, opened a new log
I20260709 12:15:33.043727 27445 ts_tablet_manager.cc:1403] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.004s
I20260709 12:15:33.046625 27444 tablet_bootstrap.cc:654] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:33.046379 27445 raft_consensus.cc:359] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.047040 27445 raft_consensus.cc:385] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:33.047322 27445 raft_consensus.cc:740] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Initialized, Role: FOLLOWER
I20260709 12:15:33.047550 27486 raft_consensus.cc:493] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.047981 27445 consensus_queue.cc:260] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.048062 27486 raft_consensus.cc:515] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.050410 27445 ts_tablet_manager.cc:1434] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:15:33.050371 27486 leader_election.cc:290] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:33.051960 27445 tablet_bootstrap.cc:492] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c: Bootstrap starting.
I20260709 12:15:33.052417 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d75d4decde4719873000d6a56e4992" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:33.054072 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d75d4decde4719873000d6a56e4992" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" is_pre_election: true
I20260709 12:15:33.054702 27399 raft_consensus.cc:2468] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 0.
W20260709 12:15:33.054764 27216 leader_election.cc:343] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:33.055763 27443 tablet_bootstrap.cc:492] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5: No bootstrap required, opened a new log
I20260709 12:15:33.056252 27443 ts_tablet_manager.cc:1403] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.004s
I20260709 12:15:33.059201 27215 leader_election.cc:304] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:33.059430 27443 raft_consensus.cc:359] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.060187 27443 raft_consensus.cc:385] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:33.060277 27450 raft_consensus.cc:2804] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:33.060514 27443 raft_consensus.cc:740] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Initialized, Role: FOLLOWER
I20260709 12:15:33.060918 27450 raft_consensus.cc:493] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.061352 27450 raft_consensus.cc:3060] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.060375 27486 raft_consensus.cc:493] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.066229 27486 raft_consensus.cc:515] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.067862 27449 raft_consensus.cc:493] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.068307 27486 leader_election.cc:290] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:33.068336 27449 raft_consensus.cc:515] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.070258 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b251acd25d84b029195fd87c744df81" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:33.070549 27449 leader_election.cc:290] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:33.070194 27450 raft_consensus.cc:515] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.071410 27324 raft_consensus.cc:2393] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05d1aaa742b64c7986de8e602f23333c in current term 1: Already voted for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in this term.
I20260709 12:15:33.061347 27443 consensus_queue.cc:260] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.080499 27445 tablet_bootstrap.cc:654] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:33.080746 27443 ts_tablet_manager.cc:1434] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.008s
I20260709 12:15:33.082507 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c" is_pre_election: true
I20260709 12:15:33.073074 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b251acd25d84b029195fd87c744df81" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" is_pre_election: true
I20260709 12:15:33.080036 27451 raft_consensus.cc:493] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.084183 27251 raft_consensus.cc:2393] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fec66eb4aa88487eb30eb41db2fc28c5 in current term 1: Already voted for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in this term.
I20260709 12:15:33.083164 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" is_pre_election: true
I20260709 12:15:33.084321 27451 raft_consensus.cc:515] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.085384 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d75d4decde4719873000d6a56e4992" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:33.086428 27450 leader_election.cc:290] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 election: Requested vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:33.086859 27290 leader_election.cc:304] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fec66eb4aa88487eb30eb41db2fc28c5; no voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:33.087158 27215 leader_election.cc:304] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c; no voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:33.089432 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d75d4decde4719873000d6a56e4992" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:33.089825 27449 raft_consensus.cc:3060] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.091611 27451 leader_election.cc:290] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:33.089998 27324 raft_consensus.cc:3060] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.092502 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c" is_pre_election: true
I20260709 12:15:33.094358 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:33.094997 27450 raft_consensus.cc:3060] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:33.095660 27364 leader_election.cc:343] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:33.095023 27326 raft_consensus.cc:2468] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 0.
I20260709 12:15:33.098654 27365 leader_election.cc:304] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 05d1aaa742b64c7986de8e602f23333c
I20260709 12:15:33.090018 27400 raft_consensus.cc:3060] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.100178 27465 raft_consensus.cc:2804] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:33.100569 27465 raft_consensus.cc:493] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.100893 27465 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.109644 27444 tablet_bootstrap.cc:492] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9: No bootstrap required, opened a new log
I20260709 12:15:33.110194 27444 ts_tablet_manager.cc:1403] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent bootstrapping tablet: real 0.081s	user 0.022s	sys 0.016s
I20260709 12:15:33.110193 27450 raft_consensus.cc:2749] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:33.115758 27324 raft_consensus.cc:2468] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 1.
I20260709 12:15:33.108959 27451 consensus_queue.cc:1048] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:15:33.118602 27216 leader_election.cc:304] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:33.119964 27450 raft_consensus.cc:2804] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:33.120464 27450 raft_consensus.cc:697] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Becoming Leader. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:33.121279 27450 consensus_queue.cc:237] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.128005 27449 raft_consensus.cc:2749] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:33.129145 27445 tablet_bootstrap.cc:492] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c: No bootstrap required, opened a new log
I20260709 12:15:33.130296 27445 ts_tablet_manager.cc:1403] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.023s
I20260709 12:15:33.132445 27445 raft_consensus.cc:359] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.132899 27445 raft_consensus.cc:385] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:33.133126 27445 raft_consensus.cc:740] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Initialized, Role: FOLLOWER
I20260709 12:15:33.123242 27465 raft_consensus.cc:515] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.133667 27445 consensus_queue.cc:260] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.134799 27451 consensus_queue.cc:1048] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.136022 27444 raft_consensus.cc:359] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.136687 27444 raft_consensus.cc:385] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:33.136986 27444 raft_consensus.cc:740] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Initialized, Role: FOLLOWER
I20260709 12:15:33.136371 27400 raft_consensus.cc:2468] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 1.
I20260709 12:15:33.138154 27444 consensus_queue.cc:260] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.138801 27445 ts_tablet_manager.cc:1434] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 12:15:33.139190 27147 catalog_manager.cc:5697] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c reported cstate change: term changed from 0 to 1, leader changed from <none> to 05d1aaa742b64c7986de8e602f23333c (127.26.123.1). New cstate: current_term: 1 leader_uuid: "05d1aaa742b64c7986de8e602f23333c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:33.145892 27444 ts_tablet_manager.cc:1434] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9: Time spent starting tablet: real 0.035s	user 0.013s	sys 0.001s
I20260709 12:15:33.147401 27465 leader_election.cc:290] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:33.150187 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:33.150843 27325 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.158864 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:33.162818 27250 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.170559 27325 raft_consensus.cc:2468] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 1.
I20260709 12:15:33.172469 27365 leader_election.cc:304] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:33.173661 27465 raft_consensus.cc:2804] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:33.174348 27465 raft_consensus.cc:697] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Becoming Leader. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:33.175208 27465 consensus_queue.cc:237] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.175843 27250 raft_consensus.cc:2468] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 1.
I20260709 12:15:33.198961 27146 catalog_manager.cc:5697] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.285583 27465 consensus_queue.cc:1048] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:33.297252 27501 consensus_queue.cc:1048] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.301839 27501 raft_consensus.cc:493] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.302558 27501 raft_consensus.cc:515] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.304944 27501 leader_election.cc:290] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:33.305825 27250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c" is_pre_election: true
I20260709 12:15:33.306572 27250 raft_consensus.cc:2468] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 0.
I20260709 12:15:33.307140 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" is_pre_election: true
I20260709 12:15:33.307929 27325 raft_consensus.cc:2468] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 0.
I20260709 12:15:33.308161 27364 leader_election.cc:304] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:33.314043 27451 raft_consensus.cc:2804] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:33.314556 27451 raft_consensus.cc:493] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.314960 27451 raft_consensus.cc:3060] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.323499 27451 raft_consensus.cc:515] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.326623 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:33.327013 27325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:33.327445 27251 raft_consensus.cc:3060] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.327680 27325 raft_consensus.cc:3060] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.329486 27451 leader_election.cc:290] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:33.334122 27251 raft_consensus.cc:2468] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 1.
I20260709 12:15:33.335191 27325 raft_consensus.cc:2468] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 1.
I20260709 12:15:33.335292 27364 leader_election.cc:304] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:33.336141 27451 raft_consensus.cc:2804] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:33.336655 27451 raft_consensus.cc:697] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Becoming Leader. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:33.337512 27451 consensus_queue.cc:237] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.345479 27147 catalog_manager.cc:5697] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.347071 27449 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:33.358884 27251 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.371750 27487 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:33.391076 27400 raft_consensus.cc:3060] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:33.394454 27450 consensus_queue.cc:1048] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.400666 27450 consensus_queue.cc:1048] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.524269 27486 consensus_queue.cc:1048] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:33.535652 27450 consensus_queue.cc:1048] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.548281 27450 consensus_queue.cc:1048] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.557418 27450 consensus_queue.cc:1048] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:33.591528 27483 consensus_queue.cc:1048] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.607640 27486 consensus_queue.cc:1048] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.642305 27399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
mode: GRACEFUL
new_leader_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:56098
I20260709 12:15:33.643122 27399 raft_consensus.cc:606] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Received request to transfer leadership to TS 05d1aaa742b64c7986de8e602f23333c
I20260709 12:15:33.646299 27451 consensus_queue.cc:1048] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:33.651666 27501 consensus_queue.cc:1048] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.668447 27451 raft_consensus.cc:993] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9: : Instructing follower 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:33.668985 27451 raft_consensus.cc:1081] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Signalling peer 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:33.675208 27249 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:37038
I20260709 12:15:33.675999 27249 raft_consensus.cc:493] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:33.676448 27249 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.686306 27249 raft_consensus.cc:515] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.696743 27249 leader_election.cc:290] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 2 election: Requested vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:33.698630 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:33.699330 27399 raft_consensus.cc:3055] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:33.699744 27399 raft_consensus.cc:740] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:33.700827 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:33.700702 27399 consensus_queue.cc:260] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.701436 27324 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.702149 27399 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.709532 27324 raft_consensus.cc:2468] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 2.
I20260709 12:15:33.710652 27326 raft_consensus.cc:1240] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Rejecting Update request from peer ea6d93ecf57a4f6194e0ffe32f963ce9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:33.710752 27399 raft_consensus.cc:2468] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 2.
I20260709 12:15:33.711159 27216 leader_election.cc:304] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:33.712177 27483 raft_consensus.cc:2804] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:33.712705 27483 raft_consensus.cc:697] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Becoming Leader. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:33.713383 27483 consensus_queue.cc:237] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.722537 27147 catalog_manager.cc:5697] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c reported cstate change: term changed from 1 to 2, leader changed from ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3) to 05d1aaa742b64c7986de8e602f23333c (127.26.123.1). New cstate: current_term: 2 leader_uuid: "05d1aaa742b64c7986de8e602f23333c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:33.725003 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
mode: GRACEFUL
new_leader_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:48240
I20260709 12:15:33.725824 27324 raft_consensus.cc:606] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 LEADER]: Received request to transfer leadership to TS 05d1aaa742b64c7986de8e602f23333c
I20260709 12:15:33.746908 27249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
mode: GRACEFUL
new_leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:37064
I20260709 12:15:33.747489 27249 raft_consensus.cc:606] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Received request to transfer leadership to TS fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:33.753664 27399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b251acd25d84b029195fd87c744df81"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
mode: GRACEFUL
new_leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:56098
I20260709 12:15:33.754302 27399 raft_consensus.cc:606] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Received request to transfer leadership to TS fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:33.759909 27399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
mode: GRACEFUL
new_leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:56098
I20260709 12:15:33.760442 27399 raft_consensus.cc:606] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Received request to transfer leadership to TS fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:33.765913 27399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
mode: GRACEFUL
new_leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:56098
I20260709 12:15:33.766460 27399 raft_consensus.cc:606] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Received request to transfer leadership to TS fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:33.772156 27249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42d75d4decde4719873000d6a56e4992"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
mode: GRACEFUL
new_leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:37064
I20260709 12:15:33.772740 27249 raft_consensus.cc:606] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Received request to transfer leadership to TS ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:33.809248 27501 consensus_queue.cc:1048] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:33.817698 27508 raft_consensus.cc:993] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9: : Instructing follower fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:33.818331 27508 raft_consensus.cc:1081] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Signalling peer fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:33.821285 27326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:48228
I20260709 12:15:33.824054 27501 consensus_queue.cc:1048] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:33.825618 27326 raft_consensus.cc:493] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:33.826179 27326 raft_consensus.cc:3060] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.835908 27326 raft_consensus.cc:515] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.844158 27326 leader_election.cc:290] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:33.845211 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:33.845521 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:33.846482 27249 raft_consensus.cc:3060] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.846768 27399 raft_consensus.cc:3055] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:33.847146 27399 raft_consensus.cc:740] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:33.847999 27399 consensus_queue.cc:260] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.849385 27399 raft_consensus.cc:3060] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.855086 27249 raft_consensus.cc:2468] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:33.856962 27290 leader_election.cc:304] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:33.857935 27487 raft_consensus.cc:2804] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:33.858500 27487 raft_consensus.cc:697] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:33.859464 27487 consensus_queue.cc:237] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.865422 27399 raft_consensus.cc:2468] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:33.871382 27147 catalog_manager.cc:5697] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 1 to 2, leader changed from ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3) to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:33.926486 27486 raft_consensus.cc:993] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c: : Instructing follower fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:33.927027 27483 raft_consensus.cc:1081] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Signalling peer fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:33.928869 27326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:48238
I20260709 12:15:33.929569 27326 raft_consensus.cc:493] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:33.930029 27326 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.938128 27326 raft_consensus.cc:515] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.940692 27326 leader_election.cc:290] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:33.942035 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:33.942026 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:33.942677 27249 raft_consensus.cc:3055] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:33.942819 27399 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.943004 27249 raft_consensus.cc:740] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:33.943791 27249 consensus_queue.cc:260] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.944823 27249 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.950268 27249 raft_consensus.cc:2468] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:33.951259 27399 raft_consensus.cc:2468] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:33.951607 27290 leader_election.cc:304] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:33.952515 27487 raft_consensus.cc:2804] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:33.953222 27487 raft_consensus.cc:697] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:33.954399 27487 consensus_queue.cc:237] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.962569 27532 raft_consensus.cc:993] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5: : Instructing follower 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:33.963102 27533 raft_consensus.cc:1081] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 LEADER]: Signalling peer 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:33.965129 27249 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:37050
I20260709 12:15:33.965809 27249 raft_consensus.cc:493] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:33.965348 27147 catalog_manager.cc:5697] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 1 to 2, leader changed from 05d1aaa742b64c7986de8e602f23333c (127.26.123.1) to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:33.966202 27249 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.973316 27249 raft_consensus.cc:515] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.976171 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:33.976861 27326 raft_consensus.cc:3055] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:33.977226 27326 raft_consensus.cc:740] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:33.977408 27249 leader_election.cc:290] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 2 election: Requested vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:33.978400 27326 consensus_queue.cc:260] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:33.979514 27326 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.980237 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:33.980896 27399 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:33.986492 27326 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 2.
I20260709 12:15:33.987995 27216 leader_election.cc:304] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:33.989017 27483 raft_consensus.cc:2804] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:33.989560 27483 raft_consensus.cc:697] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Becoming Leader. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:33.990451 27483 consensus_queue.cc:237] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.001596 27146 catalog_manager.cc:5697] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c reported cstate change: term changed from 1 to 2, leader changed from fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2) to 05d1aaa742b64c7986de8e602f23333c (127.26.123.1). New cstate: current_term: 2 leader_uuid: "05d1aaa742b64c7986de8e602f23333c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:34.004609 27399 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 2.
I20260709 12:15:34.093850 27508 raft_consensus.cc:993] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9: : Instructing follower fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:34.094311 27465 raft_consensus.cc:1081] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Signalling peer fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:34.096009 27326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b251acd25d84b029195fd87c744df81"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:48228
I20260709 12:15:34.096630 27326 raft_consensus.cc:493] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:34.097011 27326 raft_consensus.cc:3060] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:34.104754 27326 raft_consensus.cc:515] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.105711 27324 raft_consensus.cc:1275] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.107043 27326 leader_election.cc:290] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:34.107211 27483 consensus_queue.cc:1048] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:34.108516 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b251acd25d84b029195fd87c744df81" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:34.108981 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b251acd25d84b029195fd87c744df81" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:34.109144 27399 raft_consensus.cc:3055] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:34.109565 27249 raft_consensus.cc:3060] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:34.109532 27399 raft_consensus.cc:740] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:34.110688 27399 consensus_queue.cc:260] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.113646 27399 raft_consensus.cc:3060] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:34.120515 27400 raft_consensus.cc:1275] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.122946 27450 consensus_queue.cc:1048] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.124758 27399 raft_consensus.cc:2468] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:34.126065 27290 leader_election.cc:304] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:34.126989 27533 raft_consensus.cc:2804] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:34.127488 27533 raft_consensus.cc:697] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:34.128363 27533 consensus_queue.cc:237] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.134637 27249 raft_consensus.cc:2468] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:34.138371 27450 raft_consensus.cc:993] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c: : Instructing follower ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:34.138902 27483 raft_consensus.cc:1081] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Signalling peer ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:34.142452 27400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42d75d4decde4719873000d6a56e4992"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:56086
I20260709 12:15:34.143153 27400 raft_consensus.cc:493] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:34.143576 27400 raft_consensus.cc:3060] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:34.151439 27400 raft_consensus.cc:515] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.151806 27147 catalog_manager.cc:5697] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 1 to 2, leader changed from ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3) to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:34.154526 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d75d4decde4719873000d6a56e4992" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:34.154948 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d75d4decde4719873000d6a56e4992" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:34.155248 27249 raft_consensus.cc:3055] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:34.155601 27326 raft_consensus.cc:3060] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:34.155618 27249 raft_consensus.cc:740] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:34.158574 27249 consensus_queue.cc:260] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.159703 27249 raft_consensus.cc:3060] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:34.164505 27326 raft_consensus.cc:2468] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 2.
I20260709 12:15:34.165057 27400 leader_election.cc:290] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 2 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:34.165796 27365 leader_election.cc:304] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:34.166905 27465 raft_consensus.cc:2804] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:34.167254 27249 raft_consensus.cc:2468] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 2.
I20260709 12:15:34.167450 27465 raft_consensus.cc:697] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 LEADER]: Becoming Leader. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:34.168462 27465 consensus_queue.cc:237] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.175622 27147 catalog_manager.cc:5697] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 reported cstate change: term changed from 1 to 2, leader changed from 05d1aaa742b64c7986de8e602f23333c (127.26.123.1) to ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:34.203392 27508 raft_consensus.cc:993] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9: : Instructing follower fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:34.203917 27465 raft_consensus.cc:1081] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Signalling peer fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:34.205695 27326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:48228
I20260709 12:15:34.206396 27326 raft_consensus.cc:493] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:34.206764 27326 raft_consensus.cc:3060] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:34.212970 27326 raft_consensus.cc:515] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.214820 27326 leader_election.cc:290] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:34.216547 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:34.216463 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b134736e8042e49a6f57e52e2eee59" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:34.217195 27249 raft_consensus.cc:3060] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:34.217290 27400 raft_consensus.cc:3055] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:34.217702 27400 raft_consensus.cc:740] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:34.218650 27400 consensus_queue.cc:260] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.220026 27400 raft_consensus.cc:3060] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:34.222950 27249 raft_consensus.cc:2468] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:34.224160 27290 leader_election.cc:304] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:34.225029 27533 raft_consensus.cc:2804] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:34.225445 27533 raft_consensus.cc:697] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:34.226308 27533 consensus_queue.cc:237] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:34.227254 27400 raft_consensus.cc:2468] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:34.233546 27147 catalog_manager.cc:5697] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 1 to 2, leader changed from ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3) to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:34.252454 27249 raft_consensus.cc:1275] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.254055 27533 consensus_queue.cc:1048] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.263324 27400 raft_consensus.cc:1275] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.264869 27487 consensus_queue.cc:1048] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.361222 27400 raft_consensus.cc:1275] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.362802 27487 consensus_queue.cc:1048] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.372637 27249 raft_consensus.cc:1275] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.374461 27449 consensus_queue.cc:1048] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.496222 27400 raft_consensus.cc:1275] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.497632 27483 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.507090 27326 raft_consensus.cc:1275] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.508911 27483 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.533569 27249 raft_consensus.cc:1275] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.535087 27449 consensus_queue.cc:1048] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.544492 27400 raft_consensus.cc:1275] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.546634 27533 consensus_queue.cc:1048] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.590756 27249 raft_consensus.cc:1275] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.592114 27465 consensus_queue.cc:1048] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.596809 27326 raft_consensus.cc:1275] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.598599 27465 consensus_queue.cc:1048] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.640829 27249 raft_consensus.cc:1275] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.642149 27533 consensus_queue.cc:1048] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:34.650876 27400 raft_consensus.cc:1275] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:34.652570 27533 consensus_queue.cc:1048] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:36.808797 27116 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:15:36.809871 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:36.829069 27249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
mode: GRACEFUL
new_leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:37066
I20260709 12:15:36.829603 27249 raft_consensus.cc:606] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Received request to transfer leadership to TS ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:36.841215 27326 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
mode: GRACEFUL
new_leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:48252
I20260709 12:15:36.841974 27326 raft_consensus.cc:606] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Received request to transfer leadership to TS ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:36.843500 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:36.844061 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:37.093016 27533 raft_consensus.cc:993] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5: : Instructing follower ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:37.093463 27546 raft_consensus.cc:1081] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Signalling peer ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:37.095005 27400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:56090
I20260709 12:15:37.095496 27400 raft_consensus.cc:493] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:37.095820 27400 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:37.101290 27400 raft_consensus.cc:515] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:37.103224 27400 leader_election.cc:290] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 3 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:37.104250 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:37.104385 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:37.104807 27326 raft_consensus.cc:3055] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:37.104976 27249 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:37.105221 27326 raft_consensus.cc:740] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:37.106227 27326 consensus_queue.cc:260] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:37.107407 27326 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:37.109095 27545 raft_consensus.cc:993] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c: : Instructing follower ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:37.109555 27545 raft_consensus.cc:1081] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Signalling peer ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:37.112171 27400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:56086
I20260709 12:15:37.112531 27249 raft_consensus.cc:2468] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 3.
I20260709 12:15:37.112840 27400 raft_consensus.cc:493] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:37.113284 27400 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:37.113991 27364 leader_election.cc:304] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:37.114866 27542 raft_consensus.cc:2804] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:37.115234 27326 raft_consensus.cc:2468] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 3.
I20260709 12:15:37.116746 27542 raft_consensus.cc:697] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Becoming Leader. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:37.117643 27542 consensus_queue.cc:237] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:37.120282 27400 raft_consensus.cc:515] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:37.122180 27400 leader_election.cc:290] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 3 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:37.123154 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:37.123828 27249 raft_consensus.cc:3055] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:37.123868 27326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:37.124230 27249 raft_consensus.cc:740] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:37.124519 27326 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:37.125077 27249 consensus_queue.cc:260] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:37.126526 27249 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:37.126523 27145 catalog_manager.cc:5697] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 reported cstate change: term changed from 2 to 3, leader changed from fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2) to ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3). New cstate: current_term: 3 leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:37.129963 27326 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 3.
I20260709 12:15:37.131237 27365 leader_election.cc:304] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:37.132102 27542 raft_consensus.cc:2804] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:37.132617 27542 raft_consensus.cc:697] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Becoming Leader. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:37.133865 27249 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 3.
I20260709 12:15:37.133472 27542 consensus_queue.cc:237] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:37.146081 27145 catalog_manager.cc:5697] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 reported cstate change: term changed from 2 to 3, leader changed from 05d1aaa742b64c7986de8e602f23333c (127.26.123.1) to ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3). New cstate: current_term: 3 leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:37.561630 27326 raft_consensus.cc:1275] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:37.562889 27542 consensus_queue.cc:1048] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:37.571914 27249 raft_consensus.cc:1275] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:37.574571 27249 raft_consensus.cc:1275] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:37.575464 27560 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:37.576037 27542 consensus_queue.cc:1048] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:37.589337 27326 raft_consensus.cc:1275] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:37.594499 27560 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:37.844959 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:37.850000 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:37.850426 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:38.851447 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:38.856843 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:38.857246 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:39.858517 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:39.866452 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:39.866918 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:40.868048 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:40.873371 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:40.873876 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:41.875013 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:41.880425 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:41.880869 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:42.881923 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:42.887220 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:42.887795 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:43.888911 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:43.894531 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:43.894955 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:44.896102 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:44.902432 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:44.902872 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:45.903888 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:45.909240 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:45.909827 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:46.956852 27116 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:15:46.961786 27249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
mode: GRACEFUL
new_leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:37064
I20260709 12:15:46.962479 27249 raft_consensus.cc:606] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Received request to transfer leadership to TS fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:46.967679 27249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "954e28fb958e467c9d6be24151505db4"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
mode: GRACEFUL
new_leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:37064
I20260709 12:15:46.968173 27249 raft_consensus.cc:606] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Received request to transfer leadership to TS fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:46.973273 27249 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
mode: GRACEFUL
new_leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:37064
I20260709 12:15:46.973862 27249 raft_consensus.cc:606] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Received request to transfer leadership to TS fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:46.978816 27400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
mode: GRACEFUL
new_leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:56098
I20260709 12:15:46.979332 27400 raft_consensus.cc:606] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Received request to transfer leadership to TS fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:46.984241 27400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
mode: GRACEFUL
new_leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:56098
I20260709 12:15:46.984730 27400 raft_consensus.cc:606] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Received request to transfer leadership to TS fec66eb4aa88487eb30eb41db2fc28c5
I20260709 12:15:47.047410 27572 raft_consensus.cc:993] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9: : Instructing follower fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.047850 27572 raft_consensus.cc:1081] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Signalling peer fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.049532 27324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:48228
I20260709 12:15:47.050066 27324 raft_consensus.cc:493] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:47.050410 27324 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:47.056593 27324 raft_consensus.cc:515] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.058516 27324 leader_election.cc:290] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 4 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:47.059675 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:47.059530 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd7e88f8aed47f49bed724c4173b51a" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:47.060370 27400 raft_consensus.cc:3055] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:47.060472 27249 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:47.060726 27400 raft_consensus.cc:740] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:47.061685 27400 consensus_queue.cc:260] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.062986 27400 raft_consensus.cc:3060] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:47.066195 27249 raft_consensus.cc:2468] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 4.
I20260709 12:15:47.067533 27290 leader_election.cc:304] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:47.068552 27571 raft_consensus.cc:2804] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:47.069042 27573 raft_consensus.cc:993] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9: : Instructing follower fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.068969 27400 raft_consensus.cc:2468] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 4.
I20260709 12:15:47.069584 27572 raft_consensus.cc:1081] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Signalling peer fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.069834 27571 raft_consensus.cc:697] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:47.070961 27571 consensus_queue.cc:237] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.072036 27324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:48228
I20260709 12:15:47.072611 27324 raft_consensus.cc:493] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:47.072978 27324 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:47.079602 27324 raft_consensus.cc:515] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.079842 27146 catalog_manager.cc:5697] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 3 to 4, leader changed from ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3) to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 4 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:47.081290 27324 leader_election.cc:290] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 4 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:47.082196 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:47.082664 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:47.082863 27249 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:47.083310 27400 raft_consensus.cc:3055] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:47.083699 27400 raft_consensus.cc:740] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:47.084549 27400 consensus_queue.cc:260] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.086112 27400 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:47.088999 27249 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 4.
I20260709 12:15:47.090258 27290 leader_election.cc:304] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:47.091099 27571 raft_consensus.cc:2804] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:47.091560 27571 raft_consensus.cc:697] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:47.092350 27400 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 4.
I20260709 12:15:47.092507 27571 consensus_queue.cc:237] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.103044 27146 catalog_manager.cc:5697] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 3 to 4, leader changed from ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3) to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 4 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:47.282261 27543 raft_consensus.cc:993] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c: : Instructing follower fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.282814 27581 raft_consensus.cc:1081] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Signalling peer fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.284375 27324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:48238
I20260709 12:15:47.284901 27324 raft_consensus.cc:493] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:47.285208 27324 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:47.290638 27324 raft_consensus.cc:515] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.292600 27324 leader_election.cc:290] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 3 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:47.293213 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:47.293833 27249 raft_consensus.cc:3055] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:47.293761 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:47.294193 27249 raft_consensus.cc:740] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:47.294384 27400 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:47.294924 27249 consensus_queue.cc:260] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.295835 27249 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:47.299463 27400 raft_consensus.cc:2468] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 3.
I20260709 12:15:47.300448 27249 raft_consensus.cc:2468] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 3.
I20260709 12:15:47.300608 27290 leader_election.cc:304] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:47.301447 27571 raft_consensus.cc:2804] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:47.302033 27571 raft_consensus.cc:697] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:47.302814 27571 consensus_queue.cc:237] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.310086 27146 catalog_manager.cc:5697] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 2 to 3, leader changed from 05d1aaa742b64c7986de8e602f23333c (127.26.123.1) to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:47.364689 27543 raft_consensus.cc:993] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c: : Instructing follower fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.365147 27543 raft_consensus.cc:1081] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Signalling peer fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.368173 27326 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "954e28fb958e467c9d6be24151505db4"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:48238
I20260709 12:15:47.368901 27326 raft_consensus.cc:493] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:47.369488 27326 raft_consensus.cc:3060] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:47.370273 27583 raft_consensus.cc:993] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c: : Instructing follower fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.371088 27543 raft_consensus.cc:1081] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Signalling peer fec66eb4aa88487eb30eb41db2fc28c5 to start an election
I20260709 12:15:47.373785 27324 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
 from {username='slave'} at 127.0.0.1:48238
I20260709 12:15:47.374508 27324 raft_consensus.cc:493] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:47.375146 27324 raft_consensus.cc:3060] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:47.378993 27326 raft_consensus.cc:515] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.381776 27326 leader_election.cc:290] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:47.383658 27249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954e28fb958e467c9d6be24151505db4" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:47.384176 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954e28fb958e467c9d6be24151505db4" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:47.384657 27249 raft_consensus.cc:3055] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:47.384435 27324 raft_consensus.cc:515] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.385185 27249 raft_consensus.cc:740] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:47.385226 27400 raft_consensus.cc:3060] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:47.386585 27249 consensus_queue.cc:260] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.388121 27324 leader_election.cc:290] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:47.388371 27249 raft_consensus.cc:3060] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:47.390443 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:47.391510 27251 raft_consensus.cc:3055] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:47.392163 27251 raft_consensus.cc:740] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:47.395821 27399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:47.396944 27399 raft_consensus.cc:3060] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:47.399181 27400 raft_consensus.cc:2468] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:47.393718 27251 consensus_queue.cc:260] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.401355 27290 leader_election.cc:304] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:47.403090 27251 raft_consensus.cc:3060] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:47.403640 27574 raft_consensus.cc:2804] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:47.404619 27574 raft_consensus.cc:697] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:47.406973 27249 raft_consensus.cc:2468] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:47.406659 27574 consensus_queue.cc:237] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.413031 27399 raft_consensus.cc:2468] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:47.415532 27290 leader_election.cc:304] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea6d93ecf57a4f6194e0ffe32f963ce9, fec66eb4aa88487eb30eb41db2fc28c5; no voters: 
I20260709 12:15:47.416320 27251 raft_consensus.cc:2468] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec66eb4aa88487eb30eb41db2fc28c5 in term 2.
I20260709 12:15:47.417088 27571 raft_consensus.cc:2804] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:47.419710 27571 raft_consensus.cc:697] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Leader. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:47.421064 27571 consensus_queue.cc:237] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:47.423364 27146 catalog_manager.cc:5697] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 1 to 2, leader changed from 05d1aaa742b64c7986de8e602f23333c (127.26.123.1) to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:47.443495 27146 catalog_manager.cc:5697] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 reported cstate change: term changed from 1 to 2, leader changed from 05d1aaa742b64c7986de8e602f23333c (127.26.123.1) to fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:47.466478 27251 raft_consensus.cc:1275] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:47.468269 27571 consensus_queue.cc:1048] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:15:47.487988 27399 raft_consensus.cc:1275] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:47.492442 27585 consensus_queue.cc:1048] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:15:47.552831 27251 raft_consensus.cc:1275] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:47.554155 27571 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:47.564527 27399 raft_consensus.cc:1275] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:47.566166 27571 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:47.736253 27251 raft_consensus.cc:1275] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:47.737598 27571 consensus_queue.cc:1048] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:47.748562 27400 raft_consensus.cc:1275] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:47.751471 27574 consensus_queue.cc:1048] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:47.819170 27251 raft_consensus.cc:1275] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:47.820497 27574 consensus_queue.cc:1048] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:47.830408 27400 raft_consensus.cc:1275] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:47.831961 27585 consensus_queue.cc:1048] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:47.848417 27251 raft_consensus.cc:1275] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:47.849725 27585 consensus_queue.cc:1048] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:47.859982 27400 raft_consensus.cc:1275] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Refusing update from remote peer fec66eb4aa88487eb30eb41db2fc28c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:47.861617 27585 consensus_queue.cc:1048] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:50.030000 27116 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:15:50.030913 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:50.038313 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
mode: GRACEFUL
new_leader_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:48252
I20260709 12:15:50.039023 27324 raft_consensus.cc:606] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 LEADER]: Received request to transfer leadership to TS 05d1aaa742b64c7986de8e602f23333c
I20260709 12:15:50.042757 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
mode: GRACEFUL
new_leader_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:48252
I20260709 12:15:50.043282 27324 raft_consensus.cc:606] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Received request to transfer leadership to TS 05d1aaa742b64c7986de8e602f23333c
I20260709 12:15:50.046001 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
mode: GRACEFUL
new_leader_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:48252
I20260709 12:15:50.046504 27324 raft_consensus.cc:606] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Received request to transfer leadership to TS 05d1aaa742b64c7986de8e602f23333c
I20260709 12:15:50.050019 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
mode: GRACEFUL
new_leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:48252
I20260709 12:15:50.050554 27324 raft_consensus.cc:606] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 LEADER]: Received request to transfer leadership to TS ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:50.053424 27324 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "954e28fb958e467c9d6be24151505db4"
dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
mode: GRACEFUL
new_leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:48252
I20260709 12:15:50.054028 27324 raft_consensus.cc:606] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Received request to transfer leadership to TS ea6d93ecf57a4f6194e0ffe32f963ce9
I20260709 12:15:50.055501 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:50.056063 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:50.075376 27571 raft_consensus.cc:993] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5: : Instructing follower ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:50.076015 27600 raft_consensus.cc:1081] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Signalling peer ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:50.077980 27400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "954e28fb958e467c9d6be24151505db4"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:56090
I20260709 12:15:50.078722 27400 raft_consensus.cc:493] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:50.079078 27400 raft_consensus.cc:3060] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:50.084812 27400 raft_consensus.cc:515] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.087057 27400 leader_election.cc:290] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 3 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:50.088289 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954e28fb958e467c9d6be24151505db4" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:50.088390 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "954e28fb958e467c9d6be24151505db4" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:50.089058 27324 raft_consensus.cc:3055] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:50.089145 27251 raft_consensus.cc:3060] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:50.089345 27324 raft_consensus.cc:740] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:50.090122 27324 consensus_queue.cc:260] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.091109 27324 raft_consensus.cc:3060] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:50.094574 27251 raft_consensus.cc:2468] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 3.
I20260709 12:15:50.096163 27364 leader_election.cc:304] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:50.096716 27324 raft_consensus.cc:2468] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 3.
I20260709 12:15:50.097147 27572 raft_consensus.cc:2804] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:50.098323 27572 raft_consensus.cc:697] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Becoming Leader. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:50.099313 27572 consensus_queue.cc:237] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.107355 27145 catalog_manager.cc:5697] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 reported cstate change: term changed from 2 to 3, leader changed from fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2) to ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3). New cstate: current_term: 3 leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:50.176316 27571 raft_consensus.cc:993] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5: : Instructing follower 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:50.176887 27600 raft_consensus.cc:1081] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Signalling peer 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:50.178520 27251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:37050
I20260709 12:15:50.179010 27251 raft_consensus.cc:493] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:50.179291 27251 raft_consensus.cc:3060] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:50.184574 27251 raft_consensus.cc:515] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.186849 27251 leader_election.cc:290] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 3 election: Requested vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:50.188027 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:50.188081 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e212ead0131461882a03aed471bb0b7" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:50.188712 27324 raft_consensus.cc:3055] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:50.188781 27400 raft_consensus.cc:3060] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:50.188994 27324 raft_consensus.cc:740] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:50.189713 27324 consensus_queue.cc:260] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.190721 27324 raft_consensus.cc:3060] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:50.194124 27400 raft_consensus.cc:2468] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 3.
I20260709 12:15:50.195477 27215 leader_election.cc:304] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:50.195950 27324 raft_consensus.cc:2468] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 3.
I20260709 12:15:50.197490 27608 raft_consensus.cc:2804] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:50.198827 27608 raft_consensus.cc:697] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 3 LEADER]: Becoming Leader. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:50.199936 27608 consensus_queue.cc:237] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.208717 27145 catalog_manager.cc:5697] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c reported cstate change: term changed from 2 to 3, leader changed from fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2) to 05d1aaa742b64c7986de8e602f23333c (127.26.123.1). New cstate: current_term: 3 leader_uuid: "05d1aaa742b64c7986de8e602f23333c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:50.260210 27571 raft_consensus.cc:993] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5: : Instructing follower ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:50.260689 27600 raft_consensus.cc:1081] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 LEADER]: Signalling peer ea6d93ecf57a4f6194e0ffe32f963ce9 to start an election
I20260709 12:15:50.262395 27400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc"
dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
 from {username='slave'} at 127.0.0.1:56090
I20260709 12:15:50.262879 27400 raft_consensus.cc:493] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:50.263154 27400 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:50.268042 27400 raft_consensus.cc:515] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.269616 27400 leader_election.cc:290] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 4 election: Requested vote from peers 05d1aaa742b64c7986de8e602f23333c (127.26.123.1:46627), fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647)
I20260709 12:15:50.270862 27251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
I20260709 12:15:50.270874 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61c1a911eff14ce4ad34d1401e44e6bc" candidate_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:50.271526 27251 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:50.271592 27324 raft_consensus.cc:3055] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:50.271914 27324 raft_consensus.cc:740] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:50.272541 27324 consensus_queue.cc:260] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.273500 27324 raft_consensus.cc:3060] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:50.276620 27251 raft_consensus.cc:2468] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 4.
I20260709 12:15:50.277820 27364 leader_election.cc:304] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:50.278678 27324 raft_consensus.cc:2468] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea6d93ecf57a4f6194e0ffe32f963ce9 in term 4.
I20260709 12:15:50.278702 27572 raft_consensus.cc:2804] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:50.279295 27572 raft_consensus.cc:697] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 LEADER]: Becoming Leader. State: Replica: ea6d93ecf57a4f6194e0ffe32f963ce9, State: Running, Role: LEADER
I20260709 12:15:50.280220 27572 consensus_queue.cc:237] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.287282 27145 catalog_manager.cc:5697] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 reported cstate change: term changed from 3 to 4, leader changed from fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2) to ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3). New cstate: current_term: 4 leader_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:50.464457 27571 raft_consensus.cc:993] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5: : Instructing follower 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:50.464892 27600 raft_consensus.cc:1081] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 LEADER]: Signalling peer 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:50.466415 27251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:37050
I20260709 12:15:50.466895 27251 raft_consensus.cc:493] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:50.467188 27251 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:15:50.472410 27251 raft_consensus.cc:515] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.474136 27251 leader_election.cc:290] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 5 election: Requested vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:50.475072 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:50.475297 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d96a0ba5324569a4286fd261c93c8c" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:50.475674 27324 raft_consensus.cc:3055] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:15:50.475790 27400 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:15:50.475966 27324 raft_consensus.cc:740] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:50.476755 27324 consensus_queue.cc:260] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.478052 27324 raft_consensus.cc:3060] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:15:50.481366 27400 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 5.
I20260709 12:15:50.482556 27215 leader_election.cc:304] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:50.483274 27608 raft_consensus.cc:2804] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:15:50.483196 27324 raft_consensus.cc:2468] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 5.
I20260709 12:15:50.483834 27608 raft_consensus.cc:697] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 5 LEADER]: Becoming Leader. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:50.484786 27608 consensus_queue.cc:237] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.492550 27145 catalog_manager.cc:5697] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c reported cstate change: term changed from 4 to 5, leader changed from fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2) to 05d1aaa742b64c7986de8e602f23333c (127.26.123.1). New cstate: current_term: 5 leader_uuid: "05d1aaa742b64c7986de8e602f23333c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:50.510540 27571 raft_consensus.cc:993] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5: : Instructing follower 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:50.511014 27600 raft_consensus.cc:1081] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Signalling peer 05d1aaa742b64c7986de8e602f23333c to start an election
I20260709 12:15:50.512526 27251 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3"
dest_uuid: "05d1aaa742b64c7986de8e602f23333c"
 from {username='slave'} at 127.0.0.1:37050
I20260709 12:15:50.513000 27251 raft_consensus.cc:493] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:50.513296 27251 raft_consensus.cc:3060] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:50.518381 27251 raft_consensus.cc:515] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.519877 27251 leader_election.cc:290] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 3 election: Requested vote from peers fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2:37647), ea6d93ecf57a4f6194e0ffe32f963ce9 (127.26.123.3:34435)
I20260709 12:15:50.520905 27324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fec66eb4aa88487eb30eb41db2fc28c5"
I20260709 12:15:50.520920 27400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f632f7ba89546b9a966c6a2ce46f5f3" candidate_uuid: "05d1aaa742b64c7986de8e602f23333c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9"
I20260709 12:15:50.521461 27324 raft_consensus.cc:3055] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:50.521487 27400 raft_consensus.cc:3060] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:50.521840 27324 raft_consensus.cc:740] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fec66eb4aa88487eb30eb41db2fc28c5, State: Running, Role: LEADER
I20260709 12:15:50.522570 27324 consensus_queue.cc:260] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.523489 27324 raft_consensus.cc:3060] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:50.526386 27400 raft_consensus.cc:2468] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 3.
I20260709 12:15:50.527397 27215 leader_election.cc:304] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d1aaa742b64c7986de8e602f23333c, ea6d93ecf57a4f6194e0ffe32f963ce9; no voters: 
I20260709 12:15:50.528188 27608 raft_consensus.cc:2804] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:50.528425 27324 raft_consensus.cc:2468] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d1aaa742b64c7986de8e602f23333c in term 3.
I20260709 12:15:50.528607 27608 raft_consensus.cc:697] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 3 LEADER]: Becoming Leader. State: Replica: 05d1aaa742b64c7986de8e602f23333c, State: Running, Role: LEADER
I20260709 12:15:50.529584 27608 consensus_queue.cc:237] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } }
I20260709 12:15:50.537057 27145 catalog_manager.cc:5697] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c reported cstate change: term changed from 2 to 3, leader changed from fec66eb4aa88487eb30eb41db2fc28c5 (127.26.123.2) to 05d1aaa742b64c7986de8e602f23333c (127.26.123.1). New cstate: current_term: 3 leader_uuid: "05d1aaa742b64c7986de8e602f23333c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:50.582275 27251 raft_consensus.cc:1275] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.583575 27572 consensus_queue.cc:1048] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:50.594219 27324 raft_consensus.cc:1275] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.596133 27602 consensus_queue.cc:1048] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:50.725013 27324 raft_consensus.cc:1275] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.726245 27608 consensus_queue.cc:1048] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:50.738597 27400 raft_consensus.cc:1275] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.740145 27620 consensus_queue.cc:1048] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:50.782076 27251 raft_consensus.cc:1275] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:50.783370 27602 consensus_queue.cc:1048] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05d1aaa742b64c7986de8e602f23333c" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:50.793143 27324 raft_consensus.cc:1275] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Refusing update from remote peer ea6d93ecf57a4f6194e0ffe32f963ce9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:15:50.795331 27572 consensus_queue.cc:1048] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:50.892652 27324 raft_consensus.cc:1275] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 5 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:15:50.894410 27620 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:15:50.905063 27400 raft_consensus.cc:1275] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 5 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:15:50.907102 27608 consensus_queue.cc:1048] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:15:50.937037 27400 raft_consensus.cc:1275] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.938761 27620 consensus_queue.cc:1048] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea6d93ecf57a4f6194e0ffe32f963ce9" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:50.948866 27324 raft_consensus.cc:1275] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Refusing update from remote peer 05d1aaa742b64c7986de8e602f23333c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.951494 27608 consensus_queue.cc:1048] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec66eb4aa88487eb30eb41db2fc28c5" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:51.057412 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:51.063484 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:51.063973 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:52.065106 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:52.070631 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:52.071173 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:53.072428 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:53.077653 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:53.078177 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:54.079377 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:54.086267 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:54.086709 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:55.087868 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:55.093716 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:55.094211 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:56.095323 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:56.100607 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:56.101022 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:57.102355 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:57.108006 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:57.108428 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:58.109548 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:58.115221 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:58.115664 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:59.117012 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:59.122912 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:59.123312 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:00.156522 27116 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:16:00.157176 27116 tablet_server.cc:179] TabletServer@127.26.123.1:0 shutting down...
I20260709 12:16:00.184011 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:00.184793 27116 tablet_replica.cc:333] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c: stopping tablet replica
I20260709 12:16:00.185632 27116 raft_consensus.cc:2243] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.186363 27116 raft_consensus.cc:2272] T a6b134736e8042e49a6f57e52e2eee59 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.188922 27116 tablet_replica.cc:333] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c: stopping tablet replica
I20260709 12:16:00.189488 27116 raft_consensus.cc:2243] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.190384 27116 raft_consensus.cc:2272] T 5f632f7ba89546b9a966c6a2ce46f5f3 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.192313 27116 tablet_replica.cc:333] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c: stopping tablet replica
I20260709 12:16:00.192836 27116 raft_consensus.cc:2243] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.193710 27116 raft_consensus.cc:2272] T 20d96a0ba5324569a4286fd261c93c8c P 05d1aaa742b64c7986de8e602f23333c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.195703 27116 tablet_replica.cc:333] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c: stopping tablet replica
I20260709 12:16:00.196250 27116 raft_consensus.cc:2243] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.197093 27116 raft_consensus.cc:2272] T 3e212ead0131461882a03aed471bb0b7 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.198982 27116 tablet_replica.cc:333] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c: stopping tablet replica
I20260709 12:16:00.199534 27116 raft_consensus.cc:2243] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.200052 27116 raft_consensus.cc:2272] T 61c1a911eff14ce4ad34d1401e44e6bc P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.201980 27116 tablet_replica.cc:333] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c: stopping tablet replica
I20260709 12:16:00.202575 27116 raft_consensus.cc:2243] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.202998 27116 raft_consensus.cc:2272] T 42d75d4decde4719873000d6a56e4992 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.204770 27116 tablet_replica.cc:333] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c: stopping tablet replica
I20260709 12:16:00.205299 27116 raft_consensus.cc:2243] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.205763 27116 raft_consensus.cc:2272] T 6b251acd25d84b029195fd87c744df81 P 05d1aaa742b64c7986de8e602f23333c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.207557 27116 tablet_replica.cc:333] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c: stopping tablet replica
I20260709 12:16:00.208043 27116 raft_consensus.cc:2243] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.208501 27116 raft_consensus.cc:2272] T 954e28fb958e467c9d6be24151505db4 P 05d1aaa742b64c7986de8e602f23333c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.210371 27116 tablet_replica.cc:333] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c: stopping tablet replica
I20260709 12:16:00.210865 27116 raft_consensus.cc:2243] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.211340 27116 raft_consensus.cc:2272] T 6dd7e88f8aed47f49bed724c4173b51a P 05d1aaa742b64c7986de8e602f23333c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.299939 27116 tablet_server.cc:196] TabletServer@127.26.123.1:0 shutdown complete.
I20260709 12:16:00.321642 27116 tablet_server.cc:179] TabletServer@127.26.123.2:0 shutting down...
I20260709 12:16:00.346856 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:00.347574 27116 tablet_replica.cc:333] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5: stopping tablet replica
I20260709 12:16:00.348201 27116 raft_consensus.cc:2243] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.349090 27116 raft_consensus.cc:2272] T a6b134736e8042e49a6f57e52e2eee59 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.351178 27116 tablet_replica.cc:333] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5: stopping tablet replica
I20260709 12:16:00.351809 27116 raft_consensus.cc:2243] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.352241 27116 raft_consensus.cc:2272] T 5f632f7ba89546b9a966c6a2ce46f5f3 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.354236 27116 tablet_replica.cc:333] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5: stopping tablet replica
I20260709 12:16:00.354801 27116 raft_consensus.cc:2243] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.355266 27116 raft_consensus.cc:2272] T 3e212ead0131461882a03aed471bb0b7 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.357149 27116 tablet_replica.cc:333] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5: stopping tablet replica
I20260709 12:16:00.357692 27116 raft_consensus.cc:2243] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.358209 27116 raft_consensus.cc:2272] T 20d96a0ba5324569a4286fd261c93c8c P fec66eb4aa88487eb30eb41db2fc28c5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.359887 27116 tablet_replica.cc:333] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5: stopping tablet replica
I20260709 12:16:00.360419 27116 raft_consensus.cc:2243] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.360836 27116 raft_consensus.cc:2272] T 42d75d4decde4719873000d6a56e4992 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.362577 27116 tablet_replica.cc:333] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5: stopping tablet replica
I20260709 12:16:00.363054 27116 raft_consensus.cc:2243] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.363461 27116 raft_consensus.cc:2272] T 61c1a911eff14ce4ad34d1401e44e6bc P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.365096 27116 tablet_replica.cc:333] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5: stopping tablet replica
I20260709 12:16:00.365583 27116 raft_consensus.cc:2243] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.366021 27116 raft_consensus.cc:2272] T 954e28fb958e467c9d6be24151505db4 P fec66eb4aa88487eb30eb41db2fc28c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.368064 27116 tablet_replica.cc:333] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5: stopping tablet replica
I20260709 12:16:00.368536 27116 raft_consensus.cc:2243] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.369325 27116 raft_consensus.cc:2272] T 6b251acd25d84b029195fd87c744df81 P fec66eb4aa88487eb30eb41db2fc28c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.371052 27116 tablet_replica.cc:333] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5: stopping tablet replica
I20260709 12:16:00.371510 27116 raft_consensus.cc:2243] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.372310 27116 raft_consensus.cc:2272] T 6dd7e88f8aed47f49bed724c4173b51a P fec66eb4aa88487eb30eb41db2fc28c5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.402539 27116 tablet_server.cc:196] TabletServer@127.26.123.2:0 shutdown complete.
I20260709 12:16:00.421782 27116 tablet_server.cc:179] TabletServer@127.26.123.3:0 shutting down...
I20260709 12:16:00.446986 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:00.447746 27116 tablet_replica.cc:333] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9: stopping tablet replica
I20260709 12:16:00.448362 27116 raft_consensus.cc:2243] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.448871 27116 raft_consensus.cc:2272] T 6dd7e88f8aed47f49bed724c4173b51a P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.450994 27116 tablet_replica.cc:333] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9: stopping tablet replica
I20260709 12:16:00.451543 27116 raft_consensus.cc:2243] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.452059 27116 raft_consensus.cc:2272] T 3e212ead0131461882a03aed471bb0b7 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.454064 27116 tablet_replica.cc:333] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9: stopping tablet replica
I20260709 12:16:00.454624 27116 raft_consensus.cc:2243] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.455106 27116 raft_consensus.cc:2272] T 20d96a0ba5324569a4286fd261c93c8c P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.457033 27116 tablet_replica.cc:333] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9: stopping tablet replica
I20260709 12:16:00.457630 27116 raft_consensus.cc:2243] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.458587 27116 raft_consensus.cc:2272] T 954e28fb958e467c9d6be24151505db4 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.460477 27116 tablet_replica.cc:333] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9: stopping tablet replica
I20260709 12:16:00.461022 27116 raft_consensus.cc:2243] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.462021 27116 raft_consensus.cc:2272] T 61c1a911eff14ce4ad34d1401e44e6bc P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.463894 27116 tablet_replica.cc:333] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9: stopping tablet replica
I20260709 12:16:00.464391 27116 raft_consensus.cc:2243] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.464826 27116 raft_consensus.cc:2272] T 6b251acd25d84b029195fd87c744df81 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.466743 27116 tablet_replica.cc:333] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9: stopping tablet replica
I20260709 12:16:00.467228 27116 raft_consensus.cc:2243] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.468042 27116 raft_consensus.cc:2272] T 42d75d4decde4719873000d6a56e4992 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.469848 27116 tablet_replica.cc:333] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9: stopping tablet replica
I20260709 12:16:00.470331 27116 raft_consensus.cc:2243] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.470861 27116 raft_consensus.cc:2272] T 5f632f7ba89546b9a966c6a2ce46f5f3 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.472586 27116 tablet_replica.cc:333] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9: stopping tablet replica
I20260709 12:16:00.473042 27116 raft_consensus.cc:2243] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.473434 27116 raft_consensus.cc:2272] T a6b134736e8042e49a6f57e52e2eee59 P ea6d93ecf57a4f6194e0ffe32f963ce9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.502043 27116 tablet_server.cc:196] TabletServer@127.26.123.3:0 shutdown complete.
I20260709 12:16:00.521761 27116 master.cc:562] Master@127.26.123.62:44811 shutting down...
I20260709 12:16:00.755628 27116 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:00.756354 27116 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.756719 27116 tablet_replica.cc:333] T 00000000000000000000000000000000 P 967bd590434a476eb193ea471ecd8707: stopping tablet replica
I20260709 12:16:00.776551 27116 master.cc:584] Master@127.26.123.62:44811 shutdown complete.
05d1aaa742b64c7986de8e602f23333c  4
ea6d93ecf57a4f6194e0ffe32f963ce9  1
fec66eb4aa88487eb30eb41db2fc28c5  4
05d1aaa742b64c7986de8e602f23333c  3
ea6d93ecf57a4f6194e0ffe32f963ce9  3
fec66eb4aa88487eb30eb41db2fc28c5  3
05d1aaa742b64c7986de8e602f23333c  0
ea6d93ecf57a4f6194e0ffe32f963ce9  1
fec66eb4aa88487eb30eb41db2fc28c5  8
05d1aaa742b64c7986de8e602f23333c  3
ea6d93ecf57a4f6194e0ffe32f963ce9  3
fec66eb4aa88487eb30eb41db2fc28c5  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31320 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:16:00.820955 27116 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.123.62:45905
I20260709 12:16:00.822201 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:00.827906 27637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:00.827907 27638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:00.829272 27640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:00.830641 27116 server_base.cc:1061] running on GCE node
I20260709 12:16:00.831611 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:00.831823 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:00.831986 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599360831962 us; error 0 us; skew 500 ppm
I20260709 12:16:00.832516 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:00.835366 27116 webserver.cc:533] Webserver started at http://127.26.123.62:46201/ using document root <none> and password file <none>
I20260709 12:16:00.835882 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:00.836056 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:00.836334 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:00.837541 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/master-0-root/instance:
uuid: "89fd35d3a02546d2bb0785515f93eb89"
format_stamp: "Formatted at 2026-07-09 12:16:00 on dist-test-slave-9q7n"
I20260709 12:16:00.842123 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:16:00.845506 27645 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:00.846261 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:16:00.846526 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/master-0-root
uuid: "89fd35d3a02546d2bb0785515f93eb89"
format_stamp: "Formatted at 2026-07-09 12:16:00 on dist-test-slave-9q7n"
I20260709 12:16:00.846807 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:00.861507 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:00.862916 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:00.906978 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.62:45905
I20260709 12:16:00.907075 27696 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.62:45905 every 8 connection(s)
I20260709 12:16:00.911188 27697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:00.921710 27697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89: Bootstrap starting.
I20260709 12:16:00.926957 27697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.931568 27697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89: No bootstrap required, opened a new log
I20260709 12:16:00.934098 27697 raft_consensus.cc:359] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fd35d3a02546d2bb0785515f93eb89" member_type: VOTER }
I20260709 12:16:00.934583 27697 raft_consensus.cc:385] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.934826 27697 raft_consensus.cc:740] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89fd35d3a02546d2bb0785515f93eb89, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.935396 27697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fd35d3a02546d2bb0785515f93eb89" member_type: VOTER }
I20260709 12:16:00.935868 27697 raft_consensus.cc:399] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:00.936088 27697 raft_consensus.cc:493] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:00.936357 27697 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.940914 27697 raft_consensus.cc:515] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fd35d3a02546d2bb0785515f93eb89" member_type: VOTER }
I20260709 12:16:00.941504 27697 leader_election.cc:304] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89fd35d3a02546d2bb0785515f93eb89; no voters: 
I20260709 12:16:00.942773 27697 leader_election.cc:290] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:00.943208 27700 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:00.944694 27700 raft_consensus.cc:697] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 1 LEADER]: Becoming Leader. State: Replica: 89fd35d3a02546d2bb0785515f93eb89, State: Running, Role: LEADER
I20260709 12:16:00.945377 27700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fd35d3a02546d2bb0785515f93eb89" member_type: VOTER }
I20260709 12:16:00.945919 27697 sys_catalog.cc:565] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:00.950318 27702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89fd35d3a02546d2bb0785515f93eb89. Latest consensus state: current_term: 1 leader_uuid: "89fd35d3a02546d2bb0785515f93eb89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fd35d3a02546d2bb0785515f93eb89" member_type: VOTER } }
I20260709 12:16:00.950163 27701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89fd35d3a02546d2bb0785515f93eb89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fd35d3a02546d2bb0785515f93eb89" member_type: VOTER } }
I20260709 12:16:00.950927 27702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:00.951051 27701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:00.956375 27707 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:00.962076 27707 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:00.964867 27116 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:00.971580 27707 catalog_manager.cc:1383] Generated new cluster ID: 7a777a14c9f94ca78a671ccd3ed718ea
I20260709 12:16:00.971894 27707 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:01.007905 27707 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:01.009330 27707 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:01.021590 27707 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89: Generated new TSK 0
I20260709 12:16:01.022320 27707 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:01.031661 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:01.038408 27718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:01.038952 27719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:01.039970 27721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:01.041222 27116 server_base.cc:1061] running on GCE node
I20260709 12:16:01.042253 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:01.042454 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:01.042582 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599361042570 us; error 0 us; skew 500 ppm
I20260709 12:16:01.043092 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:01.045437 27116 webserver.cc:533] Webserver started at http://127.26.123.1:38747/ using document root <none> and password file <none>
I20260709 12:16:01.045979 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:01.046149 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:01.046380 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:01.047508 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-0-root/instance:
uuid: "88925d3120dd465bb8d39599142710b9"
format_stamp: "Formatted at 2026-07-09 12:16:01 on dist-test-slave-9q7n"
I20260709 12:16:01.052107 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:01.055444 27726 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:01.056156 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:16:01.056453 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "88925d3120dd465bb8d39599142710b9"
format_stamp: "Formatted at 2026-07-09 12:16:01 on dist-test-slave-9q7n"
I20260709 12:16:01.056725 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:01.070768 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:01.072144 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:01.073722 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:01.076145 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:01.076359 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:01.076598 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:01.076766 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:01.120636 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:43347
I20260709 12:16:01.120754 27788 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:43347 every 8 connection(s)
I20260709 12:16:01.125924 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:01.133210 27793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:01.133916 27794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:01.136929 27796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:01.139452 27116 server_base.cc:1061] running on GCE node
I20260709 12:16:01.141134 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:01.141412 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:01.141577 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599361141560 us; error 0 us; skew 500 ppm
I20260709 12:16:01.142289 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:01.144659 27789 heartbeater.cc:344] Connected to a master server at 127.26.123.62:45905
I20260709 12:16:01.145036 27789 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:01.145637 27116 webserver.cc:533] Webserver started at http://127.26.123.2:44347/ using document root <none> and password file <none>
I20260709 12:16:01.145825 27789 heartbeater.cc:507] Master 127.26.123.62:45905 requested a full tablet report, sending...
I20260709 12:16:01.146275 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:01.146485 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:01.146790 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:01.148239 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-1-root/instance:
uuid: "1371160c6a5049e3b8231f1deb73844b"
format_stamp: "Formatted at 2026-07-09 12:16:01 on dist-test-slave-9q7n"
I20260709 12:16:01.148460 27662 ts_manager.cc:194] Registered new tserver with Master: 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347)
I20260709 12:16:01.150611 27662 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42864
I20260709 12:16:01.153910 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:16:01.157717 27801 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:01.158795 27116 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:16:01.159122 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-1-root
uuid: "1371160c6a5049e3b8231f1deb73844b"
format_stamp: "Formatted at 2026-07-09 12:16:01 on dist-test-slave-9q7n"
I20260709 12:16:01.159404 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:01.179594 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:01.180965 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:01.182705 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:01.185177 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:01.185396 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:01.185644 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:01.185828 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:16:01.231204 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.2:42387
I20260709 12:16:01.231340 27863 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.2:42387 every 8 connection(s)
I20260709 12:16:01.236284 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:01.244577 27867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:01.245936 27868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:01.249990 27864 heartbeater.cc:344] Connected to a master server at 127.26.123.62:45905
I20260709 12:16:01.250300 27116 server_base.cc:1061] running on GCE node
I20260709 12:16:01.250507 27864 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:01.251562 27864 heartbeater.cc:507] Master 127.26.123.62:45905 requested a full tablet report, sending...
W20260709 12:16:01.252018 27870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:01.253060 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:01.253304 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:01.253525 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599361253500 us; error 0 us; skew 500 ppm
I20260709 12:16:01.254149 27662 ts_manager.cc:194] Registered new tserver with Master: 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:01.254395 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:01.256156 27662 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42866
I20260709 12:16:01.257647 27116 webserver.cc:533] Webserver started at http://127.26.123.3:37267/ using document root <none> and password file <none>
I20260709 12:16:01.258421 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:01.258620 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:01.258955 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:01.260267 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-2-root/instance:
uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
format_stamp: "Formatted at 2026-07-09 12:16:01 on dist-test-slave-9q7n"
I20260709 12:16:01.265478 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:16:01.268983 27875 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:01.269806 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:16:01.270114 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-2-root
uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
format_stamp: "Formatted at 2026-07-09 12:16:01 on dist-test-slave-9q7n"
I20260709 12:16:01.270424 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599329322122-27116-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:01.284631 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:01.286197 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:01.287842 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:01.290366 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:01.290575 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:01.290838 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:01.290997 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:01.337502 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.3:36647
I20260709 12:16:01.337615 27937 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.3:36647 every 8 connection(s)
I20260709 12:16:01.352674 27938 heartbeater.cc:344] Connected to a master server at 127.26.123.62:45905
I20260709 12:16:01.353061 27938 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:01.353901 27938 heartbeater.cc:507] Master 127.26.123.62:45905 requested a full tablet report, sending...
I20260709 12:16:01.355883 27662 ts_manager.cc:194] Registered new tserver with Master: 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:01.356910 27116 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015994095s
I20260709 12:16:01.357501 27662 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42872
I20260709 12:16:01.384375 27662 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42882:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:01.387475 27662 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:01.458400 27903 tablet_service.cc:1511] Processing CreateTablet for tablet c7f8ca86217b481caf7179dd6b1b2c8d (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:01.460177 27903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f8ca86217b481caf7179dd6b1b2c8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.463822 27902 tablet_service.cc:1511] Processing CreateTablet for tablet ff1f2bac20394d3ea0849035ed450586 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:01.465495 27902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff1f2bac20394d3ea0849035ed450586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.467483 27901 tablet_service.cc:1511] Processing CreateTablet for tablet b99723e1f56a47cd9762f41e71061c4c (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:01.469972 27901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99723e1f56a47cd9762f41e71061c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.472918 27900 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce6d589c5d845c0adbbbad521d9733a (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:01.474587 27900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce6d589c5d845c0adbbbad521d9733a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.479730 27899 tablet_service.cc:1511] Processing CreateTablet for tablet 124c98e6d8f14ad4a19fb65b95779dc7 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:01.481418 27899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 124c98e6d8f14ad4a19fb65b95779dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.487758 27898 tablet_service.cc:1511] Processing CreateTablet for tablet 93ab913080d2442e9f14f6c35f480620 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:01.489616 27898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ab913080d2442e9f14f6c35f480620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.495615 27897 tablet_service.cc:1511] Processing CreateTablet for tablet e166fabf328942638baed5d4c053db85 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:01.497133 27897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e166fabf328942638baed5d4c053db85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.500785 27829 tablet_service.cc:1511] Processing CreateTablet for tablet c7f8ca86217b481caf7179dd6b1b2c8d (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:01.502327 27829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f8ca86217b481caf7179dd6b1b2c8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.508327 27896 tablet_service.cc:1511] Processing CreateTablet for tablet a2d32c67cfb44ffdb4ae9774e8a4d71b (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:01.509341 27828 tablet_service.cc:1511] Processing CreateTablet for tablet ff1f2bac20394d3ea0849035ed450586 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:01.510020 27896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2d32c67cfb44ffdb4ae9774e8a4d71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.510170 27952 tablet_bootstrap.cc:492] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:01.510854 27828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff1f2bac20394d3ea0849035ed450586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.515388 27827 tablet_service.cc:1511] Processing CreateTablet for tablet b99723e1f56a47cd9762f41e71061c4c (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:01.515762 27826 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce6d589c5d845c0adbbbad521d9733a (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:01.520789 27952 tablet_bootstrap.cc:654] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.528304 27826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce6d589c5d845c0adbbbad521d9733a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.535104 27825 tablet_service.cc:1511] Processing CreateTablet for tablet 124c98e6d8f14ad4a19fb65b95779dc7 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:01.536586 27825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 124c98e6d8f14ad4a19fb65b95779dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.538640 27901 tablet_service.cc:1511] Processing CreateTablet for tablet 5bf0f32ff2c6416e868adb7ddfe6dfa3 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:01.544451 27823 tablet_service.cc:1511] Processing CreateTablet for tablet e166fabf328942638baed5d4c053db85 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:01.547114 27824 tablet_service.cc:1511] Processing CreateTablet for tablet 93ab913080d2442e9f14f6c35f480620 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:01.548512 27821 tablet_service.cc:1511] Processing CreateTablet for tablet a2d32c67cfb44ffdb4ae9774e8a4d71b (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:01.551011 27820 tablet_service.cc:1511] Processing CreateTablet for tablet 5bf0f32ff2c6416e868adb7ddfe6dfa3 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:01.547017 27822 tablet_service.cc:1511] Processing CreateTablet for tablet 6acb9f89fc3646adb2ab1ec2460465ed (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:01.560710 27823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e166fabf328942638baed5d4c053db85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.561488 27821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2d32c67cfb44ffdb4ae9774e8a4d71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.564633 27827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99723e1f56a47cd9762f41e71061c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.566107 27822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6acb9f89fc3646adb2ab1ec2460465ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.570396 27820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bf0f32ff2c6416e868adb7ddfe6dfa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.574203 27824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ab913080d2442e9f14f6c35f480620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.586138 27901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bf0f32ff2c6416e868adb7ddfe6dfa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.558579 27903 tablet_service.cc:1511] Processing CreateTablet for tablet 6acb9f89fc3646adb2ab1ec2460465ed (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:01.594587 27903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6acb9f89fc3646adb2ab1ec2460465ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.595116 27754 tablet_service.cc:1511] Processing CreateTablet for tablet c7f8ca86217b481caf7179dd6b1b2c8d (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:01.597534 27753 tablet_service.cc:1511] Processing CreateTablet for tablet ff1f2bac20394d3ea0849035ed450586 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:01.598280 27754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f8ca86217b481caf7179dd6b1b2c8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.599032 27753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff1f2bac20394d3ea0849035ed450586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.604235 27752 tablet_service.cc:1511] Processing CreateTablet for tablet b99723e1f56a47cd9762f41e71061c4c (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:01.605818 27752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b99723e1f56a47cd9762f41e71061c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.597013 27952 tablet_bootstrap.cc:492] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:01.612708 27952 ts_tablet_manager.cc:1403] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.103s	user 0.023s	sys 0.040s
I20260709 12:16:01.619213 27751 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce6d589c5d845c0adbbbad521d9733a (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:01.620720 27751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce6d589c5d845c0adbbbad521d9733a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.621218 27952 raft_consensus.cc:359] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.622066 27952 raft_consensus.cc:385] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.622385 27952 raft_consensus.cc:740] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.623229 27952 consensus_queue.cc:260] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.631678 27750 tablet_service.cc:1511] Processing CreateTablet for tablet 124c98e6d8f14ad4a19fb65b95779dc7 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:01.632249 27748 tablet_service.cc:1511] Processing CreateTablet for tablet e166fabf328942638baed5d4c053db85 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:01.633265 27750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 124c98e6d8f14ad4a19fb65b95779dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.634642 27747 tablet_service.cc:1511] Processing CreateTablet for tablet 6acb9f89fc3646adb2ab1ec2460465ed (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:01.636042 27747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6acb9f89fc3646adb2ab1ec2460465ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.641446 27955 tablet_bootstrap.cc:492] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:01.652477 27748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e166fabf328942638baed5d4c053db85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.653343 27746 tablet_service.cc:1511] Processing CreateTablet for tablet a2d32c67cfb44ffdb4ae9774e8a4d71b (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:01.654840 27746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2d32c67cfb44ffdb4ae9774e8a4d71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.658888 27749 tablet_service.cc:1511] Processing CreateTablet for tablet 93ab913080d2442e9f14f6c35f480620 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:01.660364 27749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ab913080d2442e9f14f6c35f480620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.641592 27745 tablet_service.cc:1511] Processing CreateTablet for tablet 5bf0f32ff2c6416e868adb7ddfe6dfa3 (DEFAULT_TABLE table=test-workload [id=36599ce202c34c8d90d6f8161148665d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:01.665127 27745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bf0f32ff2c6416e868adb7ddfe6dfa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:01.669804 27954 tablet_bootstrap.cc:492] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:01.707494 27938 heartbeater.cc:499] Master 127.26.123.62:45905 was elected leader, sending a full tablet report...
I20260709 12:16:01.708690 27955 tablet_bootstrap.cc:654] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.713042 27952 ts_tablet_manager.cc:1434] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.096s	user 0.002s	sys 0.007s
I20260709 12:16:01.714203 27952 tablet_bootstrap.cc:492] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:01.733335 27954 tablet_bootstrap.cc:654] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.742050 27955 tablet_bootstrap.cc:492] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:01.742672 27955 ts_tablet_manager.cc:1403] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.102s	user 0.015s	sys 0.000s
I20260709 12:16:01.746116 27955 raft_consensus.cc:359] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:01.746924 27955 raft_consensus.cc:385] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.747309 27955 raft_consensus.cc:740] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.748111 27955 consensus_queue.cc:260] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:01.748543 27954 tablet_bootstrap.cc:492] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:01.749701 27954 ts_tablet_manager.cc:1403] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.080s	user 0.016s	sys 0.008s
I20260709 12:16:01.750226 27789 heartbeater.cc:499] Master 127.26.123.62:45905 was elected leader, sending a full tablet report...
I20260709 12:16:01.750896 27952 tablet_bootstrap.cc:654] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.752863 27954 raft_consensus.cc:359] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.753631 27954 raft_consensus.cc:385] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.754005 27954 raft_consensus.cc:740] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.754994 27955 ts_tablet_manager.cc:1434] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:16:01.754827 27954 consensus_queue.cc:260] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.756165 27955 tablet_bootstrap.cc:492] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:01.764153 27955 tablet_bootstrap.cc:654] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.766727 27952 tablet_bootstrap.cc:492] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:01.769116 27952 ts_tablet_manager.cc:1403] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.007s
I20260709 12:16:01.774149 27954 ts_tablet_manager.cc:1434] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.016s
I20260709 12:16:01.773519 27952 raft_consensus.cc:359] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:01.775122 27864 heartbeater.cc:499] Master 127.26.123.62:45905 was elected leader, sending a full tablet report...
I20260709 12:16:01.775190 27952 raft_consensus.cc:385] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.775795 27952 raft_consensus.cc:740] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.777098 27955 tablet_bootstrap.cc:492] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:01.776767 27952 consensus_queue.cc:260] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:01.777630 27955 ts_tablet_manager.cc:1403] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.008s
I20260709 12:16:01.779628 27954 tablet_bootstrap.cc:492] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:01.780434 27952 ts_tablet_manager.cc:1434] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:16:01.780880 27955 raft_consensus.cc:359] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.781433 27952 tablet_bootstrap.cc:492] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:01.781625 27955 raft_consensus.cc:385] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.782023 27955 raft_consensus.cc:740] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.782785 27955 consensus_queue.cc:260] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.785421 27955 ts_tablet_manager.cc:1434] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:01.786454 27955 tablet_bootstrap.cc:492] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:01.788803 27952 tablet_bootstrap.cc:654] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.790043 27954 tablet_bootstrap.cc:654] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.793516 27955 tablet_bootstrap.cc:654] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.801136 27952 tablet_bootstrap.cc:492] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:01.801812 27952 ts_tablet_manager.cc:1403] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.005s
I20260709 12:16:01.805349 27952 raft_consensus.cc:359] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.806937 27952 raft_consensus.cc:385] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.807260 27952 raft_consensus.cc:740] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.808056 27952 consensus_queue.cc:260] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.809032 27954 tablet_bootstrap.cc:492] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:01.809590 27954 ts_tablet_manager.cc:1403] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.017s
I20260709 12:16:01.810446 27952 ts_tablet_manager.cc:1434] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:01.811524 27952 tablet_bootstrap.cc:492] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:01.812789 27954 raft_consensus.cc:359] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:01.813560 27954 raft_consensus.cc:385] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.813898 27954 raft_consensus.cc:740] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.814889 27954 consensus_queue.cc:260] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:01.817487 27954 ts_tablet_manager.cc:1434] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:01.818573 27954 tablet_bootstrap.cc:492] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:01.820252 27952 tablet_bootstrap.cc:654] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.820698 27955 tablet_bootstrap.cc:492] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:01.821278 27955 ts_tablet_manager.cc:1403] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.035s	user 0.030s	sys 0.004s
I20260709 12:16:01.824666 27955 raft_consensus.cc:359] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.825598 27955 raft_consensus.cc:385] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.826028 27955 raft_consensus.cc:740] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.826097 27954 tablet_bootstrap.cc:654] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.827260 27955 consensus_queue.cc:260] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.830235 27955 ts_tablet_manager.cc:1434] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:01.831473 27955 tablet_bootstrap.cc:492] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:01.832062 27956 raft_consensus.cc:493] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.832692 27956 raft_consensus.cc:515] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.834956 27952 tablet_bootstrap.cc:492] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:01.835578 27952 ts_tablet_manager.cc:1403] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 12:16:01.839025 27955 tablet_bootstrap.cc:654] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.839072 27956 leader_election.cc:290] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:01.850275 27952 raft_consensus.cc:359] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.851141 27952 raft_consensus.cc:385] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.851470 27952 raft_consensus.cc:740] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.853601 27952 consensus_queue.cc:260] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.855204 27954 tablet_bootstrap.cc:492] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:01.855849 27954 ts_tablet_manager.cc:1403] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.009s
I20260709 12:16:01.859277 27952 ts_tablet_manager.cc:1434] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:16:01.860473 27952 tablet_bootstrap.cc:492] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:01.864032 27954 raft_consensus.cc:359] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.864931 27954 raft_consensus.cc:385] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.865406 27954 raft_consensus.cc:740] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.870350 27954 consensus_queue.cc:260] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.871876 27952 tablet_bootstrap.cc:654] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.880689 27956 raft_consensus.cc:493] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.881297 27956 raft_consensus.cc:515] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.883162 27954 ts_tablet_manager.cc:1434] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.003s
I20260709 12:16:01.883445 27956 leader_election.cc:290] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:01.884404 27954 tablet_bootstrap.cc:492] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:01.892592 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
I20260709 12:16:01.895210 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
W20260709 12:16:01.897382 27878 leader_election.cc:343] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:01.898273 27878 leader_election.cc:343] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.898311 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:01.900004 27763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
W20260709 12:16:01.900005 27878 leader_election.cc:343] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.900848 27763 raft_consensus.cc:2468] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 0.
I20260709 12:16:01.901051 27954 tablet_bootstrap.cc:654] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.901396 27878 leader_election.cc:304] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9
I20260709 12:16:01.902354 27956 raft_consensus.cc:2749] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:01.902827 27878 leader_election.cc:304] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 1371160c6a5049e3b8231f1deb73844b
I20260709 12:16:01.903642 27956 raft_consensus.cc:2804] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:01.903980 27955 tablet_bootstrap.cc:492] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:01.904224 27952 tablet_bootstrap.cc:492] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:01.903998 27956 raft_consensus.cc:493] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:01.904672 27955 ts_tablet_manager.cc:1403] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.073s	user 0.024s	sys 0.043s
I20260709 12:16:01.904767 27952 ts_tablet_manager.cc:1403] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.045s	user 0.021s	sys 0.010s
I20260709 12:16:01.904664 27956 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.908098 27955 raft_consensus.cc:359] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.908265 27952 raft_consensus.cc:359] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.908887 27955 raft_consensus.cc:385] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.908949 27952 raft_consensus.cc:385] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.909297 27955 raft_consensus.cc:740] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.909334 27952 raft_consensus.cc:740] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.910338 27952 consensus_queue.cc:260] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.910336 27955 consensus_queue.cc:260] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.912976 27955 ts_tablet_manager.cc:1434] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:01.914220 27955 tablet_bootstrap.cc:492] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:01.914136 27961 raft_consensus.cc:493] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.918041 27961 raft_consensus.cc:515] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.921919 27961 leader_election.cc:290] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:01.926944 27955 tablet_bootstrap.cc:654] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.933698 27954 tablet_bootstrap.cc:492] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:01.934427 27954 ts_tablet_manager.cc:1403] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.050s	user 0.023s	sys 0.016s
I20260709 12:16:01.938067 27954 raft_consensus.cc:359] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.938982 27954 raft_consensus.cc:385] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.939306 27954 raft_consensus.cc:740] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.940584 27954 consensus_queue.cc:260] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.943208 27954 ts_tablet_manager.cc:1434] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 12:16:01.944010 27956 raft_consensus.cc:515] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.944312 27954 tablet_bootstrap.cc:492] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:01.946920 27763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:01.947628 27763 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.952133 27954 tablet_bootstrap.cc:654] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.952838 27959 raft_consensus.cc:493] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.953404 27959 raft_consensus.cc:515] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.956427 27959 leader_election.cc:290] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:01.958928 27956 leader_election.cc:290] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
W20260709 12:16:01.960047 27710 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:01.962682 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:01.963737 27961 raft_consensus.cc:493] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.964222 27961 raft_consensus.cc:515] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.966531 27961 leader_election.cc:290] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
W20260709 12:16:01.969345 27878 leader_election.cc:343] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.970037 27954 tablet_bootstrap.cc:492] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:01.970664 27954 ts_tablet_manager.cc:1403] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.007s
I20260709 12:16:01.973879 27763 raft_consensus.cc:2468] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 1.
I20260709 12:16:01.974136 27952 ts_tablet_manager.cc:1434] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.069s	user 0.018s	sys 0.027s
I20260709 12:16:01.973717 27954 raft_consensus.cc:359] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.980490 27952 tablet_bootstrap.cc:492] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:01.982036 27954 raft_consensus.cc:385] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.982955 27954 raft_consensus.cc:740] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.986581 27954 consensus_queue.cc:260] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:01.993261 27954 ts_tablet_manager.cc:1434] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:16:01.989357 27952 tablet_bootstrap.cc:654] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.996502 27954 tablet_bootstrap.cc:492] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:01.994825 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:01.994784 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce6d589c5d845c0adbbbad521d9733a" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:01.997910 27913 raft_consensus.cc:2468] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 0.
W20260709 12:16:01.998934 27803 leader_election.cc:343] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.999713 27763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce6d589c5d845c0adbbbad521d9733a" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:01.999980 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:02.000294 27803 leader_election.cc:304] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:02.000658 27764 raft_consensus.cc:2468] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 0.
I20260709 12:16:02.001276 27961 raft_consensus.cc:2804] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:02.001269 27804 leader_election.cc:343] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.001946 27955 tablet_bootstrap.cc:492] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:02.005590 27955 ts_tablet_manager.cc:1403] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.014s
I20260709 12:16:02.003144 27804 leader_election.cc:304] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:02.001669 27961 raft_consensus.cc:493] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.006103 27954 tablet_bootstrap.cc:654] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.007114 27961 raft_consensus.cc:3060] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.010116 27878 leader_election.cc:304] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 1371160c6a5049e3b8231f1deb73844b
I20260709 12:16:02.011160 27972 raft_consensus.cc:2804] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.011669 27972 raft_consensus.cc:697] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 LEADER]: Becoming Leader. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Running, Role: LEADER
I20260709 12:16:02.012557 27972 consensus_queue.cc:237] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.015069 27956 raft_consensus.cc:493] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.022511 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
I20260709 12:16:02.015599 27956 raft_consensus.cc:515] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.023331 27839 raft_consensus.cc:2468] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 0.
I20260709 12:16:02.024971 27956 leader_election.cc:290] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:02.025074 27729 leader_election.cc:304] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:02.026382 27959 raft_consensus.cc:2804] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.028481 27959 raft_consensus.cc:493] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.031473 27959 raft_consensus.cc:3060] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.034870 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
W20260709 12:16:02.037436 27878 leader_election.cc:343] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.041361 27961 raft_consensus.cc:515] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.034873 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
W20260709 12:16:02.043495 27878 leader_election.cc:343] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.043972 27878 leader_election.cc:304] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9
I20260709 12:16:02.044938 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce6d589c5d845c0adbbbad521d9733a" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:02.045591 27913 raft_consensus.cc:3060] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.045121 27956 raft_consensus.cc:2749] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.051728 27955 raft_consensus.cc:359] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.052574 27955 raft_consensus.cc:385] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.052891 27955 raft_consensus.cc:740] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.053814 27955 consensus_queue.cc:260] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.057752 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce6d589c5d845c0adbbbad521d9733a" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:02.055727 27913 raft_consensus.cc:2468] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 1.
I20260709 12:16:02.061477 27959 raft_consensus.cc:515] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.062722 27803 leader_election.cc:304] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:02.055507 27961 leader_election.cc:290] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.063932 27952 tablet_bootstrap.cc:492] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:02.064782 27981 raft_consensus.cc:2804] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.065532 27952 ts_tablet_manager.cc:1403] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.085s	user 0.008s	sys 0.021s
I20260709 12:16:02.065495 27981 raft_consensus.cc:493] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.066066 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:02.066395 27981 raft_consensus.cc:3060] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.070169 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
W20260709 12:16:02.064077 27804 leader_election.cc:343] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.071405 27961 raft_consensus.cc:2804] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.097723 27961 raft_consensus.cc:697] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Becoming Leader. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:02.098440 27954 tablet_bootstrap.cc:492] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:02.099095 27954 ts_tablet_manager.cc:1403] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.103s	user 0.024s	sys 0.021s
I20260709 12:16:02.098743 27961 consensus_queue.cc:237] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.099925 27952 raft_consensus.cc:359] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.100699 27952 raft_consensus.cc:385] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.100993 27952 raft_consensus.cc:740] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
W20260709 12:16:02.101564 27728 leader_election.cc:343] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.101840 27952 consensus_queue.cc:260] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.099193 27659 catalog_manager.cc:5697] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:02.105639 27985 raft_consensus.cc:493] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.104359 27952 ts_tablet_manager.cc:1434] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.003s
I20260709 12:16:02.106314 27985 raft_consensus.cc:515] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:02.107414 27955 ts_tablet_manager.cc:1434] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.101s	user 0.008s	sys 0.000s
I20260709 12:16:02.107937 27952 tablet_bootstrap.cc:492] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:02.109869 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:02.110107 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.110550 27764 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 0.
I20260709 12:16:02.104728 27959 leader_election.cc:290] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.110762 27912 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 0.
I20260709 12:16:02.102584 27954 raft_consensus.cc:359] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.112004 27954 raft_consensus.cc:385] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.112354 27954 raft_consensus.cc:740] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.112685 27981 raft_consensus.cc:515] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.113173 27954 consensus_queue.cc:260] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.113929 27839 raft_consensus.cc:2393] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88925d3120dd465bb8d39599142710b9 in current term 1: Already voted for candidate 1371160c6a5049e3b8231f1deb73844b in this term.
I20260709 12:16:02.108647 27985 leader_election.cc:290] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647), 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347)
I20260709 12:16:02.112121 27804 leader_election.cc:304] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:02.105342 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:02.115415 27913 raft_consensus.cc:3060] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.115713 27985 raft_consensus.cc:2804] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.116153 27985 raft_consensus.cc:493] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.116518 27985 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.108435 27955 tablet_bootstrap.cc:492] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:02.121799 27952 tablet_bootstrap.cc:654] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.122499 27959 raft_consensus.cc:493] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.124081 27959 raft_consensus.cc:515] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.124848 27955 tablet_bootstrap.cc:654] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.127501 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
I20260709 12:16:02.127828 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.128496 27912 raft_consensus.cc:2468] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 0.
W20260709 12:16:02.129082 27729 leader_election.cc:343] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.130116 27728 leader_election.cc:304] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 1371160c6a5049e3b8231f1deb73844b
I20260709 12:16:02.130612 27913 raft_consensus.cc:2468] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 1.
I20260709 12:16:02.131379 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:02.132375 27764 raft_consensus.cc:2393] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1371160c6a5049e3b8231f1deb73844b in current term 1: Already voted for candidate 88925d3120dd465bb8d39599142710b9 in this term.
I20260709 12:16:02.131729 27728 leader_election.cc:304] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 1371160c6a5049e3b8231f1deb73844b
I20260709 12:16:02.132541 27959 leader_election.cc:290] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.134593 27981 leader_election.cc:290] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.135998 27959 raft_consensus.cc:2804] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.136627 27990 raft_consensus.cc:2804] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.136930 27959 raft_consensus.cc:493] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.137168 27990 raft_consensus.cc:697] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:02.137470 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:02.138437 27912 raft_consensus.cc:2393] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1371160c6a5049e3b8231f1deb73844b in current term 1: Already voted for candidate 88925d3120dd465bb8d39599142710b9 in this term.
I20260709 12:16:02.138136 27990 consensus_queue.cc:237] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.137449 27959 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.140748 27661 catalog_manager.cc:5697] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1371160c6a5049e3b8231f1deb73844b (127.26.123.2). New cstate: current_term: 1 leader_uuid: "1371160c6a5049e3b8231f1deb73844b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.138315 27985 raft_consensus.cc:515] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:02.144618 27803 leader_election.cc:304] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b; no voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9
I20260709 12:16:02.145679 27989 raft_consensus.cc:2749] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.147334 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:02.147986 27764 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.149271 27954 ts_tablet_manager.cc:1434] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.007s
I20260709 12:16:02.149596 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:02.150262 27912 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.150494 27954 tablet_bootstrap.cc:492] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:02.160837 27985 leader_election.cc:290] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Requested vote from peers 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647), 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347)
I20260709 12:16:02.164216 27912 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 1.
I20260709 12:16:02.169801 27954 tablet_bootstrap.cc:654] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.169508 27803 leader_election.cc:304] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:02.172315 27764 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 1.
I20260709 12:16:02.172787 27959 raft_consensus.cc:515] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.175323 27955 tablet_bootstrap.cc:492] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:02.175299 27959 leader_election.cc:290] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.176069 27955 ts_tablet_manager.cc:1403] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.068s	user 0.012s	sys 0.008s
I20260709 12:16:02.177699 27660 catalog_manager.cc:5697] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.179697 27952 tablet_bootstrap.cc:492] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:02.180274 27952 ts_tablet_manager.cc:1403] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.073s	user 0.016s	sys 0.004s
I20260709 12:16:02.180812 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:02.178910 27985 raft_consensus.cc:2804] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.181977 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:02.181694 27955 raft_consensus.cc:359] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.182562 27955 raft_consensus.cc:385] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.182591 27912 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.182919 27955 raft_consensus.cc:740] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
W20260709 12:16:02.183418 27729 leader_election.cc:343] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.182129 27985 raft_consensus.cc:697] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Becoming Leader. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:02.183724 27955 consensus_queue.cc:260] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.184471 27952 raft_consensus.cc:359] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.184695 27985 consensus_queue.cc:237] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:02.185263 27952 raft_consensus.cc:385] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.185585 27952 raft_consensus.cc:740] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.186240 27955 ts_tablet_manager.cc:1434] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:16:02.187340 27955 tablet_bootstrap.cc:492] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:02.186553 27952 consensus_queue.cc:260] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.190816 27952 ts_tablet_manager.cc:1434] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 12:16:02.192044 27952 tablet_bootstrap.cc:492] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:02.193423 27989 raft_consensus.cc:493] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.194058 27989 raft_consensus.cc:515] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.195465 27912 raft_consensus.cc:2468] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 1.
I20260709 12:16:02.196991 27728 leader_election.cc:304] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 1371160c6a5049e3b8231f1deb73844b
I20260709 12:16:02.198330 27959 raft_consensus.cc:2804] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.199113 27959 raft_consensus.cc:697] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:02.198951 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:02.198951 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.200150 27764 raft_consensus.cc:2468] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 0.
I20260709 12:16:02.200232 27959 consensus_queue.cc:237] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
W20260709 12:16:02.201484 27803 leader_election.cc:343] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.201714 27989 leader_election.cc:290] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.202360 27804 leader_election.cc:304] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:02.203274 27989 raft_consensus.cc:2804] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.203667 27989 raft_consensus.cc:493] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.204757 27989 raft_consensus.cc:3060] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.218739 27955 tablet_bootstrap.cc:654] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.219749 27989 raft_consensus.cc:515] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.226809 27989 leader_election.cc:290] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.231323 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:02.232180 27952 tablet_bootstrap.cc:654] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.232627 27954 tablet_bootstrap.cc:492] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:02.233204 27954 ts_tablet_manager.cc:1403] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.083s	user 0.016s	sys 0.020s
I20260709 12:16:02.234263 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:02.235172 27764 raft_consensus.cc:3060] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.236588 27954 raft_consensus.cc:359] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.237380 27954 raft_consensus.cc:385] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:02.239681 27803 leader_election.cc:343] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.240247 27954 raft_consensus.cc:740] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.246157 27954 consensus_queue.cc:260] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.246883 27764 raft_consensus.cc:2468] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 1.
I20260709 12:16:02.249352 27954 ts_tablet_manager.cc:1434] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.005s
I20260709 12:16:02.250471 27804 leader_election.cc:304] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:02.250831 27954 tablet_bootstrap.cc:492] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:02.252269 27989 raft_consensus.cc:2804] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.252889 27989 raft_consensus.cc:697] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Becoming Leader. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:02.253935 27989 consensus_queue.cc:237] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.255488 27985 raft_consensus.cc:493] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.256057 27985 raft_consensus.cc:515] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.259796 27954 tablet_bootstrap.cc:654] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.260231 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.261523 27985 leader_election.cc:290] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
W20260709 12:16:02.262876 27803 leader_election.cc:343] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.266281 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:02.267081 27764 raft_consensus.cc:2468] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 0.
I20260709 12:16:02.267601 27955 tablet_bootstrap.cc:492] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:02.268242 27955 ts_tablet_manager.cc:1403] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.081s	user 0.023s	sys 0.009s
I20260709 12:16:02.268323 27804 leader_election.cc:304] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:02.269865 27989 raft_consensus.cc:2804] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.270335 27989 raft_consensus.cc:493] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.270749 27989 raft_consensus.cc:3060] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.271664 27952 tablet_bootstrap.cc:492] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:02.272320 27952 ts_tablet_manager.cc:1403] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.081s	user 0.008s	sys 0.013s
I20260709 12:16:02.271785 27955 raft_consensus.cc:359] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.277664 27955 raft_consensus.cc:385] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.278102 27955 raft_consensus.cc:740] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.267588 27954 tablet_bootstrap.cc:492] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:02.280790 27954 ts_tablet_manager.cc:1403] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.001s
I20260709 12:16:02.296674 27952 raft_consensus.cc:359] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.297657 27952 raft_consensus.cc:385] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.297550 27954 raft_consensus.cc:359] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.298081 27952 raft_consensus.cc:740] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.298381 27954 raft_consensus.cc:385] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.298700 27954 raft_consensus.cc:740] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.274143 27659 catalog_manager.cc:5697] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1371160c6a5049e3b8231f1deb73844b (127.26.123.2). New cstate: current_term: 1 leader_uuid: "1371160c6a5049e3b8231f1deb73844b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.299419 27954 consensus_queue.cc:260] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.303715 27954 ts_tablet_manager.cc:1434] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.003s
I20260709 12:16:02.304783 27954 tablet_bootstrap.cc:492] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:02.304453 27989 raft_consensus.cc:515] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.299518 27952 consensus_queue.cc:260] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.300015 27955 consensus_queue.cc:260] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.305078 27661 catalog_manager.cc:5697] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.307860 27955 ts_tablet_manager.cc:1434] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.005s
I20260709 12:16:02.309037 27955 tablet_bootstrap.cc:492] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:02.311185 27952 ts_tablet_manager.cc:1434] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.039s	user 0.011s	sys 0.004s
I20260709 12:16:02.315819 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:02.316486 27764 raft_consensus.cc:3060] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.318174 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
W20260709 12:16:02.319901 27803 leader_election.cc:343] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.311920 27954 tablet_bootstrap.cc:654] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.314615 27989 leader_election.cc:290] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.321913 27955 tablet_bootstrap.cc:654] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.332921 27954 tablet_bootstrap.cc:492] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:02.332233 27660 catalog_manager.cc:5697] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1371160c6a5049e3b8231f1deb73844b (127.26.123.2). New cstate: current_term: 1 leader_uuid: "1371160c6a5049e3b8231f1deb73844b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.333442 27954 ts_tablet_manager.cc:1403] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.004s
I20260709 12:16:02.336602 27954 raft_consensus.cc:359] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.337339 27954 raft_consensus.cc:385] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.315830 27952 tablet_bootstrap.cc:492] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:02.337628 27954 raft_consensus.cc:740] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.340874 27954 consensus_queue.cc:260] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.338487 27764 raft_consensus.cc:2468] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 1.
I20260709 12:16:02.343938 27954 ts_tablet_manager.cc:1434] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:16:02.343648 27804 leader_election.cc:304] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:02.345103 27954 tablet_bootstrap.cc:492] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b: Bootstrap starting.
I20260709 12:16:02.345885 27989 raft_consensus.cc:2804] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.346520 27989 raft_consensus.cc:697] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Becoming Leader. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:02.346750 27952 tablet_bootstrap.cc:654] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.347555 27989 consensus_queue.cc:237] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.350697 27959 raft_consensus.cc:493] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.351287 27959 raft_consensus.cc:515] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.355204 27985 raft_consensus.cc:493] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.355783 27985 raft_consensus.cc:515] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.356405 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf0f32ff2c6416e868adb7ddfe6dfa3" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.357067 27912 raft_consensus.cc:2468] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 0.
I20260709 12:16:02.357002 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf0f32ff2c6416e868adb7ddfe6dfa3" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
I20260709 12:16:02.358184 27839 raft_consensus.cc:2468] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 0.
I20260709 12:16:02.358668 27728 leader_election.cc:304] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:02.359618 27990 raft_consensus.cc:2804] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.360028 27990 raft_consensus.cc:493] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.360414 27990 raft_consensus.cc:3060] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.361021 27985 leader_election.cc:290] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.361296 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d32c67cfb44ffdb4ae9774e8a4d71b" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
W20260709 12:16:02.363693 27804 leader_election.cc:343] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.359508 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d32c67cfb44ffdb4ae9774e8a4d71b" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.367990 27912 raft_consensus.cc:2468] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 0.
I20260709 12:16:02.368271 27954 tablet_bootstrap.cc:654] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.369684 27803 leader_election.cc:304] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 88925d3120dd465bb8d39599142710b9
I20260709 12:16:02.354298 27959 leader_election.cc:290] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.371271 27989 raft_consensus.cc:2804] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.371651 27989 raft_consensus.cc:493] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.371997 27989 raft_consensus.cc:3060] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.372711 27952 tablet_bootstrap.cc:492] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:02.373325 27952 ts_tablet_manager.cc:1403] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.058s	user 0.022s	sys 0.006s
I20260709 12:16:02.374140 27990 raft_consensus.cc:515] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.376780 27952 raft_consensus.cc:359] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.377679 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf0f32ff2c6416e868adb7ddfe6dfa3" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:02.378185 27952 raft_consensus.cc:385] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.378441 27990 leader_election.cc:290] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.378623 27952 raft_consensus.cc:740] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.378444 27839 raft_consensus.cc:3060] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.379760 27952 consensus_queue.cc:260] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.382189 27952 ts_tablet_manager.cc:1434] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:16:02.383307 27952 tablet_bootstrap.cc:492] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914: Bootstrap starting.
I20260709 12:16:02.384999 27989 raft_consensus.cc:515] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.377657 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf0f32ff2c6416e868adb7ddfe6dfa3" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:02.387310 27955 tablet_bootstrap.cc:492] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:02.387717 27912 raft_consensus.cc:3060] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.387844 27955 ts_tablet_manager.cc:1403] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.079s	user 0.027s	sys 0.012s
I20260709 12:16:02.389889 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d32c67cfb44ffdb4ae9774e8a4d71b" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:02.391500 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d32c67cfb44ffdb4ae9774e8a4d71b" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:02.392167 27913 raft_consensus.cc:3060] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.392302 27660 catalog_manager.cc:5697] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1371160c6a5049e3b8231f1deb73844b (127.26.123.2). New cstate: current_term: 1 leader_uuid: "1371160c6a5049e3b8231f1deb73844b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.393698 27839 raft_consensus.cc:2468] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 1.
I20260709 12:16:02.394573 27952 tablet_bootstrap.cc:654] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.395032 27729 leader_election.cc:304] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 
W20260709 12:16:02.398101 27804 leader_election.cc:343] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.399593 27955 raft_consensus.cc:359] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.400314 27955 raft_consensus.cc:385] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.400635 27955 raft_consensus.cc:740] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.401355 27955 consensus_queue.cc:260] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.402927 27912 raft_consensus.cc:2468] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 1.
I20260709 12:16:02.404796 27959 raft_consensus.cc:2804] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.405349 27959 raft_consensus.cc:697] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:02.406273 27959 consensus_queue.cc:237] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.407157 27913 raft_consensus.cc:2468] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 1.
I20260709 12:16:02.408592 27954 tablet_bootstrap.cc:492] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b: No bootstrap required, opened a new log
I20260709 12:16:02.408478 27803 leader_election.cc:304] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 88925d3120dd465bb8d39599142710b9
I20260709 12:16:02.409166 27954 ts_tablet_manager.cc:1403] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.003s
I20260709 12:16:02.409440 27985 raft_consensus.cc:2804] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.410050 27985 raft_consensus.cc:697] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Becoming Leader. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:02.411013 27985 consensus_queue.cc:237] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.412190 27955 ts_tablet_manager.cc:1434] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:16:02.412519 27954 raft_consensus.cc:359] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.413311 27955 tablet_bootstrap.cc:492] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:02.413318 27954 raft_consensus.cc:385] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.414047 27954 raft_consensus.cc:740] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.415514 27954 consensus_queue.cc:260] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.415864 27989 leader_election.cc:290] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.417882 27954 ts_tablet_manager.cc:1434] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 12:16:02.419612 27952 tablet_bootstrap.cc:492] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914: No bootstrap required, opened a new log
I20260709 12:16:02.420112 27952 ts_tablet_manager.cc:1403] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent bootstrapping tablet: real 0.037s	user 0.023s	sys 0.003s
I20260709 12:16:02.423415 27952 raft_consensus.cc:359] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.424019 27952 raft_consensus.cc:385] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.424245 27952 raft_consensus.cc:740] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.424867 27952 consensus_queue.cc:260] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.427192 27952 ts_tablet_manager.cc:1434] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:16:02.432809 27955 tablet_bootstrap.cc:654] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.433287 27661 catalog_manager.cc:5697] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.434729 27659 catalog_manager.cc:5697] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1371160c6a5049e3b8231f1deb73844b (127.26.123.2). New cstate: current_term: 1 leader_uuid: "1371160c6a5049e3b8231f1deb73844b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.438197 27989 raft_consensus.cc:493] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.438819 27989 raft_consensus.cc:515] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.439940 27955 tablet_bootstrap.cc:492] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:02.440450 27955 ts_tablet_manager.cc:1403] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.000s
I20260709 12:16:02.442075 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:02.443392 27989 leader_election.cc:290] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.444267 27955 raft_consensus.cc:359] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.444981 27955 raft_consensus.cc:385] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.445014 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.445343 27955 raft_consensus.cc:740] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.445960 27913 raft_consensus.cc:2393] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1371160c6a5049e3b8231f1deb73844b in current term 1: Already voted for candidate 88925d3120dd465bb8d39599142710b9 in this term.
I20260709 12:16:02.447382 27803 leader_election.cc:304] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b; no voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9
I20260709 12:16:02.448223 27989 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.450629 27982 consensus_queue.cc:1048] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:02.447057 27955 consensus_queue.cc:260] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.453992 27955 ts_tablet_manager.cc:1434] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:16:02.454979 27955 tablet_bootstrap.cc:492] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9: Bootstrap starting.
I20260709 12:16:02.455943 27989 raft_consensus.cc:2749] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.461876 27955 tablet_bootstrap.cc:654] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.482877 27839 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.483351 27955 tablet_bootstrap.cc:492] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9: No bootstrap required, opened a new log
I20260709 12:16:02.483914 27955 ts_tablet_manager.cc:1403] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.016s
I20260709 12:16:02.486801 27955 raft_consensus.cc:359] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.487520 27955 raft_consensus.cc:385] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.487797 27955 raft_consensus.cc:740] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.488487 27955 consensus_queue.cc:260] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.490527 27955 ts_tablet_manager.cc:1434] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
W20260709 12:16:02.490660 27865 tablet.cc:2406] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:02.492864 27989 raft_consensus.cc:493] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.493373 27989 raft_consensus.cc:515] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.495476 28017 consensus_queue.cc:1048] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:02.497802 27989 leader_election.cc:290] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.498526 27763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:02.500401 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.501973 27803 leader_election.cc:304] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b; no voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9
I20260709 12:16:02.502656 27985 consensus_queue.cc:1048] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.512698 27985 raft_consensus.cc:2749] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:02.514744 28017 raft_consensus.cc:493] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.515304 28017 raft_consensus.cc:515] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.519380 28017 leader_election.cc:290] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:02.520138 27763 raft_consensus.cc:3060] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.520023 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:02.520916 27764 raft_consensus.cc:2393] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in current term 1: Already voted for candidate 1371160c6a5049e3b8231f1deb73844b in this term.
I20260709 12:16:02.521078 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
I20260709 12:16:02.524335 27878 leader_election.cc:304] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9
I20260709 12:16:02.526983 28017 raft_consensus.cc:3060] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.531915 27989 consensus_queue.cc:1048] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.535305 28017 raft_consensus.cc:2749] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.599849 27959 raft_consensus.cc:493] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.600481 27959 raft_consensus.cc:515] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.602955 27959 leader_election.cc:290] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.603379 27990 consensus_queue.cc:1048] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.604247 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
I20260709 12:16:02.604614 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.604954 27839 raft_consensus.cc:2468] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 0.
I20260709 12:16:02.605261 27913 raft_consensus.cc:2468] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 0.
I20260709 12:16:02.606107 27729 leader_election.cc:304] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:02.607075 27990 raft_consensus.cc:2804] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.607517 27990 raft_consensus.cc:493] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.607884 27990 raft_consensus.cc:3060] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.613044 27959 consensus_queue.cc:1048] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.617111 27990 raft_consensus.cc:515] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.619544 27986 consensus_queue.cc:1048] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:02.620656 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:02.621254 27913 raft_consensus.cc:3060] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.625057 27990 leader_election.cc:290] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.628192 27913 raft_consensus.cc:2468] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 1.
I20260709 12:16:02.629117 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:02.629328 27959 consensus_queue.cc:1048] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.629799 27838 raft_consensus.cc:3060] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.632236 27728 leader_election.cc:304] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:02.634016 27959 raft_consensus.cc:2804] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.634594 27959 raft_consensus.cc:697] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:02.635573 27959 consensus_queue.cc:237] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.646034 27838 raft_consensus.cc:2468] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 1.
I20260709 12:16:02.648645 28024 raft_consensus.cc:493] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.649312 28024 raft_consensus.cc:515] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.656108 28024 leader_election.cc:290] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:02.659390 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d32c67cfb44ffdb4ae9774e8a4d71b" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" is_pre_election: true
I20260709 12:16:02.660447 27911 raft_consensus.cc:2393] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88925d3120dd465bb8d39599142710b9 in current term 1: Already voted for candidate 1371160c6a5049e3b8231f1deb73844b in this term.
I20260709 12:16:02.692605 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d32c67cfb44ffdb4ae9774e8a4d71b" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
I20260709 12:16:02.695043 27729 leader_election.cc:304] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88925d3120dd465bb8d39599142710b9; no voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:02.696063 27959 raft_consensus.cc:3060] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.699951 27660 catalog_manager.cc:5697] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.705047 27959 raft_consensus.cc:2749] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.713572 28017 raft_consensus.cc:493] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.714182 28017 raft_consensus.cc:515] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:02.716439 28017 leader_election.cc:290] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:02.717640 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88925d3120dd465bb8d39599142710b9" is_pre_election: true
I20260709 12:16:02.718493 27764 raft_consensus.cc:2393] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in current term 1: Already voted for candidate 1371160c6a5049e3b8231f1deb73844b in this term.
I20260709 12:16:02.719846 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1371160c6a5049e3b8231f1deb73844b" is_pre_election: true
I20260709 12:16:02.721261 27878 leader_election.cc:304] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9
I20260709 12:16:02.723083 28017 raft_consensus.cc:3060] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.731248 28017 raft_consensus.cc:2749] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.764554 27989 consensus_queue.cc:1048] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.776780 27985 consensus_queue.cc:1048] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:02.780265 27985 consensus_queue.cc:1048] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.794765 27989 consensus_queue.cc:1048] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.863722 27989 consensus_queue.cc:1048] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.875357 27981 consensus_queue.cc:1048] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.896829 27959 consensus_queue.cc:1048] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.907886 27990 consensus_queue.cc:1048] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.956698 27981 consensus_queue.cc:1048] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.995894 27838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
mode: GRACEFUL
new_leader_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:50470
I20260709 12:16:02.996644 27838 raft_consensus.cc:606] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Received request to transfer leadership to TS 88925d3120dd465bb8d39599142710b9
I20260709 12:16:02.996899 27985 consensus_queue.cc:1048] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:03.011166 27838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ce6d589c5d845c0adbbbad521d9733a"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
mode: GRACEFUL
new_leader_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:50470
I20260709 12:16:03.011746 27838 raft_consensus.cc:606] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Received request to transfer leadership to TS 88925d3120dd465bb8d39599142710b9
I20260709 12:16:03.017791 27838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
mode: GRACEFUL
new_leader_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:50470
I20260709 12:16:03.018353 27838 raft_consensus.cc:606] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Received request to transfer leadership to TS 88925d3120dd465bb8d39599142710b9
I20260709 12:16:03.027107 28024 consensus_queue.cc:1048] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:03.033950 27913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620"
dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
mode: GRACEFUL
new_leader_uuid: "1371160c6a5049e3b8231f1deb73844b"
 from {username='slave'} at 127.0.0.1:52698
I20260709 12:16:03.034646 27913 raft_consensus.cc:606] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 LEADER]: Received request to transfer leadership to TS 1371160c6a5049e3b8231f1deb73844b
I20260709 12:16:03.040804 27959 consensus_queue.cc:1048] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:03.056613 27764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e166fabf328942638baed5d4c053db85"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
mode: GRACEFUL
new_leader_uuid: "1371160c6a5049e3b8231f1deb73844b"
 from {username='slave'} at 127.0.0.1:48126
I20260709 12:16:03.057148 27764 raft_consensus.cc:606] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Received request to transfer leadership to TS 1371160c6a5049e3b8231f1deb73844b
I20260709 12:16:03.068069 27764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bf0f32ff2c6416e868adb7ddfe6dfa3"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
mode: GRACEFUL
new_leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:48126
I20260709 12:16:03.068620 27764 raft_consensus.cc:606] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Received request to transfer leadership to TS 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:03.301395 28032 raft_consensus.cc:993] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b: : Instructing follower 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:03.301903 27985 raft_consensus.cc:1081] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Signalling peer 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:03.303546 27764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:57312
I20260709 12:16:03.304103 27764 raft_consensus.cc:493] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:03.304427 27764 raft_consensus.cc:3060] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.305449 27956 raft_consensus.cc:993] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914: : Instructing follower 1371160c6a5049e3b8231f1deb73844b to start an election
I20260709 12:16:03.305935 28017 raft_consensus.cc:1081] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 LEADER]: Signalling peer 1371160c6a5049e3b8231f1deb73844b to start an election
I20260709 12:16:03.308234 27838 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
 from {username='slave'} at 127.0.0.1:58852
I20260709 12:16:03.308957 27838 raft_consensus.cc:493] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:03.309391 27838 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.313210 27764 raft_consensus.cc:515] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.315574 27764 leader_election.cc:290] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 2 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:03.316455 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:03.316582 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "124c98e6d8f14ad4a19fb65b95779dc7" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:03.317267 27913 raft_consensus.cc:3060] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.317168 27839 raft_consensus.cc:3055] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:03.317109 27838 raft_consensus.cc:515] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.317883 27839 raft_consensus.cc:740] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:03.318708 27839 consensus_queue.cc:260] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.319913 27839 raft_consensus.cc:3060] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.320690 27838 leader_election.cc:290] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 2 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:03.322021 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:03.323767 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:03.325027 27764 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.325809 27912 raft_consensus.cc:3055] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:03.326063 27913 raft_consensus.cc:2468] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 2.
I20260709 12:16:03.326187 27912 raft_consensus.cc:740] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Running, Role: LEADER
I20260709 12:16:03.327479 27728 leader_election.cc:304] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:03.327303 27912 consensus_queue.cc:260] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.328302 27959 raft_consensus.cc:2804] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:03.328675 27959 raft_consensus.cc:697] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:03.329633 27959 consensus_queue.cc:237] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.333120 27764 raft_consensus.cc:2468] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 2.
I20260709 12:16:03.334566 27804 leader_election.cc:304] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:03.335970 28032 raft_consensus.cc:2804] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:03.336588 27989 raft_consensus.cc:993] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b: : Instructing follower 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:03.337033 27989 raft_consensus.cc:1081] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Signalling peer 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:03.336489 28032 raft_consensus.cc:697] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Becoming Leader. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:03.337402 27661 catalog_manager.cc:5697] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 1 to 2, leader changed from 1371160c6a5049e3b8231f1deb73844b (127.26.123.2) to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:03.338374 28032 consensus_queue.cc:237] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.338750 27764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:57312
I20260709 12:16:03.339413 27764 raft_consensus.cc:493] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:03.339821 27764 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.328627 27912 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.346932 27839 raft_consensus.cc:2468] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 2.
I20260709 12:16:03.348158 27764 raft_consensus.cc:515] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:03.349280 27912 raft_consensus.cc:2468] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 2.
I20260709 12:16:03.351370 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:03.352013 27913 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.351750 27659 catalog_manager.cc:5697] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b reported cstate change: term changed from 1 to 2, leader changed from 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3) to 1371160c6a5049e3b8231f1deb73844b (127.26.123.2). New cstate: current_term: 2 leader_uuid: "1371160c6a5049e3b8231f1deb73844b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:03.352523 27764 leader_election.cc:290] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 2 election: Requested vote from peers 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:03.352726 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:03.353344 27839 raft_consensus.cc:3055] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:03.353650 27839 raft_consensus.cc:740] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:03.354478 27839 consensus_queue.cc:260] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:03.355623 27839 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.363292 27839 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 2.
I20260709 12:16:03.363292 27913 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 2.
I20260709 12:16:03.364727 27729 leader_election.cc:304] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:03.365618 27959 raft_consensus.cc:2804] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:03.366133 27959 raft_consensus.cc:697] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:03.366935 27959 consensus_queue.cc:237] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:03.373718 27659 catalog_manager.cc:5697] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 1 to 2, leader changed from 1371160c6a5049e3b8231f1deb73844b (127.26.123.2) to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:03.447871 28023 raft_consensus.cc:993] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9: : Instructing follower 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:03.448367 27959 raft_consensus.cc:1081] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Signalling peer 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:03.450019 27913 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bf0f32ff2c6416e868adb7ddfe6dfa3"
dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:52694
I20260709 12:16:03.450528 27913 raft_consensus.cc:493] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:03.450836 27913 raft_consensus.cc:3060] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.456795 27913 raft_consensus.cc:515] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.458774 27913 leader_election.cc:290] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 2 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:03.459872 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf0f32ff2c6416e868adb7ddfe6dfa3" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:03.459713 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf0f32ff2c6416e868adb7ddfe6dfa3" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:03.460400 27839 raft_consensus.cc:3060] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.460500 27764 raft_consensus.cc:3055] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:03.460865 27764 raft_consensus.cc:740] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:03.461694 27764 consensus_queue.cc:260] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.462817 27764 raft_consensus.cc:3060] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.466233 27839 raft_consensus.cc:2468] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 2.
I20260709 12:16:03.467518 27878 leader_election.cc:304] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:03.468372 28017 raft_consensus.cc:2804] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:03.468549 27764 raft_consensus.cc:2468] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 2.
I20260709 12:16:03.468801 28017 raft_consensus.cc:697] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 LEADER]: Becoming Leader. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Running, Role: LEADER
I20260709 12:16:03.469779 28017 consensus_queue.cc:237] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.477062 27659 catalog_manager.cc:5697] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 reported cstate change: term changed from 1 to 2, leader changed from 88925d3120dd465bb8d39599142710b9 (127.26.123.1) to 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:03.495514 27985 raft_consensus.cc:993] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b: : Instructing follower 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:03.495960 28032 raft_consensus.cc:1081] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Signalling peer 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:03.497443 27764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ce6d589c5d845c0adbbbad521d9733a"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:57312
I20260709 12:16:03.498064 27764 raft_consensus.cc:493] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:03.498474 27764 raft_consensus.cc:3060] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.504742 27764 raft_consensus.cc:515] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.506664 27764 leader_election.cc:290] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 2 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:03.507682 27839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce6d589c5d845c0adbbbad521d9733a" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:03.508352 27839 raft_consensus.cc:3055] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:03.508386 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce6d589c5d845c0adbbbad521d9733a" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:03.508826 27839 raft_consensus.cc:740] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:03.509068 27913 raft_consensus.cc:3060] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.509771 27839 consensus_queue.cc:260] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.511109 27839 raft_consensus.cc:3060] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.517040 27913 raft_consensus.cc:2468] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 2.
I20260709 12:16:03.518559 27728 leader_election.cc:304] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:03.518754 27839 raft_consensus.cc:2468] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 2.
I20260709 12:16:03.519553 27959 raft_consensus.cc:2804] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:03.520007 27959 raft_consensus.cc:697] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:03.520753 27959 consensus_queue.cc:237] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.527920 27659 catalog_manager.cc:5697] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 1 to 2, leader changed from 1371160c6a5049e3b8231f1deb73844b (127.26.123.2) to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:03.617148 27986 raft_consensus.cc:993] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9: : Instructing follower 1371160c6a5049e3b8231f1deb73844b to start an election
I20260709 12:16:03.617672 27986 raft_consensus.cc:1081] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Signalling peer 1371160c6a5049e3b8231f1deb73844b to start an election
I20260709 12:16:03.619980 27839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e166fabf328942638baed5d4c053db85"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
 from {username='slave'} at 127.0.0.1:50464
I20260709 12:16:03.620517 27839 raft_consensus.cc:493] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:03.620841 27839 raft_consensus.cc:3060] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.629158 27839 raft_consensus.cc:515] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.631336 27839 leader_election.cc:290] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 2 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:03.632210 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:03.632565 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:03.632941 27764 raft_consensus.cc:3055] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:03.633319 27913 raft_consensus.cc:3060] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.633458 27764 raft_consensus.cc:740] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:03.634758 27764 consensus_queue.cc:260] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.635866 27764 raft_consensus.cc:3060] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:03.642140 27913 raft_consensus.cc:2468] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 2.
I20260709 12:16:03.642433 27764 raft_consensus.cc:2468] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 2.
I20260709 12:16:03.643498 27803 leader_election.cc:304] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:03.644268 27985 raft_consensus.cc:2804] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:03.644752 27985 raft_consensus.cc:697] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Becoming Leader. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:03.645507 27985 consensus_queue.cc:237] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:03.652406 27659 catalog_manager.cc:5697] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b reported cstate change: term changed from 1 to 2, leader changed from 88925d3120dd465bb8d39599142710b9 (127.26.123.1) to 1371160c6a5049e3b8231f1deb73844b (127.26.123.2). New cstate: current_term: 2 leader_uuid: "1371160c6a5049e3b8231f1deb73844b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:03.751335 27764 raft_consensus.cc:1275] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Refusing update from remote peer 1371160c6a5049e3b8231f1deb73844b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.752745 27985 consensus_queue.cc:1048] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.765096 27913 raft_consensus.cc:1275] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Refusing update from remote peer 1371160c6a5049e3b8231f1deb73844b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.766932 28032 consensus_queue.cc:1048] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.797461 27913 raft_consensus.cc:1275] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.797825 27839 raft_consensus.cc:1275] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.798756 27959 consensus_queue.cc:1048] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.799513 28023 consensus_queue.cc:1048] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.846357 27913 raft_consensus.cc:1275] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.847836 27986 consensus_queue.cc:1048] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.857149 27839 raft_consensus.cc:1275] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.858873 28023 consensus_queue.cc:1048] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.895200 27839 raft_consensus.cc:1275] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.896551 28017 consensus_queue.cc:1048] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.904793 27764 raft_consensus.cc:1275] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.906567 27956 consensus_queue.cc:1048] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.918311 27913 raft_consensus.cc:1275] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.920002 28023 consensus_queue.cc:1048] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:03.930936 27839 raft_consensus.cc:1275] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:03.932631 27959 consensus_queue.cc:1048] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:04.083005 27913 raft_consensus.cc:1275] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Refusing update from remote peer 1371160c6a5049e3b8231f1deb73844b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:04.084368 28032 consensus_queue.cc:1048] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:04.095212 27764 raft_consensus.cc:1275] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Refusing update from remote peer 1371160c6a5049e3b8231f1deb73844b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:04.096587 27985 consensus_queue.cc:1048] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:06.100051 27116 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:16:06.100739 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:06.116501 27764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
mode: GRACEFUL
new_leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:48134
I20260709 12:16:06.117002 27764 raft_consensus.cc:606] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Received request to transfer leadership to TS 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:06.119340 27764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
mode: GRACEFUL
new_leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:48134
I20260709 12:16:06.119776 27764 raft_consensus.cc:606] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Received request to transfer leadership to TS 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:06.120682 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:06.121135 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:06.207296 28062 raft_consensus.cc:993] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9: : Instructing follower 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:06.207736 28071 raft_consensus.cc:1081] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Signalling peer 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:06.209187 27913 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c"
dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:52694
I20260709 12:16:06.209643 27913 raft_consensus.cc:493] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:06.209955 27913 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:06.215312 27913 raft_consensus.cc:515] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:06.216971 27913 leader_election.cc:290] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 2 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:06.217828 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:06.218107 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:06.218453 27764 raft_consensus.cc:3055] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:06.218624 27838 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:06.218851 27764 raft_consensus.cc:740] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:06.219687 27764 consensus_queue.cc:260] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:06.220988 27764 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:06.226691 27838 raft_consensus.cc:2468] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 2.
I20260709 12:16:06.228161 27878 leader_election.cc:304] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:06.228449 27764 raft_consensus.cc:2468] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 2.
I20260709 12:16:06.228986 28068 raft_consensus.cc:2804] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:06.230227 28068 raft_consensus.cc:697] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 LEADER]: Becoming Leader. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Running, Role: LEADER
I20260709 12:16:06.231041 28068 consensus_queue.cc:237] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:06.238396 27659 catalog_manager.cc:5697] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 reported cstate change: term changed from 1 to 2, leader changed from 88925d3120dd465bb8d39599142710b9 (127.26.123.1) to 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:06.307523 28062 raft_consensus.cc:993] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9: : Instructing follower 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:06.307968 28062 raft_consensus.cc:1081] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Signalling peer 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:06.309454 27913 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d"
dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:52694
I20260709 12:16:06.309916 27913 raft_consensus.cc:493] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:06.310154 27913 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:06.314708 27913 raft_consensus.cc:515] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:06.316094 27913 leader_election.cc:290] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 3 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347)
I20260709 12:16:06.317008 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:06.317227 27764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:06.317492 27838 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:06.317715 27764 raft_consensus.cc:3055] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:06.317989 27764 raft_consensus.cc:740] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:06.318544 27764 consensus_queue.cc:260] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:06.319577 27764 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:06.322433 27838 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 3.
I20260709 12:16:06.323376 27878 leader_election.cc:304] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:06.324007 28068 raft_consensus.cc:2804] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:06.324364 27764 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 3.
I20260709 12:16:06.324368 28068 raft_consensus.cc:697] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 LEADER]: Becoming Leader. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Running, Role: LEADER
I20260709 12:16:06.325304 28068 consensus_queue.cc:237] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:06.332684 27659 catalog_manager.cc:5697] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 reported cstate change: term changed from 2 to 3, leader changed from 88925d3120dd465bb8d39599142710b9 (127.26.123.1) to 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3). New cstate: current_term: 3 leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:06.630873 27838 raft_consensus.cc:1275] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:06.632114 28068 consensus_queue.cc:1048] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:06.648064 27764 raft_consensus.cc:1275] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:06.650100 28081 consensus_queue.cc:1048] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:06.723658 27838 raft_consensus.cc:1275] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:06.725035 28081 consensus_queue.cc:1048] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:06.734375 27764 raft_consensus.cc:1275] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:06.735982 28072 consensus_queue.cc:1048] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:07.122223 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:07.127676 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:07.128144 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:08.129263 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:08.135007 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:08.135533 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:09.136638 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:09.142973 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:09.143677 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:10.145197 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:10.152078 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:10.152698 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:11.154203 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:11.160388 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:11.160866 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:12.161966 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:12.167342 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:12.167747 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:13.168782 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:13.174811 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:13.175268 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:14.176314 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:14.182031 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:14.182639 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:15.183809 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:15.189220 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:15.189762 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:16.221967 27116 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:16:16.227105 27913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d"
dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
mode: GRACEFUL
new_leader_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:52698
I20260709 12:16:16.227785 27913 raft_consensus.cc:606] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 LEADER]: Received request to transfer leadership to TS 88925d3120dd465bb8d39599142710b9
I20260709 12:16:16.235940 27913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c"
dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
mode: GRACEFUL
new_leader_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:52698
I20260709 12:16:16.236425 27913 raft_consensus.cc:606] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 LEADER]: Received request to transfer leadership to TS 88925d3120dd465bb8d39599142710b9
I20260709 12:16:16.246172 27838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
mode: GRACEFUL
new_leader_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:50470
I20260709 12:16:16.246645 27838 raft_consensus.cc:606] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Received request to transfer leadership to TS 88925d3120dd465bb8d39599142710b9
I20260709 12:16:16.251266 27838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e166fabf328942638baed5d4c053db85"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
mode: GRACEFUL
new_leader_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:50470
I20260709 12:16:16.251711 27838 raft_consensus.cc:606] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Received request to transfer leadership to TS 88925d3120dd465bb8d39599142710b9
I20260709 12:16:16.256124 27838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
mode: GRACEFUL
new_leader_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:50470
I20260709 12:16:16.256565 27838 raft_consensus.cc:606] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Received request to transfer leadership to TS 88925d3120dd465bb8d39599142710b9
I20260709 12:16:16.411036 28095 raft_consensus.cc:993] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b: : Instructing follower 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.411667 28099 raft_consensus.cc:1081] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Signalling peer 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.413579 27763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e166fabf328942638baed5d4c053db85"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:57312
I20260709 12:16:16.414278 27763 raft_consensus.cc:493] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:16.414645 27763 raft_consensus.cc:3060] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:16.421043 27763 raft_consensus.cc:515] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.423206 27763 leader_election.cc:290] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 3 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:16.424119 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:16.424304 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e166fabf328942638baed5d4c053db85" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:16.424899 27838 raft_consensus.cc:3055] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:16.424949 27913 raft_consensus.cc:3060] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:16.425176 27838 raft_consensus.cc:740] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:16.426066 27838 consensus_queue.cc:260] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.427140 27838 raft_consensus.cc:3060] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:16.431345 27913 raft_consensus.cc:2468] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 3.
I20260709 12:16:16.432804 27728 leader_election.cc:304] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:16.433175 27838 raft_consensus.cc:2468] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 3.
I20260709 12:16:16.433873 28097 raft_consensus.cc:2804] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:16.434860 28097 raft_consensus.cc:697] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:16.435737 28097 consensus_queue.cc:237] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.442930 27661 catalog_manager.cc:5697] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 2 to 3, leader changed from 1371160c6a5049e3b8231f1deb73844b (127.26.123.2) to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 3 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.500573 28094 raft_consensus.cc:993] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914: : Instructing follower 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.501111 28098 raft_consensus.cc:1081] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 LEADER]: Signalling peer 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.502511 27763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:57306
I20260709 12:16:16.502988 27763 raft_consensus.cc:493] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:16.503276 27763 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:16.508755 27763 raft_consensus.cc:515] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.510424 27763 leader_election.cc:290] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 3 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:16.511281 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:16.511457 27913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:16.511765 27838 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:16.511987 27913 raft_consensus.cc:3055] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:16.512260 27913 raft_consensus.cc:740] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Running, Role: LEADER
I20260709 12:16:16.512903 27913 consensus_queue.cc:260] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.513880 27913 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:16.516647 27838 raft_consensus.cc:2468] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 3.
I20260709 12:16:16.517839 27729 leader_election.cc:304] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:16.518658 28097 raft_consensus.cc:2804] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:16.518929 27913 raft_consensus.cc:2468] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 3.
I20260709 12:16:16.519176 28097 raft_consensus.cc:697] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:16.520210 28097 consensus_queue.cc:237] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.527377 27661 catalog_manager.cc:5697] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 2 to 3, leader changed from 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3) to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 3 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.594659 28095 raft_consensus.cc:993] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b: : Instructing follower 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.595093 28099 raft_consensus.cc:1081] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Signalling peer 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.596433 27763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:57312
I20260709 12:16:16.596930 27763 raft_consensus.cc:493] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:16.597262 27763 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:16.602802 27763 raft_consensus.cc:515] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.604645 27763 leader_election.cc:290] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 3 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:16.605876 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:16.606066 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:16.606575 27838 raft_consensus.cc:3055] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:16.606667 27912 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:16.606962 27838 raft_consensus.cc:740] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:16.607898 27838 consensus_queue.cc:260] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.609176 27838 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:16.614828 27912 raft_consensus.cc:2468] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 3.
I20260709 12:16:16.615291 27838 raft_consensus.cc:2468] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 3.
I20260709 12:16:16.616269 27728 leader_election.cc:304] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:16.617026 28097 raft_consensus.cc:2804] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:16.617431 28097 raft_consensus.cc:697] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:16.618228 28097 consensus_queue.cc:237] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.626019 27661 catalog_manager.cc:5697] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 2 to 3, leader changed from 1371160c6a5049e3b8231f1deb73844b (127.26.123.2) to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 3 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.633229 28094 raft_consensus.cc:993] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914: : Instructing follower 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.633714 28098 raft_consensus.cc:1081] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 LEADER]: Signalling peer 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.635155 27763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:57306
I20260709 12:16:16.635620 27763 raft_consensus.cc:493] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:16.635911 27763 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:16.641141 27763 raft_consensus.cc:515] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:16.642831 27763 leader_election.cc:290] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 4 election: Requested vote from peers 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:16.643754 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:16.643852 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f8ca86217b481caf7179dd6b1b2c8d" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:16.644405 27912 raft_consensus.cc:3055] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:16.644479 27838 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:16.644690 27912 raft_consensus.cc:740] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Running, Role: LEADER
I20260709 12:16:16.645435 27912 consensus_queue.cc:260] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:16.646498 27912 raft_consensus.cc:3060] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:16.649705 27838 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 4.
I20260709 12:16:16.650625 27729 leader_election.cc:304] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:16.651275 28097 raft_consensus.cc:2804] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:16.651758 28097 raft_consensus.cc:697] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 4 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:16.652127 27912 raft_consensus.cc:2468] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 4.
I20260709 12:16:16.652673 28097 consensus_queue.cc:237] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } }
I20260709 12:16:16.660059 27661 catalog_manager.cc:5697] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 3 to 4, leader changed from 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3) to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 4 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:16.679801 28095 raft_consensus.cc:993] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b: : Instructing follower 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.680265 28099 raft_consensus.cc:1081] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Signalling peer 88925d3120dd465bb8d39599142710b9 to start an election
I20260709 12:16:16.681584 27763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
 from {username='slave'} at 127.0.0.1:57312
I20260709 12:16:16.682092 27763 raft_consensus.cc:493] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:16.682386 27763 raft_consensus.cc:3060] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:16.687611 27763 raft_consensus.cc:515] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.689216 27763 leader_election.cc:290] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 2 election: Requested vote from peers 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:16.690160 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:16.690303 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "88925d3120dd465bb8d39599142710b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:16.690625 27838 raft_consensus.cc:3055] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:16.690917 27838 raft_consensus.cc:740] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:16.690897 27912 raft_consensus.cc:3060] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:16.691725 27838 consensus_queue.cc:260] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.692619 27838 raft_consensus.cc:3060] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:16.696413 27912 raft_consensus.cc:2468] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 2.
I20260709 12:16:16.697042 27838 raft_consensus.cc:2468] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88925d3120dd465bb8d39599142710b9 in term 2.
I20260709 12:16:16.697546 27728 leader_election.cc:304] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1eaae7b7eb49fb954d0e5cfeae1914, 88925d3120dd465bb8d39599142710b9; no voters: 
I20260709 12:16:16.698524 28100 raft_consensus.cc:2804] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:16.698977 28100 raft_consensus.cc:697] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Becoming Leader. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:16.699860 28100 consensus_queue.cc:237] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:16.706828 27661 catalog_manager.cc:5697] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 reported cstate change: term changed from 1 to 2, leader changed from 1371160c6a5049e3b8231f1deb73844b (127.26.123.2) to 88925d3120dd465bb8d39599142710b9 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "88925d3120dd465bb8d39599142710b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:16.828009 27912 raft_consensus.cc:1275] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:16.829550 28100 consensus_queue.cc:1048] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:16.841712 27838 raft_consensus.cc:1275] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:16.843505 28112 consensus_queue.cc:1048] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:16.965070 27912 raft_consensus.cc:1275] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:16.966367 28112 consensus_queue.cc:1048] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:16.975688 27838 raft_consensus.cc:1275] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:16.977537 28100 consensus_queue.cc:1048] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:17.068574 27912 raft_consensus.cc:1275] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:17.070071 28112 consensus_queue.cc:1048] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:17.072429 27838 raft_consensus.cc:1275] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:17.074226 28112 consensus_queue.cc:1048] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:17.088814 27912 raft_consensus.cc:1275] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:17.090376 28112 consensus_queue.cc:1048] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:17.101665 27838 raft_consensus.cc:1275] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:17.103569 28100 consensus_queue.cc:1048] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:17.268065 27839 raft_consensus.cc:1275] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:17.269496 28100 consensus_queue.cc:1048] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:17.278702 27912 raft_consensus.cc:1275] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Refusing update from remote peer 88925d3120dd465bb8d39599142710b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:17.280434 28112 consensus_queue.cc:1048] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:19.299852 27116 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:16:19.300807 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:19.310009 27763 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
mode: GRACEFUL
new_leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:48134
I20260709 12:16:19.310722 27763 raft_consensus.cc:606] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Received request to transfer leadership to TS 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:19.313611 27763 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
mode: GRACEFUL
new_leader_uuid: "1371160c6a5049e3b8231f1deb73844b"
 from {username='slave'} at 127.0.0.1:48134
I20260709 12:16:19.314217 27763 raft_consensus.cc:606] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Received request to transfer leadership to TS 1371160c6a5049e3b8231f1deb73844b
I20260709 12:16:19.317442 27763 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
mode: GRACEFUL
new_leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:48134
I20260709 12:16:19.318229 27763 raft_consensus.cc:606] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Received request to transfer leadership to TS 1c1eaae7b7eb49fb954d0e5cfeae1914
I20260709 12:16:19.321426 27763 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586"
dest_uuid: "88925d3120dd465bb8d39599142710b9"
mode: GRACEFUL
new_leader_uuid: "1371160c6a5049e3b8231f1deb73844b"
 from {username='slave'} at 127.0.0.1:48134
I20260709 12:16:19.322067 27763 raft_consensus.cc:606] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Received request to transfer leadership to TS 1371160c6a5049e3b8231f1deb73844b
I20260709 12:16:19.323251 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:16:19.323786 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:19.428761 28124 raft_consensus.cc:993] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9: : Instructing follower 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:19.429303 28097 raft_consensus.cc:1081] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Signalling peer 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:19.430768 27912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c"
dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:52694
I20260709 12:16:19.431216 27912 raft_consensus.cc:493] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:19.431494 27912 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:19.436770 27912 raft_consensus.cc:515] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.438727 27912 leader_election.cc:290] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 4 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:19.439975 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:19.439975 27763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b99723e1f56a47cd9762f41e71061c4c" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:19.440771 27838 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:19.440829 27763 raft_consensus.cc:3055] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:19.441218 27763 raft_consensus.cc:740] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:19.442098 27763 consensus_queue.cc:260] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.443397 27763 raft_consensus.cc:3060] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:19.448971 27838 raft_consensus.cc:2468] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 4.
I20260709 12:16:19.450448 27878 leader_election.cc:304] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:19.450834 27763 raft_consensus.cc:2468] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 4.
I20260709 12:16:19.451148 28098 raft_consensus.cc:2804] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:19.452392 28098 raft_consensus.cc:697] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 LEADER]: Becoming Leader. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Running, Role: LEADER
I20260709 12:16:19.453204 28098 consensus_queue.cc:237] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.460650 27661 catalog_manager.cc:5697] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 reported cstate change: term changed from 3 to 4, leader changed from 88925d3120dd465bb8d39599142710b9 (127.26.123.1) to 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3). New cstate: current_term: 4 leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.521801 28124 raft_consensus.cc:993] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9: : Instructing follower 1371160c6a5049e3b8231f1deb73844b to start an election
I20260709 12:16:19.522400 28097 raft_consensus.cc:1081] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Signalling peer 1371160c6a5049e3b8231f1deb73844b to start an election
I20260709 12:16:19.524067 27838 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
 from {username='slave'} at 127.0.0.1:50464
I20260709 12:16:19.524526 27838 raft_consensus.cc:493] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:19.524785 27838 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:19.530010 27838 raft_consensus.cc:515] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.532132 27838 leader_election.cc:290] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 4 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:19.532999 27763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:19.533361 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ab913080d2442e9f14f6c35f480620" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:19.533591 27763 raft_consensus.cc:3055] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:19.533915 27912 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:19.533944 27763 raft_consensus.cc:740] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:19.534799 27763 consensus_queue.cc:260] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.536027 27763 raft_consensus.cc:3060] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:19.539256 27912 raft_consensus.cc:2468] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 4.
I20260709 12:16:19.540573 27803 leader_election.cc:304] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:19.541411 28099 raft_consensus.cc:2804] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:19.541652 27763 raft_consensus.cc:2468] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 4.
I20260709 12:16:19.542855 28099 raft_consensus.cc:697] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 4 LEADER]: Becoming Leader. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:19.543759 28099 consensus_queue.cc:237] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.551177 27661 catalog_manager.cc:5697] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b reported cstate change: term changed from 3 to 4, leader changed from 88925d3120dd465bb8d39599142710b9 (127.26.123.1) to 1371160c6a5049e3b8231f1deb73844b (127.26.123.2). New cstate: current_term: 4 leader_uuid: "1371160c6a5049e3b8231f1deb73844b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:19.594465 28124 raft_consensus.cc:993] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9: : Instructing follower 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:19.594974 28097 raft_consensus.cc:1081] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Signalling peer 1c1eaae7b7eb49fb954d0e5cfeae1914 to start an election
I20260709 12:16:19.596642 27912 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed"
dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
 from {username='slave'} at 127.0.0.1:52694
I20260709 12:16:19.597122 27912 raft_consensus.cc:493] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:19.597384 27912 raft_consensus.cc:3060] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:19.602470 27912 raft_consensus.cc:515] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.604147 27912 leader_election.cc:290] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 3 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387)
I20260709 12:16:19.605058 27763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:19.605376 27838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6acb9f89fc3646adb2ab1ec2460465ed" candidate_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
I20260709 12:16:19.605675 27763 raft_consensus.cc:3055] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:19.605906 27838 raft_consensus.cc:3060] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:19.606015 27763 raft_consensus.cc:740] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:19.606699 27763 consensus_queue.cc:260] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.607621 27763 raft_consensus.cc:3060] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:19.611263 27838 raft_consensus.cc:2468] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 3.
I20260709 12:16:19.612486 27878 leader_election.cc:304] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:19.612857 27763 raft_consensus.cc:2468] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1eaae7b7eb49fb954d0e5cfeae1914 in term 3.
I20260709 12:16:19.613389 28098 raft_consensus.cc:2804] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:19.613878 28098 raft_consensus.cc:697] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 LEADER]: Becoming Leader. State: Replica: 1c1eaae7b7eb49fb954d0e5cfeae1914, State: Running, Role: LEADER
I20260709 12:16:19.614715 28098 consensus_queue.cc:237] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.622190 27661 catalog_manager.cc:5697] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 reported cstate change: term changed from 2 to 3, leader changed from 88925d3120dd465bb8d39599142710b9 (127.26.123.1) to 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3). New cstate: current_term: 3 leader_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.853837 28124 raft_consensus.cc:993] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9: : Instructing follower 1371160c6a5049e3b8231f1deb73844b to start an election
I20260709 12:16:19.854596 28097 raft_consensus.cc:1081] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Signalling peer 1371160c6a5049e3b8231f1deb73844b to start an election
I20260709 12:16:19.856302 27838 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586"
dest_uuid: "1371160c6a5049e3b8231f1deb73844b"
 from {username='slave'} at 127.0.0.1:50464
I20260709 12:16:19.856914 27838 raft_consensus.cc:493] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:19.857231 27838 raft_consensus.cc:3060] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:19.865123 27838 raft_consensus.cc:515] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.866981 27838 leader_election.cc:290] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 2 election: Requested vote from peers 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347), 1c1eaae7b7eb49fb954d0e5cfeae1914 (127.26.123.3:36647)
I20260709 12:16:19.868539 27763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88925d3120dd465bb8d39599142710b9"
I20260709 12:16:19.868516 27912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f2bac20394d3ea0849035ed450586" candidate_uuid: "1371160c6a5049e3b8231f1deb73844b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914"
I20260709 12:16:19.869479 27763 raft_consensus.cc:3055] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:19.869563 27912 raft_consensus.cc:3060] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:19.869902 27763 raft_consensus.cc:740] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88925d3120dd465bb8d39599142710b9, State: Running, Role: LEADER
I20260709 12:16:19.870868 27763 consensus_queue.cc:260] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.872701 27763 raft_consensus.cc:3060] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:19.879269 27912 raft_consensus.cc:2468] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 2.
I20260709 12:16:19.880923 27803 leader_election.cc:304] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1371160c6a5049e3b8231f1deb73844b, 1c1eaae7b7eb49fb954d0e5cfeae1914; no voters: 
I20260709 12:16:19.881326 27763 raft_consensus.cc:2468] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1371160c6a5049e3b8231f1deb73844b in term 2.
I20260709 12:16:19.882145 28099 raft_consensus.cc:2804] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:19.882669 28099 raft_consensus.cc:697] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Becoming Leader. State: Replica: 1371160c6a5049e3b8231f1deb73844b, State: Running, Role: LEADER
I20260709 12:16:19.883646 28099 consensus_queue.cc:237] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } }
I20260709 12:16:19.892798 27661 catalog_manager.cc:5697] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b reported cstate change: term changed from 1 to 2, leader changed from 88925d3120dd465bb8d39599142710b9 (127.26.123.1) to 1371160c6a5049e3b8231f1deb73844b (127.26.123.2). New cstate: current_term: 2 leader_uuid: "1371160c6a5049e3b8231f1deb73844b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:19.940021 27763 raft_consensus.cc:1275] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:19.941439 28098 consensus_queue.cc:1048] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:19.954836 27838 raft_consensus.cc:1275] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:19.956801 28125 consensus_queue.cc:1048] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:20.003439 27912 raft_consensus.cc:1275] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Refusing update from remote peer 1371160c6a5049e3b8231f1deb73844b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:20.004752 28099 consensus_queue.cc:1048] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:20.008496 27763 raft_consensus.cc:1275] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Refusing update from remote peer 1371160c6a5049e3b8231f1deb73844b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:20.009878 28099 consensus_queue.cc:1048] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:20.197389 27838 raft_consensus.cc:1275] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:20.198750 28138 consensus_queue.cc:1048] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1371160c6a5049e3b8231f1deb73844b" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:20.211512 27763 raft_consensus.cc:1275] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Refusing update from remote peer 1c1eaae7b7eb49fb954d0e5cfeae1914: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:20.213184 28098 consensus_queue.cc:1048] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:20.325125 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:20.328449 27763 raft_consensus.cc:1275] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Refusing update from remote peer 1371160c6a5049e3b8231f1deb73844b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:20.330108 28131 consensus_queue.cc:1048] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "88925d3120dd465bb8d39599142710b9" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:20.332973 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:20.333544 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:20.341259 27913 raft_consensus.cc:1275] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Refusing update from remote peer 1371160c6a5049e3b8231f1deb73844b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:20.343189 28143 consensus_queue.cc:1048] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1eaae7b7eb49fb954d0e5cfeae1914" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:21.335064 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:21.341185 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:21.341639 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:22.342666 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:22.348546 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:22.348989 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:23.350028 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:23.355772 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:23.356235 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:24.357918 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:24.364065 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:24.364622 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:25.365769 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:25.372226 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:25.372776 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:26.375250 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:26.397948 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:26.398993 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:27.400662 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:27.407730 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:27.408314 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:28.409343 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:28.416222 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:28.416687 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:29.453222 27116 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:16:29.453894 27116 tablet_server.cc:179] TabletServer@127.26.123.1:0 shutting down...
W20260709 12:16:29.479944 27878 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:29.489861 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:29.490760 27116 tablet_replica.cc:333] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
W20260709 12:16:29.490811 27878 consensus_peers.cc:597] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 -> Peer 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347): Couldn't send request to peer 88925d3120dd465bb8d39599142710b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:29.491588 27116 raft_consensus.cc:2243] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.492516 27116 raft_consensus.cc:2272] T 0ce6d589c5d845c0adbbbad521d9733a P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.494747 27116 tablet_replica.cc:333] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
I20260709 12:16:29.495380 27116 raft_consensus.cc:2243] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.496258 27116 raft_consensus.cc:2272] T e166fabf328942638baed5d4c053db85 P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.498471 27116 tablet_replica.cc:333] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
I20260709 12:16:29.499058 27116 raft_consensus.cc:2243] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.499902 27116 raft_consensus.cc:2272] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.502178 27116 tablet_replica.cc:333] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
I20260709 12:16:29.502810 27116 raft_consensus.cc:2243] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.503297 27116 raft_consensus.cc:2272] T b99723e1f56a47cd9762f41e71061c4c P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.505856 27116 tablet_replica.cc:333] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
I20260709 12:16:29.506605 27116 raft_consensus.cc:2243] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.507047 27116 raft_consensus.cc:2272] T ff1f2bac20394d3ea0849035ed450586 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:29.507508 27878 consensus_peers.cc:597] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 -> Peer 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347): Couldn't send request to peer 88925d3120dd465bb8d39599142710b9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:29.508934 27116 tablet_replica.cc:333] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
I20260709 12:16:29.509514 27116 raft_consensus.cc:2243] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.509971 27116 raft_consensus.cc:2272] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 88925d3120dd465bb8d39599142710b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.511969 27116 tablet_replica.cc:333] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
I20260709 12:16:29.512486 27116 raft_consensus.cc:2243] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.513310 27116 raft_consensus.cc:2272] T 124c98e6d8f14ad4a19fb65b95779dc7 P 88925d3120dd465bb8d39599142710b9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.515189 27116 tablet_replica.cc:333] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
I20260709 12:16:29.515944 27116 raft_consensus.cc:2243] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.516551 27116 raft_consensus.cc:2272] T 6acb9f89fc3646adb2ab1ec2460465ed P 88925d3120dd465bb8d39599142710b9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.518827 27116 tablet_replica.cc:333] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
I20260709 12:16:29.519369 27116 raft_consensus.cc:2243] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.519801 27116 raft_consensus.cc:2272] T 93ab913080d2442e9f14f6c35f480620 P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.521620 27116 tablet_replica.cc:333] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9: stopping tablet replica
I20260709 12:16:29.522149 27116 raft_consensus.cc:2243] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.522996 27116 raft_consensus.cc:2272] T c7f8ca86217b481caf7179dd6b1b2c8d P 88925d3120dd465bb8d39599142710b9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.556562 27116 tablet_server.cc:196] TabletServer@127.26.123.1:0 shutdown complete.
I20260709 12:16:29.581034 27116 tablet_server.cc:179] TabletServer@127.26.123.2:0 shutting down...
I20260709 12:16:29.608457 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:29.609192 27116 tablet_replica.cc:333] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
I20260709 12:16:29.610157 27116 raft_consensus.cc:2243] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.610821 27116 raft_consensus.cc:2272] T e166fabf328942638baed5d4c053db85 P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.613483 27116 tablet_replica.cc:333] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
I20260709 12:16:29.614243 27116 raft_consensus.cc:2243] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.614811 27116 raft_consensus.cc:2272] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:29.615768 27804 consensus_peers.cc:597] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b -> Peer 88925d3120dd465bb8d39599142710b9 (127.26.123.1:43347): Couldn't send request to peer 88925d3120dd465bb8d39599142710b9. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:29.617365 27116 tablet_replica.cc:333] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
I20260709 12:16:29.618006 27116 raft_consensus.cc:2243] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.618439 27116 raft_consensus.cc:2272] T b99723e1f56a47cd9762f41e71061c4c P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.620373 27116 tablet_replica.cc:333] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
I20260709 12:16:29.620913 27116 raft_consensus.cc:2243] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.621702 27116 raft_consensus.cc:2272] T 93ab913080d2442e9f14f6c35f480620 P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.623593 27116 tablet_replica.cc:333] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
I20260709 12:16:29.624076 27116 raft_consensus.cc:2243] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.624511 27116 raft_consensus.cc:2272] T c7f8ca86217b481caf7179dd6b1b2c8d P 1371160c6a5049e3b8231f1deb73844b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.626406 27116 tablet_replica.cc:333] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
I20260709 12:16:29.626881 27116 raft_consensus.cc:2243] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.627295 27116 raft_consensus.cc:2272] T 6acb9f89fc3646adb2ab1ec2460465ed P 1371160c6a5049e3b8231f1deb73844b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.628904 27116 tablet_replica.cc:333] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
I20260709 12:16:29.629443 27116 raft_consensus.cc:2243] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.630317 27116 raft_consensus.cc:2272] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1371160c6a5049e3b8231f1deb73844b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.632427 27116 tablet_replica.cc:333] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
I20260709 12:16:29.632973 27116 raft_consensus.cc:2243] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.633409 27116 raft_consensus.cc:2272] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.635372 27116 tablet_replica.cc:333] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
I20260709 12:16:29.636032 27116 raft_consensus.cc:2243] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.636502 27116 raft_consensus.cc:2272] T 0ce6d589c5d845c0adbbbad521d9733a P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.639037 27116 tablet_replica.cc:333] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b: stopping tablet replica
W20260709 12:16:29.639070 27878 consensus_peers.cc:597] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 -> Peer 1371160c6a5049e3b8231f1deb73844b (127.26.123.2:42387): Couldn't send request to peer 1371160c6a5049e3b8231f1deb73844b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:29.639755 27116 raft_consensus.cc:2243] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.640530 27116 raft_consensus.cc:2272] T ff1f2bac20394d3ea0849035ed450586 P 1371160c6a5049e3b8231f1deb73844b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.677668 27116 tablet_server.cc:196] TabletServer@127.26.123.2:0 shutdown complete.
I20260709 12:16:29.714543 27116 tablet_server.cc:179] TabletServer@127.26.123.3:0 shutting down...
I20260709 12:16:29.741459 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:29.742488 27116 tablet_replica.cc:333] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.743611 27116 raft_consensus.cc:2243] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.744709 27116 raft_consensus.cc:2272] T 124c98e6d8f14ad4a19fb65b95779dc7 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.747824 27116 tablet_replica.cc:333] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.748822 27116 raft_consensus.cc:2243] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.749424 27116 raft_consensus.cc:2272] T a2d32c67cfb44ffdb4ae9774e8a4d71b P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.751756 27116 tablet_replica.cc:333] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.752475 27116 raft_consensus.cc:2243] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.753121 27116 raft_consensus.cc:2272] T ff1f2bac20394d3ea0849035ed450586 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.756641 27116 tablet_replica.cc:333] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.757558 27116 raft_consensus.cc:2243] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.758720 27116 raft_consensus.cc:2272] T 0ce6d589c5d845c0adbbbad521d9733a P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.761976 27116 tablet_replica.cc:333] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.763065 27116 raft_consensus.cc:2243] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.765393 27116 raft_consensus.cc:2272] T 6acb9f89fc3646adb2ab1ec2460465ed P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.769353 27116 tablet_replica.cc:333] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.770334 27116 raft_consensus.cc:2243] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.771025 27116 raft_consensus.cc:2272] T 93ab913080d2442e9f14f6c35f480620 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.773469 27116 tablet_replica.cc:333] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.774070 27116 raft_consensus.cc:2243] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.774557 27116 raft_consensus.cc:2272] T c7f8ca86217b481caf7179dd6b1b2c8d P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.776625 27116 tablet_replica.cc:333] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.777161 27116 raft_consensus.cc:2243] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.778546 27116 raft_consensus.cc:2272] T 5bf0f32ff2c6416e868adb7ddfe6dfa3 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.780625 27116 tablet_replica.cc:333] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.781157 27116 raft_consensus.cc:2243] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.781703 27116 raft_consensus.cc:2272] T e166fabf328942638baed5d4c053db85 P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.783669 27116 tablet_replica.cc:333] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914: stopping tablet replica
I20260709 12:16:29.784125 27116 raft_consensus.cc:2243] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.785182 27116 raft_consensus.cc:2272] T b99723e1f56a47cd9762f41e71061c4c P 1c1eaae7b7eb49fb954d0e5cfeae1914 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.818279 27116 tablet_server.cc:196] TabletServer@127.26.123.3:0 shutdown complete.
I20260709 12:16:29.843075 27116 master.cc:562] Master@127.26.123.62:45905 shutting down...
I20260709 12:16:29.866461 27116 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:29.867274 27116 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.867636 27116 tablet_replica.cc:333] T 00000000000000000000000000000000 P 89fd35d3a02546d2bb0785515f93eb89: stopping tablet replica
I20260709 12:16:29.888960 27116 master.cc:584] Master@127.26.123.62:45905 shutdown complete.
1371160c6a5049e3b8231f1deb73844b  4
1c1eaae7b7eb49fb954d0e5cfeae1914  1
88925d3120dd465bb8d39599142710b9  5
1371160c6a5049e3b8231f1deb73844b  4
1c1eaae7b7eb49fb954d0e5cfeae1914  3
88925d3120dd465bb8d39599142710b9  3
1371160c6a5049e3b8231f1deb73844b  1
1c1eaae7b7eb49fb954d0e5cfeae1914  1
88925d3120dd465bb8d39599142710b9  8
1371160c6a5049e3b8231f1deb73844b  3
1c1eaae7b7eb49fb954d0e5cfeae1914  3
88925d3120dd465bb8d39599142710b9  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (29116 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:16:29.937429 27116 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.123.62:36149
I20260709 12:16:29.938699 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:29.944716 28165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:29.944830 28166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:29.946270 28168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:29.947372 27116 server_base.cc:1061] running on GCE node
I20260709 12:16:29.948241 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:29.948441 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:29.948559 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599389948548 us; error 0 us; skew 500 ppm
I20260709 12:16:29.949076 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:29.951716 27116 webserver.cc:533] Webserver started at http://127.26.123.62:44127/ using document root <none> and password file <none>
I20260709 12:16:29.952214 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:29.952383 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:29.952634 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:29.954041 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/master-0-root/instance:
uuid: "b5f297f7df3d4c2da68fabf56f1654bd"
format_stamp: "Formatted at 2026-07-09 12:16:29 on dist-test-slave-9q7n"
I20260709 12:16:29.958835 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:16:29.962677 28173 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:29.963477 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 12:16:29.963771 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/master-0-root
uuid: "b5f297f7df3d4c2da68fabf56f1654bd"
format_stamp: "Formatted at 2026-07-09 12:16:29 on dist-test-slave-9q7n"
I20260709 12:16:29.964066 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:29.976948 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:29.978353 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:30.023910 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.62:36149
I20260709 12:16:30.024006 28224 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.62:36149 every 8 connection(s)
I20260709 12:16:30.028203 28225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.039669 28225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Bootstrap starting.
I20260709 12:16:30.044993 28225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:30.049806 28225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: No bootstrap required, opened a new log
I20260709 12:16:30.052544 28225 raft_consensus.cc:359] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f297f7df3d4c2da68fabf56f1654bd" member_type: VOTER }
I20260709 12:16:30.053115 28225 raft_consensus.cc:385] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:30.053340 28225 raft_consensus.cc:740] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f297f7df3d4c2da68fabf56f1654bd, State: Initialized, Role: FOLLOWER
I20260709 12:16:30.054000 28225 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f297f7df3d4c2da68fabf56f1654bd" member_type: VOTER }
I20260709 12:16:30.054500 28225 raft_consensus.cc:399] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:30.054746 28225 raft_consensus.cc:493] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:30.055024 28225 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:30.063932 28225 raft_consensus.cc:515] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f297f7df3d4c2da68fabf56f1654bd" member_type: VOTER }
I20260709 12:16:30.064921 28225 leader_election.cc:304] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5f297f7df3d4c2da68fabf56f1654bd; no voters: 
I20260709 12:16:30.067349 28225 leader_election.cc:290] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:30.068037 28228 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:30.070890 28228 raft_consensus.cc:697] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 1 LEADER]: Becoming Leader. State: Replica: b5f297f7df3d4c2da68fabf56f1654bd, State: Running, Role: LEADER
I20260709 12:16:30.072386 28228 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f297f7df3d4c2da68fabf56f1654bd" member_type: VOTER }
I20260709 12:16:30.073060 28225 sys_catalog.cc:565] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:30.085495 28229 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5f297f7df3d4c2da68fabf56f1654bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f297f7df3d4c2da68fabf56f1654bd" member_type: VOTER } }
I20260709 12:16:30.085377 28230 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5f297f7df3d4c2da68fabf56f1654bd. Latest consensus state: current_term: 1 leader_uuid: "b5f297f7df3d4c2da68fabf56f1654bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5f297f7df3d4c2da68fabf56f1654bd" member_type: VOTER } }
I20260709 12:16:30.086547 28229 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:30.086735 28230 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:30.092840 28236 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:30.101104 28236 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:30.102257 27116 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:30.111488 28236 catalog_manager.cc:1383] Generated new cluster ID: f0a2ce2be30840ac9219e615425a0aba
I20260709 12:16:30.111851 28236 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:30.134699 28236 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:30.136129 28236 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:30.152424 28236 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Generated new TSK 0
I20260709 12:16:30.153120 28236 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:30.168962 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:30.176698 28246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:30.178943 28247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:30.179625 28249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:30.179840 27116 server_base.cc:1061] running on GCE node
I20260709 12:16:30.180804 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:30.180995 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:30.181128 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599390181105 us; error 0 us; skew 500 ppm
I20260709 12:16:30.181653 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:30.184121 27116 webserver.cc:533] Webserver started at http://127.26.123.1:40301/ using document root <none> and password file <none>
I20260709 12:16:30.184703 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:30.184993 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:30.185326 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:30.186645 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/instance:
uuid: "a3de34b6f4cf4606a61944db6e2739c8"
format_stamp: "Formatted at 2026-07-09 12:16:30 on dist-test-slave-9q7n"
I20260709 12:16:30.191643 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:16:30.195433 28254 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:30.196215 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:16:30.196521 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "a3de34b6f4cf4606a61944db6e2739c8"
format_stamp: "Formatted at 2026-07-09 12:16:30 on dist-test-slave-9q7n"
I20260709 12:16:30.196811 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:30.220638 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:30.222107 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:30.223732 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:30.226243 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:30.226447 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:30.226692 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:30.226861 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:30.273561 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:42113
I20260709 12:16:30.273684 28316 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:42113 every 8 connection(s)
I20260709 12:16:30.279119 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:30.287273 28321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:30.288053 28322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:30.291397 28324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:30.292656 27116 server_base.cc:1061] running on GCE node
I20260709 12:16:30.293093 28317 heartbeater.cc:344] Connected to a master server at 127.26.123.62:36149
I20260709 12:16:30.293493 28317 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:30.293917 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:30.294139 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:30.294323 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599390294302 us; error 0 us; skew 500 ppm
I20260709 12:16:30.294346 28317 heartbeater.cc:507] Master 127.26.123.62:36149 requested a full tablet report, sending...
I20260709 12:16:30.295178 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:30.296979 28190 ts_manager.cc:194] Registered new tserver with Master: a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:16:30.298934 28190 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43788
I20260709 12:16:30.301937 27116 webserver.cc:533] Webserver started at http://127.26.123.2:40135/ using document root <none> and password file <none>
I20260709 12:16:30.302518 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:30.302717 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:30.302984 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:30.304262 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/instance:
uuid: "6634fa3fbd594726a5ec3c1686900284"
format_stamp: "Formatted at 2026-07-09 12:16:30 on dist-test-slave-9q7n"
I20260709 12:16:30.308954 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:30.312624 28329 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:30.313349 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:30.313637 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root
uuid: "6634fa3fbd594726a5ec3c1686900284"
format_stamp: "Formatted at 2026-07-09 12:16:30 on dist-test-slave-9q7n"
I20260709 12:16:30.313971 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:30.336434 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:30.337828 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:30.339462 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:30.342828 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:30.343111 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:30.343410 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:30.343576 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:30.387755 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.2:44701
I20260709 12:16:30.387869 28391 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.2:44701 every 8 connection(s)
I20260709 12:16:30.392761 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:30.401641 28395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:30.401716 28396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:30.405268 27116 server_base.cc:1061] running on GCE node
I20260709 12:16:30.405854 28392 heartbeater.cc:344] Connected to a master server at 127.26.123.62:36149
I20260709 12:16:30.406221 28392 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:30.406970 28392 heartbeater.cc:507] Master 127.26.123.62:36149 requested a full tablet report, sending...
W20260709 12:16:30.407056 28398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:30.408080 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:30.408293 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:30.408466 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599390408443 us; error 0 us; skew 500 ppm
I20260709 12:16:30.409057 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:30.409036 28190 ts_manager.cc:194] Registered new tserver with Master: 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:30.410902 28190 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43804
I20260709 12:16:30.411991 27116 webserver.cc:533] Webserver started at http://127.26.123.3:45931/ using document root <none> and password file <none>
I20260709 12:16:30.412535 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:30.412729 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:30.412999 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:30.414358 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/instance:
uuid: "9aea2391841b4888b4e195e5a1f4623b"
format_stamp: "Formatted at 2026-07-09 12:16:30 on dist-test-slave-9q7n"
I20260709 12:16:30.418987 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:16:30.422634 28403 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:30.423408 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:16:30.423712 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root
uuid: "9aea2391841b4888b4e195e5a1f4623b"
format_stamp: "Formatted at 2026-07-09 12:16:30 on dist-test-slave-9q7n"
I20260709 12:16:30.424012 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:30.436365 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:30.437911 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:30.439492 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:30.441977 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:30.442209 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:30.442440 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:30.442596 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:30.487216 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.3:33721
I20260709 12:16:30.487334 28465 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.3:33721 every 8 connection(s)
I20260709 12:16:30.501366 28466 heartbeater.cc:344] Connected to a master server at 127.26.123.62:36149
I20260709 12:16:30.501811 28466 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:30.502686 28466 heartbeater.cc:507] Master 127.26.123.62:36149 requested a full tablet report, sending...
I20260709 12:16:30.504703 28190 ts_manager.cc:194] Registered new tserver with Master: 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:30.504933 27116 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014442809s
I20260709 12:16:30.506645 28190 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43818
I20260709 12:16:30.533638 28190 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43832:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:30.540187 28190 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:30.778223 28357 tablet_service.cc:1511] Processing CreateTablet for tablet 23d6b97dd9e9466d9581f6088e681a11 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:30.779803 28357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23d6b97dd9e9466d9581f6088e681a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.788693 28353 tablet_service.cc:1511] Processing CreateTablet for tablet 21eeb096d8a4423bbc24e8935e0539fc (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:30.788844 28356 tablet_service.cc:1511] Processing CreateTablet for tablet c474098832a04762b58521bdc2a92b5a (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:30.790280 28353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21eeb096d8a4423bbc24e8935e0539fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.790828 28354 tablet_service.cc:1511] Processing CreateTablet for tablet 9c99f01ab7374acfa37f7669b8fcdeb2 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:30.792788 28355 tablet_service.cc:1511] Processing CreateTablet for tablet 9cdd1e39ef1841c382e814af04ea5bc9 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:30.797073 28431 tablet_service.cc:1511] Processing CreateTablet for tablet 23d6b97dd9e9466d9581f6088e681a11 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:30.797797 28351 tablet_service.cc:1511] Processing CreateTablet for tablet 1fc9bc911cb145b0a259ee6481eefdf6 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:30.798460 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23d6b97dd9e9466d9581f6088e681a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.798110 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 9c99f01ab7374acfa37f7669b8fcdeb2 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:30.799427 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c99f01ab7374acfa37f7669b8fcdeb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.799772 28350 tablet_service.cc:1511] Processing CreateTablet for tablet 7f380821c5264bfc8a5f0bf36bc52fd4 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:30.800010 28430 tablet_service.cc:1511] Processing CreateTablet for tablet c474098832a04762b58521bdc2a92b5a (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:30.800586 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 9cdd1e39ef1841c382e814af04ea5bc9 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:30.801278 28430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c474098832a04762b58521bdc2a92b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.801927 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdd1e39ef1841c382e814af04ea5bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.802994 28352 tablet_service.cc:1511] Processing CreateTablet for tablet 2e3e7c85ff90423aa9f9d234d992379b (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:30.803354 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 21eeb096d8a4423bbc24e8935e0539fc (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:30.803779 28349 tablet_service.cc:1511] Processing CreateTablet for tablet ab6d72f1fb474adfafdd159090dc63b5 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:30.804706 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21eeb096d8a4423bbc24e8935e0539fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.805125 28354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c99f01ab7374acfa37f7669b8fcdeb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.807014 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 2e3e7c85ff90423aa9f9d234d992379b (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:30.808341 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3e7c85ff90423aa9f9d234d992379b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.810801 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 1fc9bc911cb145b0a259ee6481eefdf6 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:30.812242 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fc9bc911cb145b0a259ee6481eefdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.815029 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 7f380821c5264bfc8a5f0bf36bc52fd4 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:30.816406 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f380821c5264bfc8a5f0bf36bc52fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.819083 28423 tablet_service.cc:1511] Processing CreateTablet for tablet ab6d72f1fb474adfafdd159090dc63b5 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:30.820520 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6d72f1fb474adfafdd159090dc63b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.823535 28348 tablet_service.cc:1511] Processing CreateTablet for tablet cdc4169a890e44e3ad14a98135e7071e (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:30.811273 28352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3e7c85ff90423aa9f9d234d992379b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.831179 28348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc4169a890e44e3ad14a98135e7071e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.832044 28422 tablet_service.cc:1511] Processing CreateTablet for tablet cdc4169a890e44e3ad14a98135e7071e (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:30.833077 28355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdd1e39ef1841c382e814af04ea5bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.833459 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc4169a890e44e3ad14a98135e7071e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.834061 28351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fc9bc911cb145b0a259ee6481eefdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.838068 28356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c474098832a04762b58521bdc2a92b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.838745 28349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6d72f1fb474adfafdd159090dc63b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.847918 28481 tablet_bootstrap.cc:492] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:30.855818 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f380821c5264bfc8a5f0bf36bc52fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.885897 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 582bd3fd6d214e92aefb0baa58c76aed (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:30.890522 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582bd3fd6d214e92aefb0baa58c76aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.887948 28481 tablet_bootstrap.cc:654] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:30.897864 28351 tablet_service.cc:1511] Processing CreateTablet for tablet 582bd3fd6d214e92aefb0baa58c76aed (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:30.914395 28351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582bd3fd6d214e92aefb0baa58c76aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.922959 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 1410767867a5470e81621556e44c6560 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:30.929247 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 9cdd1e39ef1841c382e814af04ea5bc9 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:30.930627 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1410767867a5470e81621556e44c6560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.931828 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 9c99f01ab7374acfa37f7669b8fcdeb2 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:30.932782 28427 tablet_service.cc:1511] Processing CreateTablet for tablet bce32c3c49414e228fe6d912d95aeaac (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:30.934147 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce32c3c49414e228fe6d912d95aeaac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.934605 28352 tablet_service.cc:1511] Processing CreateTablet for tablet 1410767867a5470e81621556e44c6560 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:30.934934 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 2e3e7c85ff90423aa9f9d234d992379b (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:30.935930 28352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1410767867a5470e81621556e44c6560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.939098 28275 tablet_service.cc:1511] Processing CreateTablet for tablet 7f380821c5264bfc8a5f0bf36bc52fd4 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:30.941601 28354 tablet_service.cc:1511] Processing CreateTablet for tablet 23a4c78997e24df7bce95fcf6d70c95c (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:30.941975 28274 tablet_service.cc:1511] Processing CreateTablet for tablet ab6d72f1fb474adfafdd159090dc63b5 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:30.942946 28354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23a4c78997e24df7bce95fcf6d70c95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.942881 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 23a4c78997e24df7bce95fcf6d70c95c (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:30.944188 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23a4c78997e24df7bce95fcf6d70c95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.943892 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 21eeb096d8a4423bbc24e8935e0539fc (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:30.946182 28273 tablet_service.cc:1511] Processing CreateTablet for tablet cdc4169a890e44e3ad14a98135e7071e (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:30.925206 28281 tablet_service.cc:1511] Processing CreateTablet for tablet c474098832a04762b58521bdc2a92b5a (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:30.937143 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 1fc9bc911cb145b0a259ee6481eefdf6 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:30.923511 28282 tablet_service.cc:1511] Processing CreateTablet for tablet 23d6b97dd9e9466d9581f6088e681a11 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:30.925978 28348 tablet_service.cc:1511] Processing CreateTablet for tablet bce32c3c49414e228fe6d912d95aeaac (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:30.930639 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdd1e39ef1841c382e814af04ea5bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.952418 28348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce32c3c49414e228fe6d912d95aeaac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.952947 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6d72f1fb474adfafdd159090dc63b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.962944 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c99f01ab7374acfa37f7669b8fcdeb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.966097 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f380821c5264bfc8a5f0bf36bc52fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.957566 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 986f172991d84af28d4f5cbe8c517e6e (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:30.967454 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986f172991d84af28d4f5cbe8c517e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.968441 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3e7c85ff90423aa9f9d234d992379b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.970142 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc4169a890e44e3ad14a98135e7071e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.974119 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fc9bc911cb145b0a259ee6481eefdf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.976344 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21eeb096d8a4423bbc24e8935e0539fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.978127 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c474098832a04762b58521bdc2a92b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.982483 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23d6b97dd9e9466d9581f6088e681a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.922271 28482 tablet_bootstrap.cc:492] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:30.997133 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 582bd3fd6d214e92aefb0baa58c76aed (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:30.998467 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 582bd3fd6d214e92aefb0baa58c76aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:30.998785 28355 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee4696ba90a447fa07c4d09407217f6 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:31.000227 28355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee4696ba90a447fa07c4d09407217f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.023406 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 1410767867a5470e81621556e44c6560 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:31.029165 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee4696ba90a447fa07c4d09407217f6 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:31.030598 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee4696ba90a447fa07c4d09407217f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.052248 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 014b394597e7430489bd093d8616dcad (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:31.054733 28423 tablet_service.cc:1511] Processing CreateTablet for tablet c59ee067797b40d396f97683aa7eca81 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:31.053604 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014b394597e7430489bd093d8616dcad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.060343 28431 tablet_service.cc:1511] Processing CreateTablet for tablet 956fb13fa38243fd862789f3046aba97 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:31.060925 28484 tablet_bootstrap.cc:492] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:31.061869 28349 tablet_service.cc:1511] Processing CreateTablet for tablet 986f172991d84af28d4f5cbe8c517e6e (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:31.062860 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 255e0140f2a64433a6c2dd64fe24e6b3 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:31.063184 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956fb13fa38243fd862789f3046aba97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.064095 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255e0140f2a64433a6c2dd64fe24e6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.063913 28355 tablet_service.cc:1511] Processing CreateTablet for tablet 014b394597e7430489bd093d8616dcad (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:31.065972 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59ee067797b40d396f97683aa7eca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.066040 28482 tablet_bootstrap.cc:654] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.070719 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1410767867a5470e81621556e44c6560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.070498 28353 tablet_service.cc:1511] Processing CreateTablet for tablet 956fb13fa38243fd862789f3046aba97 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:31.070500 28356 tablet_service.cc:1511] Processing CreateTablet for tablet 255e0140f2a64433a6c2dd64fe24e6b3 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:31.072127 28350 tablet_service.cc:1511] Processing CreateTablet for tablet c59ee067797b40d396f97683aa7eca81 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:31.063161 28349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986f172991d84af28d4f5cbe8c517e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.075374 28356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255e0140f2a64433a6c2dd64fe24e6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.078064 28353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956fb13fa38243fd862789f3046aba97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.081128 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59ee067797b40d396f97683aa7eca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.082559 28275 tablet_service.cc:1511] Processing CreateTablet for tablet bce32c3c49414e228fe6d912d95aeaac (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:31.083951 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce32c3c49414e228fe6d912d95aeaac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.087360 28481 tablet_bootstrap.cc:492] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:31.087896 28481 ts_tablet_manager.cc:1403] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.240s	user 0.041s	sys 0.049s
W20260709 12:16:31.096877 28238 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:31.102134 28355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014b394597e7430489bd093d8616dcad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.123991 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 23a4c78997e24df7bce95fcf6d70c95c (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:31.127216 28354 tablet_service.cc:1511] Processing CreateTablet for tablet 114fa146124747358c997ac1008e1b93 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:31.140987 28354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114fa146124747358c997ac1008e1b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.133649 28484 tablet_bootstrap.cc:654] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.143352 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 986f172991d84af28d4f5cbe8c517e6e (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:31.144812 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986f172991d84af28d4f5cbe8c517e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.135257 28481 raft_consensus.cc:359] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.145265 28481 raft_consensus.cc:385] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.145576 28481 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.145838 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23a4c78997e24df7bce95fcf6d70c95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.154503 28276 tablet_service.cc:1511] Processing CreateTablet for tablet c59ee067797b40d396f97683aa7eca81 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:31.155810 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59ee067797b40d396f97683aa7eca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.155624 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 014b394597e7430489bd093d8616dcad (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:31.156865 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014b394597e7430489bd093d8616dcad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.157090 28481 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.158545 28282 tablet_service.cc:1511] Processing CreateTablet for tablet 255e0140f2a64433a6c2dd64fe24e6b3 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:31.159513 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 956fb13fa38243fd862789f3046aba97 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:31.151963 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee4696ba90a447fa07c4d09407217f6 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:31.164261 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255e0140f2a64433a6c2dd64fe24e6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.163558 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 114fa146124747358c997ac1008e1b93 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:31.172783 28466 heartbeater.cc:499] Master 127.26.123.62:36149 was elected leader, sending a full tablet report...
I20260709 12:16:31.174413 28481 ts_tablet_manager.cc:1434] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.086s	user 0.012s	sys 0.001s
I20260709 12:16:31.174743 28353 tablet_service.cc:1511] Processing CreateTablet for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:31.176882 28353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.177232 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee4696ba90a447fa07c4d09407217f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.179855 28482 tablet_bootstrap.cc:492] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:31.180406 28482 ts_tablet_manager.cc:1403] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.259s	user 0.007s	sys 0.011s
I20260709 12:16:31.183554 28482 raft_consensus.cc:359] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.184298 28482 raft_consensus.cc:385] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.184621 28482 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.186275 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114fa146124747358c997ac1008e1b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.186946 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:31.189013 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.196496 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:31.197968 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.198788 28481 tablet_bootstrap.cc:492] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:31.200736 28482 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.207747 28484 tablet_bootstrap.cc:492] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:31.208354 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956fb13fa38243fd862789f3046aba97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.208374 28484 ts_tablet_manager.cc:1403] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.148s	user 0.005s	sys 0.014s
I20260709 12:16:31.219144 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 65569448c1eb448799671a8f4ba82cd4 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:31.220537 28484 raft_consensus.cc:359] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.220541 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65569448c1eb448799671a8f4ba82cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.221560 28484 raft_consensus.cc:385] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.226325 28484 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.229457 28481 tablet_bootstrap.cc:654] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.183836 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 114fa146124747358c997ac1008e1b93 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:31.235337 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114fa146124747358c997ac1008e1b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.235291 28392 heartbeater.cc:499] Master 127.26.123.62:36149 was elected leader, sending a full tablet report...
I20260709 12:16:31.236467 28355 tablet_service.cc:1511] Processing CreateTablet for tablet 65569448c1eb448799671a8f4ba82cd4 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:31.236982 28488 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:31.237466 28488 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.239169 28355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65569448c1eb448799671a8f4ba82cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.238960 28482 ts_tablet_manager.cc:1434] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.005s
I20260709 12:16:31.251513 28431 tablet_service.cc:1511] Processing CreateTablet for tablet 65569448c1eb448799671a8f4ba82cd4 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:31.252947 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65569448c1eb448799671a8f4ba82cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.255558 28487 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:31.256078 28487 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.258678 28487 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:31.263831 28275 tablet_service.cc:1511] Processing CreateTablet for tablet b0d8e709ec114a04bfd86247e24dd63c (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:31.265336 28350 tablet_service.cc:1511] Processing CreateTablet for tablet b0d8e709ec114a04bfd86247e24dd63c (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:31.266826 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d8e709ec114a04bfd86247e24dd63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.265307 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d8e709ec114a04bfd86247e24dd63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.273168 28482 tablet_bootstrap.cc:492] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:31.234218 28484 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.282442 28488 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:31.292896 28356 tablet_service.cc:1511] Processing CreateTablet for tablet 5fb143bfa1d44653b638d40442c7e955 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:31.293380 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 5fb143bfa1d44653b638d40442c7e955 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:31.293694 28422 tablet_service.cc:1511] Processing CreateTablet for tablet b0d8e709ec114a04bfd86247e24dd63c (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:31.302824 28317 heartbeater.cc:499] Master 127.26.123.62:36149 was elected leader, sending a full tablet report...
I20260709 12:16:31.303756 28356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fb143bfa1d44653b638d40442c7e955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.304797 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 6c8442db4dc44c619fd67179aa212063 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:31.306629 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 5fb143bfa1d44653b638d40442c7e955 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:31.308249 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d8e709ec114a04bfd86247e24dd63c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.315232 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fb143bfa1d44653b638d40442c7e955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.323133 28349 tablet_service.cc:1511] Processing CreateTablet for tablet 6c8442db4dc44c619fd67179aa212063 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:31.324815 28349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c8442db4dc44c619fd67179aa212063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.326478 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fb143bfa1d44653b638d40442c7e955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.337956 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 6c8442db4dc44c619fd67179aa212063 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:31.339371 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c8442db4dc44c619fd67179aa212063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.339195 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 85a68e4142444e5cb52829e957c1abf7 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:31.340500 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a68e4142444e5cb52829e957c1abf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.363860 28348 tablet_service.cc:1511] Processing CreateTablet for tablet 85a68e4142444e5cb52829e957c1abf7 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:31.365583 28348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a68e4142444e5cb52829e957c1abf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.368013 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c8442db4dc44c619fd67179aa212063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.381778 28482 tablet_bootstrap.cc:654] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.389056 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 85a68e4142444e5cb52829e957c1abf7 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:31.390549 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a68e4142444e5cb52829e957c1abf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.414004 28481 tablet_bootstrap.cc:492] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:31.415417 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:31.417043 28484 ts_tablet_manager.cc:1434] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.201s	user 0.012s	sys 0.008s
I20260709 12:16:31.440001 28484 tablet_bootstrap.cc:492] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:31.422624 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:16:31.443311 28330 leader_election.cc:343] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:31.438704 28404 leader_election.cc:343] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.446023 28493 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:31.446619 28493 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.457167 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 8d63db98cb72412194c214fa082eee94 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:31.458859 28349 tablet_service.cc:1511] Processing CreateTablet for tablet 61d225f747aa45ff89df6c0ba308faad (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:31.458889 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d63db98cb72412194c214fa082eee94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.460919 28481 ts_tablet_manager.cc:1403] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.262s	user 0.011s	sys 0.007s
I20260709 12:16:31.459204 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 61d225f747aa45ff89df6c0ba308faad (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:31.484754 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61d225f747aa45ff89df6c0ba308faad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.471021 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 61d225f747aa45ff89df6c0ba308faad (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:31.457198 28493 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:31.476328 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:31.496160 28349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61d225f747aa45ff89df6c0ba308faad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:31.497809 28404 leader_election.cc:343] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.498383 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:31.499763 28487 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:31.494501 28484 tablet_bootstrap.cc:654] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.510507 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61d225f747aa45ff89df6c0ba308faad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.535741 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 8d63db98cb72412194c214fa082eee94 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:31.546911 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:31.493454 28481 raft_consensus.cc:359] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:31.549835 28255 leader_election.cc:343] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.551473 28482 tablet_bootstrap.cc:492] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:31.551965 28482 ts_tablet_manager.cc:1403] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.279s	user 0.009s	sys 0.015s
I20260709 12:16:31.554363 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d63db98cb72412194c214fa082eee94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.561832 28481 raft_consensus.cc:385] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.562331 28481 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.562911 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:31.563424 28481 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:31.564874 28332 leader_election.cc:343] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.573889 28332 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:31.574808 28488 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:31.576247 28356 tablet_service.cc:1511] Processing CreateTablet for tablet 8d63db98cb72412194c214fa082eee94 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:31.577800 28356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d63db98cb72412194c214fa082eee94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.587133 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 54211b8ac4fe44c59e9f0338d9960177 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:31.588560 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54211b8ac4fe44c59e9f0338d9960177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.578279 28355 tablet_service.cc:1511] Processing CreateTablet for tablet 54211b8ac4fe44c59e9f0338d9960177 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:31.586040 28482 raft_consensus.cc:359] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.594400 28355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54211b8ac4fe44c59e9f0338d9960177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.576828 28353 tablet_service.cc:1511] Processing CreateTablet for tablet 705e15df427147839fee08c92bc268dc (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:31.599545 28353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 705e15df427147839fee08c92bc268dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.600108 28349 tablet_service.cc:1511] Processing CreateTablet for tablet 66010729a96e49ebb751b59d4c7f5f3a (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:31.601516 28349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66010729a96e49ebb751b59d4c7f5f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.582960 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 705e15df427147839fee08c92bc268dc (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:31.603163 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 705e15df427147839fee08c92bc268dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.576838 28481 ts_tablet_manager.cc:1434] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.099s	user 0.010s	sys 0.000s
I20260709 12:16:31.594349 28482 raft_consensus.cc:385] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.610245 28482 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.610985 28481 tablet_bootstrap.cc:492] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:31.611364 28482 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.582515 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 705e15df427147839fee08c92bc268dc (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:31.614192 28482 ts_tablet_manager.cc:1434] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.001s
I20260709 12:16:31.614774 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 705e15df427147839fee08c92bc268dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.615275 28482 tablet_bootstrap.cc:492] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:31.598440 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 66010729a96e49ebb751b59d4c7f5f3a (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:31.623013 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66010729a96e49ebb751b59d4c7f5f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.632279 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 66010729a96e49ebb751b59d4c7f5f3a (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:31.633802 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66010729a96e49ebb751b59d4c7f5f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.634071 28482 tablet_bootstrap.cc:654] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.637928 28350 tablet_service.cc:1511] Processing CreateTablet for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:31.639410 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.643396 28275 tablet_service.cc:1511] Processing CreateTablet for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:31.644810 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.650130 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:31.650888 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:31.653119 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.652969 28276 tablet_service.cc:1511] Processing CreateTablet for tablet b4036b67205e41c597049a10a71e9467 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:31.654458 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4036b67205e41c597049a10a71e9467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.622913 28431 tablet_service.cc:1511] Processing CreateTablet for tablet 54211b8ac4fe44c59e9f0338d9960177 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:31.655616 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54211b8ac4fe44c59e9f0338d9960177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.587540 28484 tablet_bootstrap.cc:492] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:31.659332 28484 ts_tablet_manager.cc:1403] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.220s	user 0.018s	sys 0.007s
W20260709 12:16:31.662551 28257 leader_election.cc:343] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.663337 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:31.682039 28348 tablet_service.cc:1511] Processing CreateTablet for tablet b4036b67205e41c597049a10a71e9467 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:31.683679 28348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4036b67205e41c597049a10a71e9467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.693447 28425 tablet_service.cc:1511] Processing CreateTablet for tablet b4036b67205e41c597049a10a71e9467 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:31.695237 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4036b67205e41c597049a10a71e9467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.698649 28482 tablet_bootstrap.cc:492] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:31.699290 28482 ts_tablet_manager.cc:1403] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.084s	user 0.021s	sys 0.012s
I20260709 12:16:31.702628 28482 raft_consensus.cc:359] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.703063 28484 raft_consensus.cc:359] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.703913 28484 raft_consensus.cc:385] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.704485 28484 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.703497 28482 raft_consensus.cc:385] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.718073 28482 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.718436 28484 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.718889 28482 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.701584 28481 tablet_bootstrap.cc:654] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.705192 28493 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:31.714775 28488 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:31.721038 28488 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.723352 28488 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:31.725271 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:31.727041 28332 leader_election.cc:343] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.717577 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 775a5161e47944f5ad46f1616a48b2fe (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:31.730105 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 775a5161e47944f5ad46f1616a48b2fe (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:31.731568 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775a5161e47944f5ad46f1616a48b2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.716583 28356 tablet_service.cc:1511] Processing CreateTablet for tablet 775a5161e47944f5ad46f1616a48b2fe (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:31.736261 28356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775a5161e47944f5ad46f1616a48b2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.737864 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775a5161e47944f5ad46f1616a48b2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.742635 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:31.743876 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac0d46a7b244f2494d81e6667cfa1bf (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:31.745385 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac0d46a7b244f2494d81e6667cfa1bf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:31.746332 28330 leader_election.cc:343] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.746845 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:31.748220 28488 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:31.750540 28350 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac0d46a7b244f2494d81e6667cfa1bf (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:31.752106 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac0d46a7b244f2494d81e6667cfa1bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.756553 28484 ts_tablet_manager.cc:1434] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.097s	user 0.004s	sys 0.003s
I20260709 12:16:31.758459 28484 tablet_bootstrap.cc:492] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:31.762889 28481 tablet_bootstrap.cc:492] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:31.763307 28482 ts_tablet_manager.cc:1434] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.006s
I20260709 12:16:31.763474 28481 ts_tablet_manager.cc:1403] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.153s	user 0.004s	sys 0.008s
I20260709 12:16:31.764612 28482 tablet_bootstrap.cc:492] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:31.766953 28481 raft_consensus.cc:359] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.767820 28481 raft_consensus.cc:385] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.769043 28481 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.770128 28481 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.772842 28481 ts_tablet_manager.cc:1434] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:16:31.776206 28481 tablet_bootstrap.cc:492] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:31.782927 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac0d46a7b244f2494d81e6667cfa1bf (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:31.785650 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac0d46a7b244f2494d81e6667cfa1bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.791679 28484 tablet_bootstrap.cc:654] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.793586 28273 tablet_service.cc:1511] Processing CreateTablet for tablet bea7beef1d4c49159d75e5f61eaaeb5a (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:31.795194 28481 tablet_bootstrap.cc:654] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.797468 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 448f610d25494d2c875c5042ec162f71 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:31.813195 28427 tablet_service.cc:1511] Processing CreateTablet for tablet b5bac9d516d84373a10b8812e66d31ed (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:31.798600 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 72871f9ecbeb417fa5bd39d4714f0a90 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:31.813670 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea7beef1d4c49159d75e5f61eaaeb5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.814653 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72871f9ecbeb417fa5bd39d4714f0a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.813791 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448f610d25494d2c875c5042ec162f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.799793 28482 tablet_bootstrap.cc:654] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.831351 28481 tablet_bootstrap.cc:492] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:31.832062 28481 ts_tablet_manager.cc:1403] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.008s
I20260709 12:16:31.831951 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 9a388b90c9b140649cada36c135d7080 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:31.804860 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 72871f9ecbeb417fa5bd39d4714f0a90 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:31.805667 28355 tablet_service.cc:1511] Processing CreateTablet for tablet 72871f9ecbeb417fa5bd39d4714f0a90 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:31.833420 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a388b90c9b140649cada36c135d7080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.834080 28355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72871f9ecbeb417fa5bd39d4714f0a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.834815 28354 tablet_service.cc:1511] Processing CreateTablet for tablet 9a388b90c9b140649cada36c135d7080 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:31.836109 28354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a388b90c9b140649cada36c135d7080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.837951 28481 raft_consensus.cc:359] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.838785 28481 raft_consensus.cc:385] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.839169 28481 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.840044 28481 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.842475 28481 ts_tablet_manager.cc:1434] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:16:31.843400 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72871f9ecbeb417fa5bd39d4714f0a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.849854 28481 tablet_bootstrap.cc:492] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:31.850096 28487 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:31.850581 28487 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.806560 28425 tablet_service.cc:1511] Processing CreateTablet for tablet bea7beef1d4c49159d75e5f61eaaeb5a (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:31.808823 28280 tablet_service.cc:1511] Processing CreateTablet for tablet b5bac9d516d84373a10b8812e66d31ed (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:31.810498 28275 tablet_service.cc:1511] Processing CreateTablet for tablet 448f610d25494d2c875c5042ec162f71 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:31.852810 28487 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:31.816037 28350 tablet_service.cc:1511] Processing CreateTablet for tablet b5bac9d516d84373a10b8812e66d31ed (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:31.854280 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5bac9d516d84373a10b8812e66d31ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.854661 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:31.856820 28481 tablet_bootstrap.cc:654] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:31.856997 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:31.822093 28278 tablet_service.cc:1511] Processing CreateTablet for tablet e2ac8850682043e5bf8caf7fc64fae05 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:31.799186 28348 tablet_service.cc:1511] Processing CreateTablet for tablet 448f610d25494d2c875c5042ec162f71 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260709 12:16:31.858935 28404 leader_election.cc:343] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.825423 28349 tablet_service.cc:1511] Processing CreateTablet for tablet e2ac8850682043e5bf8caf7fc64fae05 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:31.829528 28430 tablet_service.cc:1511] Processing CreateTablet for tablet e2ac8850682043e5bf8caf7fc64fae05 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 12:16:31.859695 28404 leader_election.cc:343] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.860129 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:31.859131 28348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448f610d25494d2c875c5042ec162f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.861268 28487 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:31.861152 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 9a388b90c9b140649cada36c135d7080 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:31.862159 28349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2ac8850682043e5bf8caf7fc64fae05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.863621 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 610cd41ac66c4074be8e2aa7b22d5e6c (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:31.865592 28352 tablet_service.cc:1511] Processing CreateTablet for tablet 610cd41ac66c4074be8e2aa7b22d5e6c (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:31.852701 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5bac9d516d84373a10b8812e66d31ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.867013 28352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610cd41ac66c4074be8e2aa7b22d5e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.867523 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610cd41ac66c4074be8e2aa7b22d5e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.868258 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2ac8850682043e5bf8caf7fc64fae05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.870137 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448f610d25494d2c875c5042ec162f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.887578 28353 tablet_service.cc:1511] Processing CreateTablet for tablet bea7beef1d4c49159d75e5f61eaaeb5a (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:31.889144 28353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea7beef1d4c49159d75e5f61eaaeb5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.897853 28429 tablet_service.cc:1511] Processing CreateTablet for tablet 610cd41ac66c4074be8e2aa7b22d5e6c (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:31.898864 28482 tablet_bootstrap.cc:492] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:31.899523 28482 ts_tablet_manager.cc:1403] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.135s	user 0.019s	sys 0.002s
I20260709 12:16:31.902945 28482 raft_consensus.cc:359] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.911062 28504 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:31.930181 28504 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.932714 28504 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:31.935236 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:31.937260 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:31.912593 28351 tablet_service.cc:1511] Processing CreateTablet for tablet 916dfa76a5254f63bfc746dbe9c5fde0 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:31.922127 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 916dfa76a5254f63bfc746dbe9c5fde0 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:31.937029 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 819ca6350ca84ff49e7254638d8755e7 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:31.939102 28351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 916dfa76a5254f63bfc746dbe9c5fde0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.939795 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819ca6350ca84ff49e7254638d8755e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.852701 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea7beef1d4c49159d75e5f61eaaeb5a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:31.942714 28257 leader_election.cc:343] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:31.944466 28255 leader_election.cc:343] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:31.944981 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:31.946436 28504 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:31.944937 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 916dfa76a5254f63bfc746dbe9c5fde0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.950110 28429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610cd41ac66c4074be8e2aa7b22d5e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.952312 28430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2ac8850682043e5bf8caf7fc64fae05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.910391 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 916dfa76a5254f63bfc746dbe9c5fde0 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:31.955178 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5bac9d516d84373a10b8812e66d31ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.955899 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 916dfa76a5254f63bfc746dbe9c5fde0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.958124 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a388b90c9b140649cada36c135d7080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.958806 28355 tablet_service.cc:1511] Processing CreateTablet for tablet 819ca6350ca84ff49e7254638d8755e7 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:31.960160 28355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819ca6350ca84ff49e7254638d8755e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.962620 28482 raft_consensus.cc:385] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.962976 28481 tablet_bootstrap.cc:492] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:31.963721 28481 ts_tablet_manager.cc:1403] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.114s	user 0.014s	sys 0.000s
I20260709 12:16:31.963075 28482 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.967504 28482 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.978942 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 819ca6350ca84ff49e7254638d8755e7 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:31.980433 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819ca6350ca84ff49e7254638d8755e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.983407 28481 raft_consensus.cc:359] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.984032 28484 tablet_bootstrap.cc:492] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:31.984452 28481 raft_consensus.cc:385] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:31.984691 28484 ts_tablet_manager.cc:1403] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.226s	user 0.013s	sys 0.013s
I20260709 12:16:31.984889 28481 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:31.985932 28481 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:31.988466 28481 ts_tablet_manager.cc:1434] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 12:16:31.993871 28431 tablet_service.cc:1511] Processing CreateTablet for tablet e77b6d26dd064626ab3b950e5a6c272f (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:31.994856 28481 tablet_bootstrap.cc:492] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:31.995520 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77b6d26dd064626ab3b950e5a6c272f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:31.997910 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 8fa1ea35643e43ea8ab0f71a59be9e29 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:31.998988 28482 ts_tablet_manager.cc:1434] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.099s	user 0.003s	sys 0.005s
I20260709 12:16:31.999418 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fa1ea35643e43ea8ab0f71a59be9e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.000213 28482 tablet_bootstrap.cc:492] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:31.990882 28281 tablet_service.cc:1511] Processing CreateTablet for tablet e77b6d26dd064626ab3b950e5a6c272f (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:32.001971 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77b6d26dd064626ab3b950e5a6c272f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.006078 28487 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.006669 28487 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.009533 28487 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:32.010334 28354 tablet_service.cc:1511] Processing CreateTablet for tablet 8fa1ea35643e43ea8ab0f71a59be9e29 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:32.011374 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.012116 28291 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:32.013348 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:32.015225 28484 raft_consensus.cc:359] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.015741 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:32.016076 28484 raft_consensus.cc:385] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.016389 28484 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
W20260709 12:16:32.016784 28404 leader_election.cc:343] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.018802 28352 tablet_service.cc:1511] Processing CreateTablet for tablet e77b6d26dd064626ab3b950e5a6c272f (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:32.023617 28487 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:32.024121 28487 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:32.024531 28487 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.017198 28484 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.027837 28354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fa1ea35643e43ea8ab0f71a59be9e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.033703 28352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77b6d26dd064626ab3b950e5a6c272f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.060153 28482 tablet_bootstrap.cc:654] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.066707 28484 ts_tablet_manager.cc:1434] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.082s	user 0.010s	sys 0.001s
I20260709 12:16:32.073349 28484 tablet_bootstrap.cc:492] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:32.079914 28505 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.080502 28505 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.082859 28505 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.084894 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.086534 28481 tablet_bootstrap.cc:654] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.088555 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:32.090081 28330 leader_election.cc:343] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:32.091998 28332 leader_election.cc:343] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.092561 28332 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:32.093933 28505 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:32.096392 28487 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:32.099623 28238 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:32.107388 28484 tablet_bootstrap.cc:654] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.111941 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:32.112675 28291 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.072990 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 8fa1ea35643e43ea8ab0f71a59be9e29 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:32.138641 28487 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:32.157275 28275 tablet_service.cc:1511] Processing CreateTablet for tablet 189fdd6055df4826a42af9c225596e34 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:32.157999 28482 tablet_bootstrap.cc:492] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:32.158526 28482 ts_tablet_manager.cc:1403] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.159s	user 0.029s	sys 0.001s
I20260709 12:16:32.159286 28431 tablet_service.cc:1511] Processing CreateTablet for tablet 189fdd6055df4826a42af9c225596e34 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:32.159677 28281 tablet_service.cc:1511] Processing CreateTablet for tablet d68008d2b3fd4f7bb72bf69f612b2998 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:32.161335 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d68008d2b3fd4f7bb72bf69f612b2998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.158393 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:16:32.163333 28404 leader_election.cc:343] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.165894 28482 raft_consensus.cc:359] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.166788 28482 raft_consensus.cc:385] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.167166 28482 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.171089 28482 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.168776 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189fdd6055df4826a42af9c225596e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.169459 28513 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.169993 28291 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:32.165067 28350 tablet_service.cc:1511] Processing CreateTablet for tablet 189fdd6055df4826a42af9c225596e34 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:32.154513 28487 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.173122 28481 tablet_bootstrap.cc:492] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:32.185252 28481 ts_tablet_manager.cc:1403] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.191s	user 0.021s	sys 0.015s
I20260709 12:16:32.185056 28487 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.186987 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:32.177175 28513 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.185559 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189fdd6055df4826a42af9c225596e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.176277 28482 ts_tablet_manager.cc:1434] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:16:32.190263 28481 raft_consensus.cc:359] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.194182 28513 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:16:32.194685 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.194993 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:32.200057 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:32.201152 28513 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:32.207067 28404 leader_election.cc:343] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.207518 28513 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:32.208489 28513 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.201614 28291 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:32.197032 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.205104 28505 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.211620 28505 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.206032 28367 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:32.206439 28428 tablet_service.cc:1511] Processing CreateTablet for tablet d68008d2b3fd4f7bb72bf69f612b2998 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:32.197311 28481 raft_consensus.cc:385] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.214813 28481 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.215874 28481 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.217315 28505 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.218104 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:32.217249 28348 tablet_service.cc:1511] Processing CreateTablet for tablet d68008d2b3fd4f7bb72bf69f612b2998 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:32.218310 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.219501 28348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d68008d2b3fd4f7bb72bf69f612b2998. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:32.219849 28332 leader_election.cc:343] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:32.220474 28330 leader_election.cc:343] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.221014 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:32.220526 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:32.225176 28505 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:32.196566 28487 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:32.232019 28482 tablet_bootstrap.cc:492] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:32.229351 28487 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:32.232847 28487 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:32.233060 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 8ab1586b52be49808b90fcc88bdf2d60 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:32.233444 28487 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.240175 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab1586b52be49808b90fcc88bdf2d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.240302 28481 ts_tablet_manager.cc:1434] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.006s
I20260709 12:16:32.235114 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189fdd6055df4826a42af9c225596e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.243204 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d68008d2b3fd4f7bb72bf69f612b2998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.252609 28505 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.253221 28505 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.255962 28351 tablet_service.cc:1511] Processing CreateTablet for tablet 8ab1586b52be49808b90fcc88bdf2d60 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:32.257583 28505 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.257478 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:32.260334 28332 leader_election.cc:343] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.262511 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:16:32.264153 28330 leader_election.cc:343] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.264840 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:32.265754 28505 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:32.257478 28351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab1586b52be49808b90fcc88bdf2d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.272253 28484 tablet_bootstrap.cc:492] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:32.272473 28504 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.273048 28504 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.275414 28504 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.277472 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:32.279395 28255 leader_election.cc:343] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.283196 28482 tablet_bootstrap.cc:654] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.283006 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:32.284955 28257 leader_election.cc:343] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.285480 28257 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:32.266289 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fa1ea35643e43ea8ab0f71a59be9e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.286290 28484 ts_tablet_manager.cc:1403] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.213s	user 0.033s	sys 0.028s
I20260709 12:16:32.286324 28504 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:32.241683 28481 tablet_bootstrap.cc:492] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:32.290971 28484 raft_consensus.cc:359] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.291841 28484 raft_consensus.cc:385] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.292169 28484 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.293023 28484 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.295837 28484 ts_tablet_manager.cc:1434] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:16:32.296322 28188 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:32.297055 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 8ab1586b52be49808b90fcc88bdf2d60 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:32.298568 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab1586b52be49808b90fcc88bdf2d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.286783 28487 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:32.307082 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 86653us
I20260709 12:16:32.308089 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:16:32.309345 28404 leader_election.cc:343] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.314177 28517 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:32.314728 28517 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:32.315096 28487 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:16:32.317451 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:32.317760 28484 tablet_bootstrap.cc:492] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:32.318158 28367 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.327231 28367 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:32.331815 28481 tablet_bootstrap.cc:654] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.332587 28482 tablet_bootstrap.cc:492] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:32.333197 28482 ts_tablet_manager.cc:1403] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.101s	user 0.011s	sys 0.035s
I20260709 12:16:32.335071 28281 tablet_service.cc:1511] Processing CreateTablet for tablet c0d1d6f8a6a04f6ea4e6f31327e92009 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:32.336545 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0d1d6f8a6a04f6ea4e6f31327e92009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.337210 28482 raft_consensus.cc:359] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.338042 28482 raft_consensus.cc:385] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.338325 28482 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.338991 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:32.339326 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:32.339208 28482 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.340006 28487 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:32.340510 28487 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:32.341380 28487 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.344596 28482 ts_tablet_manager.cc:1434] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:16:32.345845 28482 tablet_bootstrap.cc:492] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:32.315099 28517 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:32.354154 28404 leader_election.cc:343] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.359040 28482 tablet_bootstrap.cc:654] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.363498 28484 tablet_bootstrap.cc:654] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.371960 28428 tablet_service.cc:1511] Processing CreateTablet for tablet c0d1d6f8a6a04f6ea4e6f31327e92009 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:32.373785 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0d1d6f8a6a04f6ea4e6f31327e92009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.361663 28348 tablet_service.cc:1511] Processing CreateTablet for tablet c0d1d6f8a6a04f6ea4e6f31327e92009 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:32.382308 28275 tablet_service.cc:1511] Processing CreateTablet for tablet 5235a8de06744285b8c3fe8f9f857223 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:32.384003 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5235a8de06744285b8c3fe8f9f857223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.385916 28482 tablet_bootstrap.cc:492] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:32.386924 28482 ts_tablet_manager.cc:1403] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.011s
I20260709 12:16:32.389588 28348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0d1d6f8a6a04f6ea4e6f31327e92009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.390273 28350 tablet_service.cc:1511] Processing CreateTablet for tablet 5235a8de06744285b8c3fe8f9f857223 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:32.391919 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5235a8de06744285b8c3fe8f9f857223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.395712 28481 tablet_bootstrap.cc:492] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:32.396414 28481 ts_tablet_manager.cc:1403] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.155s	user 0.005s	sys 0.009s
I20260709 12:16:32.403460 28481 raft_consensus.cc:359] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.404367 28481 raft_consensus.cc:385] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.404726 28481 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.411381 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 97c4af01d02243b2b0727baa5bbacf44 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:32.412724 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97c4af01d02243b2b0727baa5bbacf44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.405767 28481 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.414399 28487 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.414901 28487 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.416333 28481 ts_tablet_manager.cc:1434] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.007s
I20260709 12:16:32.420472 28484 tablet_bootstrap.cc:492] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:32.421068 28484 ts_tablet_manager.cc:1403] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.104s	user 0.011s	sys 0.011s
I20260709 12:16:32.421258 28355 tablet_service.cc:1511] Processing CreateTablet for tablet 97c4af01d02243b2b0727baa5bbacf44 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:32.422802 28355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97c4af01d02243b2b0727baa5bbacf44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.423496 28280 tablet_service.cc:1511] Processing CreateTablet for tablet d8850fe0201a48528da0da1cdac707f0 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:32.424853 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8850fe0201a48528da0da1cdac707f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.424482 28484 raft_consensus.cc:359] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.420706 28517 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.427071 28487 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:32.428351 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.429057 28292 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:32.430176 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:32.430902 28367 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:32.432677 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:32.434252 28517 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:32.434149 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 5235a8de06744285b8c3fe8f9f857223 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:32.435014 28517 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:32.435452 28517 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:32.435873 28517 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.436533 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:32.437251 28292 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.439178 28428 tablet_service.cc:1511] Processing CreateTablet for tablet 97c4af01d02243b2b0727baa5bbacf44 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:32.439401 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:16:32.441632 28404 leader_election.cc:343] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.425302 28484 raft_consensus.cc:385] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.442418 28484 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.435683 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5235a8de06744285b8c3fe8f9f857223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.445082 28484 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.447701 28351 tablet_service.cc:1511] Processing CreateTablet for tablet d8850fe0201a48528da0da1cdac707f0 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:32.449293 28351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8850fe0201a48528da0da1cdac707f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.449404 28424 tablet_service.cc:1511] Processing CreateTablet for tablet d8850fe0201a48528da0da1cdac707f0 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:32.451133 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8850fe0201a48528da0da1cdac707f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.451181 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb896260dae4cf6bb09ba7728c603f3 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:32.452716 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb896260dae4cf6bb09ba7728c603f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.456490 28354 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb896260dae4cf6bb09ba7728c603f3 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:32.458037 28354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb896260dae4cf6bb09ba7728c603f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.458146 28428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97c4af01d02243b2b0727baa5bbacf44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.430472 28482 raft_consensus.cc:359] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.462527 28482 raft_consensus.cc:385] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.462864 28482 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.463835 28482 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.462139 28484 ts_tablet_manager.cc:1434] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.002s
I20260709 12:16:32.468040 28481 tablet_bootstrap.cc:492] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:32.481309 28431 tablet_service.cc:1511] Processing CreateTablet for tablet 9cb896260dae4cf6bb09ba7728c603f3 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:32.482506 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 777967d6330c4654813298b12226a560 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:32.484032 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 777967d6330c4654813298b12226a560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.473788 28482 ts_tablet_manager.cc:1434] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.001s
I20260709 12:16:32.474133 28484 tablet_bootstrap.cc:492] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:32.474537 28517 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.482957 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb896260dae4cf6bb09ba7728c603f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.471872 28292 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:32.497928 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:32.498623 28482 tablet_bootstrap.cc:492] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:32.499346 28487 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:32.499905 28487 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:32.500803 28487 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:32.502708 28467 tablet.cc:2406] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:32.503638 28517 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:32.506361 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:32.507014 28367 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.508930 28484 tablet_bootstrap.cc:654] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.510804 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:32.511581 28291 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.515545 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 777967d6330c4654813298b12226a560 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:32.517205 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 777967d6330c4654813298b12226a560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.520960 28349 tablet_service.cc:1511] Processing CreateTablet for tablet 777967d6330c4654813298b12226a560 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:32.523550 28349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 777967d6330c4654813298b12226a560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.529136 28481 tablet_bootstrap.cc:654] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.530665 28188 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:32.540269 28482 tablet_bootstrap.cc:654] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.547178 28484 tablet_bootstrap.cc:492] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:32.547746 28484 ts_tablet_manager.cc:1403] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.074s	user 0.006s	sys 0.006s
I20260709 12:16:32.549183 28275 tablet_service.cc:1511] Processing CreateTablet for tablet 6b2edb513f7343d2a2463b94d992b7a3 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:32.550604 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b2edb513f7343d2a2463b94d992b7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.551115 28484 raft_consensus.cc:359] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.551921 28484 raft_consensus.cc:385] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.552222 28484 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.553763 28484 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.537665 28367 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:32.563450 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:32.564494 28487 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:32.565317 28487 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:32.566308 28487 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.568765 28482 tablet_bootstrap.cc:492] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:32.568324 28187 catalog_manager.cc:5697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:32.569337 28482 ts_tablet_manager.cc:1403] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.071s	user 0.007s	sys 0.006s
I20260709 12:16:32.572448 28482 raft_consensus.cc:359] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.573393 28482 raft_consensus.cc:385] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.573768 28482 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.574685 28482 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.577404 28517 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.578032 28517 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.580363 28517 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:32.580720 28484 ts_tablet_manager.cc:1434] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 12:16:32.581621 28482 ts_tablet_manager.cc:1434] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:16:32.582171 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.582451 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:32.582769 28482 tablet_bootstrap.cc:492] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:32.592351 28482 tablet_bootstrap.cc:654] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.545563 28291 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
W20260709 12:16:32.583616 28404 leader_election.cc:343] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.592020 28350 tablet_service.cc:1511] Processing CreateTablet for tablet 6b2edb513f7343d2a2463b94d992b7a3 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:32.605561 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b2edb513f7343d2a2463b94d992b7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.606307 28481 tablet_bootstrap.cc:492] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:32.606881 28481 ts_tablet_manager.cc:1403] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.179s	user 0.020s	sys 0.010s
I20260709 12:16:32.583213 28367 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:32.610345 28481 raft_consensus.cc:359] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.611263 28481 raft_consensus.cc:385] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.611011 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:32.611617 28481 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.612263 28517 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:32.612720 28481 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.598680 28280 tablet_service.cc:1511] Processing CreateTablet for tablet d6fd84f874b14098badd4d77b5703109 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:32.596887 28431 tablet_service.cc:1511] Processing CreateTablet for tablet 6b2edb513f7343d2a2463b94d992b7a3 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:32.612752 28517 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:32.613922 28517 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.614581 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6fd84f874b14098badd4d77b5703109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.595003 28484 tablet_bootstrap.cc:492] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:32.615396 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b2edb513f7343d2a2463b94d992b7a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.622581 28189 catalog_manager.cc:5697] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:32.623873 28484 tablet_bootstrap.cc:654] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.626509 28482 tablet_bootstrap.cc:492] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:32.627126 28482 ts_tablet_manager.cc:1403] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.004s
I20260709 12:16:32.615420 28481 ts_tablet_manager.cc:1434] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:32.630983 28482 raft_consensus.cc:359] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.631897 28482 raft_consensus.cc:385] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.632265 28482 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.631547 28481 tablet_bootstrap.cc:492] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:32.636812 28348 tablet_service.cc:1511] Processing CreateTablet for tablet d6fd84f874b14098badd4d77b5703109 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:32.638525 28424 tablet_service.cc:1511] Processing CreateTablet for tablet d6fd84f874b14098badd4d77b5703109 (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:32.639906 28482 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.640733 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6fd84f874b14098badd4d77b5703109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.646427 28482 ts_tablet_manager.cc:1434] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:16:32.638494 28348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6fd84f874b14098badd4d77b5703109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.655882 28367 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.657006 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 39dfaebe2a90498f928f7cd6cc499fac (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:32.658391 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39dfaebe2a90498f928f7cd6cc499fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.656396 28517 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.661824 28517 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:32.657160 28482 tablet_bootstrap.cc:492] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:32.664270 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:32.665053 28366 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.668820 28482 tablet_bootstrap.cc:654] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.664422 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:16:32.671082 28404 leader_election.cc:343] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.675601 28366 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:32.677062 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:32.678007 28517 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:32.678589 28517 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:32.679520 28517 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.680485 28350 tablet_service.cc:1511] Processing CreateTablet for tablet 39dfaebe2a90498f928f7cd6cc499fac (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:32.681807 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 39dfaebe2a90498f928f7cd6cc499fac (DEFAULT_TABLE table=test-workload [id=c0383ca16d6f44178ce495b61104f58e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:32.683415 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39dfaebe2a90498f928f7cd6cc499fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.639011 28481 tablet_bootstrap.cc:654] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.687456 28482 tablet_bootstrap.cc:492] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:32.692871 28482 ts_tablet_manager.cc:1403] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.002s
I20260709 12:16:32.693336 28350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39dfaebe2a90498f928f7cd6cc499fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.696501 28482 raft_consensus.cc:359] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.697432 28482 raft_consensus.cc:385] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.697798 28482 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.698943 28482 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.702411 28487 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:32.714334 28482 ts_tablet_manager.cc:1434] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:16:32.715749 28482 tablet_bootstrap.cc:492] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:32.718109 28504 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.721221 28504 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.723519 28504 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.723724 28484 tablet_bootstrap.cc:492] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:32.724294 28484 ts_tablet_manager.cc:1403] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.130s	user 0.014s	sys 0.005s
I20260709 12:16:32.729725 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:32.730002 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:32.730685 28441 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:32.730679 28366 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:32.733314 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:32.737212 28504 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:32.737550 28504 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:32.737824 28504 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.739615 28482 tablet_bootstrap.cc:654] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.742125 28481 tablet_bootstrap.cc:492] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:32.742511 28505 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.743266 28505 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.746542 28505 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.738719 28484 raft_consensus.cc:359] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.747229 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.747411 28484 raft_consensus.cc:385] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.748498 28481 ts_tablet_manager.cc:1403] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.117s	user 0.017s	sys 0.009s
I20260709 12:16:32.749372 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:32.750133 28330 leader_election.cc:343] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.750128 28441 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:32.752321 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:32.753189 28505 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:32.753563 28505 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:32.753934 28505 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.754385 28504 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.748505 28484 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.756971 28484 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.760330 28481 raft_consensus.cc:359] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.761106 28481 raft_consensus.cc:385] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.761394 28481 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.764551 28504 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.766145 28481 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.768410 28481 ts_tablet_manager.cc:1434] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:16:32.769388 28481 tablet_bootstrap.cc:492] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:32.766168 28505 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.772677 28482 tablet_bootstrap.cc:492] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:32.773244 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:32.773931 28441 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.773205 28482 ts_tablet_manager.cc:1403] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.058s	user 0.010s	sys 0.003s
I20260709 12:16:32.779650 28481 tablet_bootstrap.cc:654] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.780845 28482 raft_consensus.cc:359] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.781586 28482 raft_consensus.cc:385] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.781806 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:32.781883 28482 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.785704 28482 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.785581 28513 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:32.789305 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:32.782718 28440 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.790100 28366 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.781904 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:16:32.793848 28330 leader_election.cc:343] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.794747 28513 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:32.795702 28482 ts_tablet_manager.cc:1434] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.002s
I20260709 12:16:32.796754 28482 tablet_bootstrap.cc:492] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:32.797889 28504 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.798437 28504 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.800652 28504 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.773438 28505 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.803589 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:32.813489 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:32.823303 28482 tablet_bootstrap.cc:654] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.843075 28365 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:32.847573 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:32.847884 28484 ts_tablet_manager.cc:1434] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.123s	user 0.025s	sys 0.023s
I20260709 12:16:32.849982 28484 tablet_bootstrap.cc:492] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:32.842582 28544 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.798597 28189 catalog_manager.cc:5697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:32.861249 28484 tablet_bootstrap.cc:654] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.852315 28440 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:32.882321 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:16:32.883893 28404 consensus_peers.cc:597] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:32.867405 28544 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.899406 28441 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:32.902117 28481 tablet_bootstrap.cc:492] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:32.902647 28484 tablet_bootstrap.cc:492] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:32.903424 28484 ts_tablet_manager.cc:1403] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.018s
I20260709 12:16:32.904017 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.903453 28481 ts_tablet_manager.cc:1403] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.134s	user 0.055s	sys 0.046s
I20260709 12:16:32.904672 28257 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:16:32.905958 28330 leader_election.cc:343] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:32.906047 28504 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:32.906481 28484 raft_consensus.cc:359] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.908002 28484 raft_consensus.cc:385] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.908291 28484 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.909083 28484 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.915248 28547 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:32.917593 28505 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.919148 28505 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.921095 28505 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.921325 28539 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.914584 28544 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.917003 28504 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:32.924397 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:32.924742 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:32.925537 28291 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:32.926755 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:32.926844 28504 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.928596 28482 tablet_bootstrap.cc:492] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:32.929008 28482 ts_tablet_manager.cc:1403] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.132s	user 0.005s	sys 0.013s
I20260709 12:16:32.927732 28544 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.932917 28484 ts_tablet_manager.cc:1434] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.029s	user 0.012s	sys 0.002s
I20260709 12:16:32.933858 28484 tablet_bootstrap.cc:492] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:32.935693 28482 raft_consensus.cc:359] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.936470 28482 raft_consensus.cc:385] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.936803 28482 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.937604 28482 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.931301 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:32.939630 28366 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:32.941670 28484 tablet_bootstrap.cc:654] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.942315 28552 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:32.951067 28481 raft_consensus.cc:359] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.952028 28481 raft_consensus.cc:385] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.952322 28481 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.943364 28441 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:32.955852 28332 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:32.953198 28481 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.958528 28482 ts_tablet_manager.cc:1434] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 12:16:32.959785 28482 tablet_bootstrap.cc:492] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:32.961202 28505 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:32.962683 28505 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:32.963027 28505 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.969180 28481 ts_tablet_manager.cc:1434] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.000s
I20260709 12:16:32.962013 28552 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.971267 28481 tablet_bootstrap.cc:492] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:32.971642 28544 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:32.972605 28547 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:32.976300 28482 tablet_bootstrap.cc:654] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.985054 28513 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:32.984988 28505 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.987805 28481 tablet_bootstrap.cc:654] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.988909 28505 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.989537 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:32.990144 28291 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.990082 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:32.990968 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:32.991950 28363 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:32.992705 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:32.991309 28441 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.974386 28547 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:32.993610 28440 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:32.995050 28257 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:32.990376 28552 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:32.997776 28504 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:32.998440 28504 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:32.998785 28504 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.999440 28539 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:33.005039 28484 tablet_bootstrap.cc:492] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:33.005535 28441 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:33.015187 28332 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:33.016574 28555 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.017127 28555 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:33.019304 28555 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.022262 28504 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.005719 28484 ts_tablet_manager.cc:1403] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.072s	user 0.006s	sys 0.010s
I20260709 12:16:33.013183 28513 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.030864 28487 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:33.031774 28484 raft_consensus.cc:359] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.032527 28484 raft_consensus.cc:385] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.032810 28484 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.033512 28484 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.037578 28481 tablet_bootstrap.cc:492] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:33.038146 28481 ts_tablet_manager.cc:1403] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.067s	user 0.008s	sys 0.012s
I20260709 12:16:33.040675 28513 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:33.032336 28189 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.043516 28482 tablet_bootstrap.cc:492] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:33.044052 28482 ts_tablet_manager.cc:1403] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.085s	user 0.010s	sys 0.007s
I20260709 12:16:33.009121 28291 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:33.049266 28481 raft_consensus.cc:359] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.050673 28481 raft_consensus.cc:385] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.050961 28481 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.051806 28482 raft_consensus.cc:359] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.052557 28482 raft_consensus.cc:385] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.052829 28482 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.053572 28482 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.055965 28481 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.060810 28484 ts_tablet_manager.cc:1434] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:16:33.061270 28481 ts_tablet_manager.cc:1434] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 12:16:33.061707 28482 ts_tablet_manager.cc:1434] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.003s
I20260709 12:16:33.061976 28484 tablet_bootstrap.cc:492] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:33.062362 28481 tablet_bootstrap.cc:492] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:33.062717 28482 tablet_bootstrap.cc:492] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:33.067726 28539 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.068423 28539 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.068502 28504 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.071341 28539 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.073045 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:33.073391 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:33.073764 28441 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.074136 28363 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:33.075784 28482 tablet_bootstrap.cc:654] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.075708 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:33.067569 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:33.077713 28365 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.076181 28188 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.079428 28539 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.081152 28539 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.081533 28539 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.081152 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.086560 28440 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:33.095093 28539 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.098150 28481 tablet_bootstrap.cc:654] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.110481 28482 tablet_bootstrap.cc:492] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:33.111128 28482 ts_tablet_manager.cc:1403] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.049s	user 0.005s	sys 0.019s
W20260709 12:16:33.117127 28238 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:33.118983 28365 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:33.120371 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:33.121332 28504 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.121925 28504 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:33.122430 28441 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:33.122861 28504 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.127610 28539 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.128573 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:33.128822 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:33.129256 28363 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.129354 28440 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.132370 28484 tablet_bootstrap.cc:654] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.137074 28482 raft_consensus.cc:359] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.137902 28482 raft_consensus.cc:385] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.138170 28482 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.138998 28482 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.140337 28363 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:33.141435 28482 ts_tablet_manager.cc:1434] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:16:33.142016 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:33.143172 28539 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.143738 28539 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:33.144730 28539 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.145512 28482 tablet_bootstrap.cc:492] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:33.150452 28440 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:33.154889 28187 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.157706 28189 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.161021 28481 tablet_bootstrap.cc:492] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:33.161659 28481 ts_tablet_manager.cc:1403] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.100s	user 0.016s	sys 0.018s
I20260709 12:16:33.164974 28517 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.166983 28482 tablet_bootstrap.cc:654] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.174355 28481 raft_consensus.cc:359] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.175910 28481 raft_consensus.cc:385] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.176234 28481 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.177553 28547 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.178841 28547 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.180976 28547 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.184295 28481 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.182725 28513 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:33.185300 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.187697 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:33.189982 28330 leader_election.cc:343] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.189092 28481 ts_tablet_manager.cc:1434] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 12:16:33.191653 28481 tablet_bootstrap.cc:492] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
W20260709 12:16:33.192852 28332 leader_election.cc:343] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.193369 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:33.194739 28547 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:33.201139 28484 tablet_bootstrap.cc:492] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:33.201606 28484 ts_tablet_manager.cc:1403] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.140s	user 0.027s	sys 0.012s
I20260709 12:16:33.207108 28188 catalog_manager.cc:5697] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.208887 28484 raft_consensus.cc:359] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.209705 28484 raft_consensus.cc:385] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.210026 28484 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.210958 28484 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.212095 28481 tablet_bootstrap.cc:654] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.213394 28484 ts_tablet_manager.cc:1434] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.006s
I20260709 12:16:33.214555 28484 tablet_bootstrap.cc:492] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:33.227233 28484 tablet_bootstrap.cc:654] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.228559 28481 tablet_bootstrap.cc:492] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:33.230242 28481 ts_tablet_manager.cc:1403] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.004s
I20260709 12:16:33.240293 28482 tablet_bootstrap.cc:492] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:33.240967 28482 ts_tablet_manager.cc:1403] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.096s	user 0.016s	sys 0.007s
I20260709 12:16:33.243327 28547 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.237385 28481 raft_consensus.cc:359] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.244184 28481 raft_consensus.cc:385] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.243930 28547 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.244557 28481 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.246671 28481 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.248133 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.245388 28482 raft_consensus.cc:359] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.248739 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.248849 28440 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:33.249423 28292 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:33.248816 28482 raft_consensus.cc:385] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.250182 28482 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.250859 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:33.251737 28505 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.252121 28505 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.251024 28482 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.252492 28505 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.251972 28481 ts_tablet_manager.cc:1434] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:16:33.254082 28481 tablet_bootstrap.cc:492] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:33.262890 28481 tablet_bootstrap.cc:654] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.256075 28482 ts_tablet_manager.cc:1434] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.006s
I20260709 12:16:33.264166 28482 tablet_bootstrap.cc:492] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:33.248943 28547 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.271011 28482 tablet_bootstrap.cc:654] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.274950 28484 tablet_bootstrap.cc:492] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:33.276443 28484 ts_tablet_manager.cc:1403] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.062s	user 0.029s	sys 0.021s
I20260709 12:16:33.279999 28481 tablet_bootstrap.cc:492] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:33.279902 28484 raft_consensus.cc:359] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.280680 28481 ts_tablet_manager.cc:1403] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.000s
I20260709 12:16:33.280704 28484 raft_consensus.cc:385] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.281132 28484 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.281821 28484 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.283627 28505 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.283699 28481 raft_consensus.cc:359] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.284869 28481 raft_consensus.cc:385] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.285171 28481 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.287345 28505 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.287580 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:33.288235 28440 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.288219 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:33.288975 28292 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.291352 28482 tablet_bootstrap.cc:492] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:33.291874 28482 ts_tablet_manager.cc:1403] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.000s
I20260709 12:16:33.289292 28481 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.297060 28292 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:33.297868 28482 raft_consensus.cc:359] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.298501 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:33.299657 28481 ts_tablet_manager.cc:1434] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:16:33.300719 28481 tablet_bootstrap.cc:492] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:33.300758 28484 ts_tablet_manager.cc:1434] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:16:33.301481 28505 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.302284 28505 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:33.303233 28505 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.298597 28482 raft_consensus.cc:385] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.304458 28482 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.305809 28482 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.308585 28481 tablet_bootstrap.cc:654] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.308910 28484 tablet_bootstrap.cc:492] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:33.312160 28440 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:33.314381 28482 ts_tablet_manager.cc:1434] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:16:33.316725 28482 tablet_bootstrap.cc:492] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:33.315627 28484 tablet_bootstrap.cc:654] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.321795 28481 tablet_bootstrap.cc:492] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:33.322373 28481 ts_tablet_manager.cc:1403] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 12:16:33.323688 28482 tablet_bootstrap.cc:654] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.325299 28481 raft_consensus.cc:359] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.326465 28481 raft_consensus.cc:385] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.326853 28481 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.327919 28481 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.330945 28481 ts_tablet_manager.cc:1434] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:16:33.334782 28481 tablet_bootstrap.cc:492] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:33.334378 28187 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.335371 28484 tablet_bootstrap.cc:492] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:33.335868 28484 ts_tablet_manager.cc:1403] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.004s
I20260709 12:16:33.342167 28505 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.342741 28505 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.345124 28484 raft_consensus.cc:359] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.345901 28484 raft_consensus.cc:385] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.346185 28484 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.346331 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.346988 28440 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:33.347852 28484 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.348403 28332 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:33.349313 28505 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.349958 28505 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.350703 28505 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.351140 28505 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.348464 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.353205 28292 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:33.342152 28481 tablet_bootstrap.cc:654] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.363107 28505 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.365837 28505 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.366407 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:33.367051 28440 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.367060 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:33.367815 28292 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.369529 28482 tablet_bootstrap.cc:492] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:33.370157 28482 ts_tablet_manager.cc:1403] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.009s
I20260709 12:16:33.371084 28484 ts_tablet_manager.cc:1434] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.002s
I20260709 12:16:33.372047 28547 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.372588 28547 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.374239 28505 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.365468 28481 tablet_bootstrap.cc:492] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:33.374843 28505 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.376091 28481 ts_tablet_manager.cc:1403] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.000s
I20260709 12:16:33.378252 28484 tablet_bootstrap.cc:492] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:33.380043 28440 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:33.380455 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.381511 28332 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:16:33.382330 28330 leader_election.cc:343] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.382761 28505 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.384057 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.384181 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.384728 28441 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:33.385047 28481 raft_consensus.cc:359] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.385774 28481 raft_consensus.cc:385] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.385854 28484 tablet_bootstrap.cc:654] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.382539 28555 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.386125 28481 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.386593 28555 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:33.386910 28481 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.380048 28292 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:33.386927 28547 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.386072 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:16:33.388406 28330 leader_election.cc:343] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.389864 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.390602 28441 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:33.392345 28547 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.392808 28547 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.393146 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:33.394129 28544 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.393862 28482 raft_consensus.cc:359] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.394508 28544 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.394858 28482 raft_consensus.cc:385] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.394914 28544 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.398501 28539 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.387560 28555 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.404647 28481 ts_tablet_manager.cc:1434] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:16:33.393235 28547 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.406000 28481 tablet_bootstrap.cc:492] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:33.406260 28544 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.412223 28544 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.414109 28481 tablet_bootstrap.cc:654] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.416962 28482 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.417057 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:33.418583 28487 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.428319 28487 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.429762 28544 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:33.422477 28539 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.437043 28539 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.439612 28539 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.440438 28290 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.421865 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:33.446954 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:33.431823 28487 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:33.432816 28291 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:33.454218 28330 leader_election.cc:343] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.458209 28539 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.432791 28441 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.433822 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:33.435567 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:16:33.466850 28404 leader_election.cc:343] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.428488 28482 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.448930 28189 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.473599 28365 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:33.477416 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:33.478391 28487 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.478770 28487 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.479122 28487 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.484395 28585 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.485095 28585 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.487558 28585 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.503393 28484 tablet_bootstrap.cc:492] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:33.503990 28484 ts_tablet_manager.cc:1403] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.126s	user 0.036s	sys 0.079s
I20260709 12:16:33.489190 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.489423 28487 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.493947 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.489282 28544 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:16:33.498670 28589 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.507656 28484 raft_consensus.cc:359] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.508451 28484 raft_consensus.cc:385] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.508037 28487 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:33.508774 28484 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
W20260709 12:16:33.508831 28332 leader_election.cc:343] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.509544 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:33.509845 28484 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.509282 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:33.511089 28365 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.512148 28484 ts_tablet_manager.cc:1434] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:16:33.513258 28484 tablet_bootstrap.cc:492] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:33.499373 28482 ts_tablet_manager.cc:1434] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.129s	user 0.005s	sys 0.002s
I20260709 12:16:33.500947 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.514974 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:33.515707 28482 tablet_bootstrap.cc:492] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:33.520985 28484 tablet_bootstrap.cc:654] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.523936 28482 tablet_bootstrap.cc:654] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:33.526567 28404 leader_election.cc:343] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.527518 28481 tablet_bootstrap.cc:492] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:33.529174 28481 ts_tablet_manager.cc:1403] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.123s	user 0.028s	sys 0.029s
I20260709 12:16:33.493270 28547 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.533506 28484 tablet_bootstrap.cc:492] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:33.534897 28484 ts_tablet_manager.cc:1403] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.011s
I20260709 12:16:33.538082 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:33.537981 28484 raft_consensus.cc:359] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.538967 28437 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.540609 28365 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:33.541100 28552 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.542289 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:33.544026 28487 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.544853 28487 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:33.546008 28487 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.538955 28484 raft_consensus.cc:385] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.547317 28484 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.535610 28547 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.550244 28484 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.550858 28552 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.533146 28481 raft_consensus.cc:359] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.551378 28552 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.549657 28437 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:33.554804 28552 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.554940 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:33.556130 28547 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.551934 28481 raft_consensus.cc:385] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.557168 28481 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.553514 28484 ts_tablet_manager.cc:1434] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 12:16:33.558408 28547 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:33.558736 28484 tablet_bootstrap.cc:492] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:33.558837 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.559532 28547 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.560519 28437 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:33.561861 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:33.562885 28552 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:33.563346 28552 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.563704 28552 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:33.566285 28484 tablet_bootstrap.cc:654] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.568676 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:33.569126 28481 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.573374 28481 ts_tablet_manager.cc:1434] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:16:33.574537 28481 tablet_bootstrap.cc:492] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:33.581358 28481 tablet_bootstrap.cc:654] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.583622 28482 tablet_bootstrap.cc:492] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:33.584201 28482 ts_tablet_manager.cc:1403] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.069s	user 0.010s	sys 0.005s
I20260709 12:16:33.584564 28504 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:33.587222 28482 raft_consensus.cc:359] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.587996 28482 raft_consensus.cc:385] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.588284 28482 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.588971 28482 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:33.590657 28330 leader_election.cc:343] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.591212 28330 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:33.592118 28547 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:33.596491 28482 ts_tablet_manager.cc:1434] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:16:33.602633 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:33.604429 28484 tablet_bootstrap.cc:492] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:33.604986 28484 ts_tablet_manager.cc:1403] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.009s
I20260709 12:16:33.606227 28482 tablet_bootstrap.cc:492] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:33.608018 28484 raft_consensus.cc:359] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.608806 28484 raft_consensus.cc:385] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.609114 28484 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.609963 28484 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:33.611213 28330 leader_election.cc:343] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.612185 28484 ts_tablet_manager.cc:1434] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:33.613338 28484 tablet_bootstrap.cc:492] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:33.617060 28482 tablet_bootstrap.cc:654] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.620203 28481 tablet_bootstrap.cc:492] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:33.620441 28539 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:33.620733 28481 ts_tablet_manager.cc:1403] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.009s
I20260709 12:16:33.623430 28547 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:33.623795 28481 raft_consensus.cc:359] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.624511 28481 raft_consensus.cc:385] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.624805 28481 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.626830 28481 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.629024 28551 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.629504 28551 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.631131 28484 tablet_bootstrap.cc:654] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.631873 28551 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:33.629197 28481 ts_tablet_manager.cc:1434] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 12:16:33.640211 28481 tablet_bootstrap.cc:492] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:33.597586 28441 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:33.648538 28189 catalog_manager.cc:5697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.659431 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:33.660925 28547 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.668881 28552 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.661514 28547 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:33.674929 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.654464 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:33.682111 28366 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:33.683535 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:33.684662 28487 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.685123 28487 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.685598 28487 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.688053 28505 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.688606 28505 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.688668 28547 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:33.700544 28404 leader_election.cc:343] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.702316 28601 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.704649 28505 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.704804 28292 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:16:33.705253 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:33.702232 28539 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:16:33.706460 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.703023 28601 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:33.708259 28332 leader_election.cc:343] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.702291 28487 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.719630 28552 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.738672 28517 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.742566 28517 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.747953 28484 tablet_bootstrap.cc:492] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:33.748281 28481 tablet_bootstrap.cc:654] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.748543 28484 ts_tablet_manager.cc:1403] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.136s	user 0.026s	sys 0.020s
I20260709 12:16:33.756747 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:33.741683 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.760830 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:16:33.761337 28330 leader_election.cc:343] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.762028 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:33.763907 28602 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.764076 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:33.764976 28504 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:33.762909 28589 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:33.766737 28517 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:33.769656 28544 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:33.770716 28589 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:33.771064 28589 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:33.772763 28589 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.774048 28589 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:33.777565 28601 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.780077 28484 raft_consensus.cc:359] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.780970 28484 raft_consensus.cc:385] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.781486 28484 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.782783 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.783788 28439 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 1: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:16:33.785223 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:33.786515 28601 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.788394 28484 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.783027 28487 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:33.790691 28187 catalog_manager.cc:5697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.807765 28484 ts_tablet_manager.cc:1434] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.001s
I20260709 12:16:33.811044 28484 tablet_bootstrap.cc:492] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:33.819099 28505 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.830824 28482 tablet_bootstrap.cc:492] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:33.831486 28482 ts_tablet_manager.cc:1403] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.226s	user 0.026s	sys 0.017s
I20260709 12:16:33.832082 28539 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.839898 28539 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.842515 28539 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:16:33.829944 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 67603.1us
I20260709 12:16:33.844564 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:33.850973 28255 leader_election.cc:343] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.858502 28482 raft_consensus.cc:359] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.859615 28482 raft_consensus.cc:385] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.859966 28482 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.860754 28482 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.862686 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.866781 28547 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:33.868167 28187 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.874583 28482 ts_tablet_manager.cc:1434] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.001s
I20260709 12:16:33.875432 28482 tablet_bootstrap.cc:492] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
W20260709 12:16:33.874090 28257 leader_election.cc:343] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.889679 28547 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.896006 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:33.896822 28365 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:33.905185 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:33.906098 28365 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.891813 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:33.912923 28484 tablet_bootstrap.cc:654] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.906548 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:33.911581 28601 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:33.917963 28504 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:33.923594 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52847.2us
I20260709 12:16:33.931070 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:33.932071 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.912081 28547 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.933306 28547 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.935839 28481 tablet_bootstrap.cc:492] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:33.936358 28481 ts_tablet_manager.cc:1403] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.296s	user 0.007s	sys 0.013s
I20260709 12:16:33.938128 28547 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:33.942886 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:33.943632 28439 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:33.943866 28481 raft_consensus.cc:359] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.944680 28481 raft_consensus.cc:385] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.945045 28481 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.944932 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:33.945976 28547 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.946465 28547 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.946818 28547 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.947086 28481 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.949254 28481 ts_tablet_manager.cc:1434] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:16:33.951448 28481 tablet_bootstrap.cc:492] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:33.951653 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:33.952881 28517 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.952991 28288 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:33.953321 28517 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.953811 28517 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.957214 28482 tablet_bootstrap.cc:654] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:33.958110 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 48346.5us
I20260709 12:16:33.950284 28589 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:16:33.962461 28517 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:33.964344 28365 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:33.964672 28517 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:33.974284 28547 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:33.985545 28404 leader_election.cc:343] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:33.986457 28404 leader_election.cc:343] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.001490 28482 tablet_bootstrap.cc:492] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:34.002074 28481 tablet_bootstrap.cc:654] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.000058 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:34.002074 28482 ts_tablet_manager.cc:1403] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.127s	user 0.016s	sys 0.001s
I20260709 12:16:34.003911 28608 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.004479 28608 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:34.009402 28484 tablet_bootstrap.cc:492] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.008931 28608 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.010049 28484 ts_tablet_manager.cc:1403] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.201s	user 0.016s	sys 0.011s
I20260709 12:16:34.013492 28482 raft_consensus.cc:359] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.014354 28482 raft_consensus.cc:385] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.014685 28482 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.015439 28482 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.023667 28482 ts_tablet_manager.cc:1434] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:16:34.024189 28589 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.025044 28482 tablet_bootstrap.cc:492] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:34.032330 28547 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.038756 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:34.041538 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:34.044660 28365 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.039578 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:34.046096 28287 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.046509 28484 raft_consensus.cc:359] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.047219 28484 raft_consensus.cc:385] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.047503 28484 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.048974 28484 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.049607 28440 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.053349 28484 ts_tablet_manager.cc:1434] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 12:16:34.052474 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:34.055190 28484 tablet_bootstrap.cc:492] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.056494 28188 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:34.058914 28440 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:34.062415 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:34.063352 28589 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.063848 28589 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:34.064678 28589 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.071228 28290 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.076396 28287 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:34.086403 28505 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.088829 28365 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:34.089550 28610 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:34.159646 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:34.160564 28404 leader_election.cc:343] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:34.161163 28238 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:34.171131 28481 tablet_bootstrap.cc:492] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:34.171752 28481 ts_tablet_manager.cc:1403] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.221s	user 0.012s	sys 0.023s
I20260709 12:16:34.172955 28482 tablet_bootstrap.cc:654] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.177825 28187 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.178310 28481 raft_consensus.cc:359] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.179142 28481 raft_consensus.cc:385] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.179457 28481 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.180171 28505 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.180285 28481 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.183473 28484 tablet_bootstrap.cc:654] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.187513 28481 ts_tablet_manager.cc:1434] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 12:16:34.188772 28481 tablet_bootstrap.cc:492] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:34.161384 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:34.194185 28610 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.200903 28610 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:34.201933 28610 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.207048 28544 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.229194 28187 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:34.242100 28481 tablet_bootstrap.cc:654] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.244295 28482 tablet_bootstrap.cc:492] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:34.244839 28482 ts_tablet_manager.cc:1403] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.220s	user 0.013s	sys 0.023s
I20260709 12:16:34.247684 28482 raft_consensus.cc:359] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.248406 28482 raft_consensus.cc:385] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.248678 28482 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.249318 28482 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.250415 28484 tablet_bootstrap.cc:492] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.250929 28484 ts_tablet_manager.cc:1403] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.196s	user 0.014s	sys 0.000s
I20260709 12:16:34.251865 28482 ts_tablet_manager.cc:1434] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:34.253796 28484 raft_consensus.cc:359] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.254555 28484 raft_consensus.cc:385] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.254756 28482 tablet_bootstrap.cc:492] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:34.254842 28484 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.255788 28484 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.258039 28484 ts_tablet_manager.cc:1434] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:34.262176 28482 tablet_bootstrap.cc:654] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.263721 28484 tablet_bootstrap.cc:492] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.273557 28481 tablet_bootstrap.cc:492] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:34.274287 28481 ts_tablet_manager.cc:1403] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.086s	user 0.020s	sys 0.001s
I20260709 12:16:34.275336 28484 tablet_bootstrap.cc:654] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.277596 28481 raft_consensus.cc:359] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.278486 28481 raft_consensus.cc:385] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.278887 28481 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.280055 28482 tablet_bootstrap.cc:492] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:34.280598 28482 ts_tablet_manager.cc:1403] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.026s	user 0.003s	sys 0.014s
I20260709 12:16:34.280448 28481 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.283915 28482 raft_consensus.cc:359] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.284628 28482 raft_consensus.cc:385] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.284898 28482 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.285895 28482 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.291127 28482 ts_tablet_manager.cc:1434] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:16:34.292344 28482 tablet_bootstrap.cc:492] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:34.296097 28481 ts_tablet_manager.cc:1434] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:16:34.296890 28544 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.297441 28544 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.298589 28484 tablet_bootstrap.cc:492] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.299309 28484 ts_tablet_manager.cc:1403] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.013s
I20260709 12:16:34.299686 28544 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.299978 28482 tablet_bootstrap.cc:654] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.301460 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:34.301878 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.302199 28484 raft_consensus.cc:359] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.302597 28287 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:34.302879 28484 raft_consensus.cc:385] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.303174 28484 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
W20260709 12:16:34.303627 28332 leader_election.cc:343] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.304047 28484 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.304865 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:34.306100 28544 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.306478 28544 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.306802 28544 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.308624 28481 tablet_bootstrap.cc:492] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:34.312615 28484 ts_tablet_manager.cc:1434] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:16:34.313845 28484 tablet_bootstrap.cc:492] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.316303 28481 tablet_bootstrap.cc:654] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.317463 28482 tablet_bootstrap.cc:492] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:34.318075 28482 ts_tablet_manager.cc:1403] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.006s
I20260709 12:16:34.319871 28484 tablet_bootstrap.cc:654] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.333348 28482 raft_consensus.cc:359] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.334359 28482 raft_consensus.cc:385] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.334748 28482 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.335668 28482 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.337493 28482 ts_tablet_manager.cc:1434] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:16:34.338882 28482 tablet_bootstrap.cc:492] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:34.345428 28482 tablet_bootstrap.cc:654] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.349624 28484 tablet_bootstrap.cc:492] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.341079 28544 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.350325 28484 ts_tablet_manager.cc:1403] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.019s
I20260709 12:16:34.352743 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:34.353417 28287 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.353850 28481 tablet_bootstrap.cc:492] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:34.353523 28484 raft_consensus.cc:359] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.354365 28481 ts_tablet_manager.cc:1403] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.046s	user 0.022s	sys 0.003s
I20260709 12:16:34.354523 28484 raft_consensus.cc:385] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.354923 28484 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.355751 28484 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.357234 28610 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.357996 28484 ts_tablet_manager.cc:1434] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:16:34.357764 28610 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.359562 28481 raft_consensus.cc:359] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.361275 28481 raft_consensus.cc:385] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.359882 28610 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:34.360976 28484 tablet_bootstrap.cc:492] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.361724 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:34.362434 28366 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:34.362963 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.363270 28287 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:34.361606 28481 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.363632 28290 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:34.363767 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:34.364619 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:34.365551 28589 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.366039 28589 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:34.364738 28610 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.367115 28544 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.366928 28589 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.368323 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:34.369432 28484 tablet_bootstrap.cc:654] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:34.369905 28332 leader_election.cc:343] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.366806 28610 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.371120 28610 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.376188 28482 tablet_bootstrap.cc:492] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:34.376948 28482 ts_tablet_manager.cc:1403] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.005s
I20260709 12:16:34.380236 28482 raft_consensus.cc:359] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.381209 28482 raft_consensus.cc:385] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.381572 28482 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.382400 28482 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.384651 28482 ts_tablet_manager.cc:1434] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:34.385670 28482 tablet_bootstrap.cc:492] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:34.388403 28189 catalog_manager.cc:5697] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.366806 28481 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.392141 28482 tablet_bootstrap.cc:654] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.396736 28481 ts_tablet_manager.cc:1434] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 12:16:34.398051 28481 tablet_bootstrap.cc:492] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:34.412465 28287 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.416453 28517 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.418370 28517 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.418959 28517 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.426975 28544 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.427647 28517 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:34.427573 28544 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.430923 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.431748 28288 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:34.434247 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.434589 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:34.436431 28484 tablet_bootstrap.cc:492] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.436939 28484 ts_tablet_manager.cc:1403] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.076s	user 0.029s	sys 0.043s
I20260709 12:16:34.437036 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:34.438261 28547 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:34.438339 28404 leader_election.cc:343] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.438704 28547 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.439069 28547 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:34.439122 28404 leader_election.cc:343] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.439606 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:34.439909 28484 raft_consensus.cc:359] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.440608 28484 raft_consensus.cc:385] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.440857 28484 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.442611 28484 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.444489 28484 ts_tablet_manager.cc:1434] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:16:34.445027 28609 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:34.445636 28544 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.445853 28609 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.447697 28482 tablet_bootstrap.cc:492] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:34.448206 28482 ts_tablet_manager.cc:1403] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.063s	user 0.033s	sys 0.007s
I20260709 12:16:34.448819 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:34.449590 28438 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:34.450291 28484 tablet_bootstrap.cc:492] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.403268 28610 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.452431 28482 raft_consensus.cc:359] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.453853 28482 raft_consensus.cc:385] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.454989 28482 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.456256 28482 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.457192 28484 tablet_bootstrap.cc:654] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.458909 28482 ts_tablet_manager.cc:1434] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:16:34.460036 28482 tablet_bootstrap.cc:492] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:34.463672 28539 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.464263 28539 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.465773 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:34.466567 28365 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.467154 28539 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.467955 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:34.453050 28610 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:34.453308 28544 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.470571 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:34.471171 28290 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.470575 28544 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.473170 28544 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.465605 28547 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.476554 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:34.476768 28539 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.477283 28539 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.478472 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:34.477494 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.481959 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:34.482697 28289 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:34.483006 28332 leader_election.cc:343] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.483603 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:34.484273 28363 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:34.485145 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:34.485975 28290 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:34.486148 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:34.488833 28365 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:34.491488 28481 tablet_bootstrap.cc:654] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.492447 28610 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.493024 28289 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
W20260709 12:16:34.494570 28330 leader_election.cc:343] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.497843 28610 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.499938 28601 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.504056 28601 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.504446 28539 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.504515 28601 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.503561 28504 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:34.507498 28484 tablet_bootstrap.cc:492] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.502919 28330 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:34.508109 28484 ts_tablet_manager.cc:1403] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.014s
I20260709 12:16:34.508474 28610 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:34.509709 28544 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:34.510231 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:34.503139 28547 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.513904 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:34.514294 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:34.514607 28484 raft_consensus.cc:359] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.515617 28484 raft_consensus.cc:385] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.503361 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:34.515892 28484 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.512420 28544 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.514561 28438 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.516724 28544 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:34.516806 28440 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:34.517668 28544 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.518186 28610 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.518718 28610 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:34.519562 28610 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.521721 28484 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.525494 28484 ts_tablet_manager.cc:1434] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:16:34.526553 28484 tablet_bootstrap.cc:492] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.527029 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:34.527746 28365 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:34.549245 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.540525 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:34.553628 28484 tablet_bootstrap.cc:654] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.553311 28189 catalog_manager.cc:5697] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.547343 28438 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:34.556380 28482 tablet_bootstrap.cc:654] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.556106 28601 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.558041 28188 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:34.560669 28289 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:34.557121 28610 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.568259 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:34.569235 28366 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.569787 28601 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.573280 28547 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.573948 28547 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.576452 28547 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.578269 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:34.578948 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.580390 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:16:34.581018 28332 leader_election.cc:343] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.580405 28482 tablet_bootstrap.cc:492] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:34.581903 28482 ts_tablet_manager.cc:1403] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.122s	user 0.010s	sys 0.005s
I20260709 12:16:34.577668 28366 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
W20260709 12:16:34.582384 28330 leader_election.cc:343] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.583454 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:34.584321 28547 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:34.568470 28610 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.584950 28610 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.586824 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:34.587754 28601 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.588446 28601 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:34.589551 28601 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.594115 28547 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.592235 28481 tablet_bootstrap.cc:492] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:34.595573 28481 ts_tablet_manager.cc:1403] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.198s	user 0.014s	sys 0.002s
I20260709 12:16:34.601603 28484 tablet_bootstrap.cc:492] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.602838 28484 ts_tablet_manager.cc:1403] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.077s	user 0.021s	sys 0.002s
I20260709 12:16:34.604987 28484 raft_consensus.cc:359] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.605528 28484 raft_consensus.cc:385] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.594508 28482 raft_consensus.cc:359] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.605824 28484 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.606663 28484 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.607474 28481 raft_consensus.cc:359] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.608233 28481 raft_consensus.cc:385] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.608589 28481 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.608896 28484 ts_tablet_manager.cc:1434] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 12:16:34.609517 28482 raft_consensus.cc:385] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.610045 28484 tablet_bootstrap.cc:492] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.609972 28482 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.610910 28482 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.616989 28484 tablet_bootstrap.cc:654] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.619200 28482 ts_tablet_manager.cc:1434] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 12:16:34.619825 28481 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.620749 28289 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.621486 28482 tablet_bootstrap.cc:492] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:34.636108 28644 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.639496 28555 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.648963 28517 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.664652 28610 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.666040 28438 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:34.666024 28481 ts_tablet_manager.cc:1434] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 12:16:34.670132 28481 tablet_bootstrap.cc:492] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:34.675383 28504 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.676002 28504 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.678630 28539 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.679229 28539 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.679541 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:34.679620 28504 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.681696 28539 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:16:34.682597 28255 leader_election.cc:343] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.683498 28610 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:34.684760 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:34.682844 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:34.685563 28440 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:34.687819 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:34.688207 28257 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:34.687821 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:34.689698 28365 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:16:34.689976 28257 leader_election.cc:343] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.688511 28363 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:34.691449 28539 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.691808 28539 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.692149 28539 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.693581 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:34.694710 28504 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:34.697973 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:34.699996 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:34.699563 28188 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.702853 28517 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:34.727713 28482 tablet_bootstrap.cc:654] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.709110 28539 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.751298 28481 tablet_bootstrap.cc:654] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.755537 28484 tablet_bootstrap.cc:492] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.756160 28484 ts_tablet_manager.cc:1403] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.146s	user 0.070s	sys 0.052s
I20260709 12:16:34.760643 28539 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:16:34.761080 28332 consensus_peers.cc:597] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:34.761606 28484 raft_consensus.cc:359] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.763523 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:34.765319 28439 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.762161 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:34.770030 28363 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.773790 28481 tablet_bootstrap.cc:492] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:34.774312 28481 ts_tablet_manager.cc:1403] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.000s
I20260709 12:16:34.776490 28363 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:34.776564 28481 raft_consensus.cc:359] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.777546 28481 raft_consensus.cc:385] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.777834 28481 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.777671 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:34.778838 28601 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.778558 28481 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.779439 28601 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:34.780308 28601 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.780951 28439 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:34.781628 28481 ts_tablet_manager.cc:1434] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:34.782685 28481 tablet_bootstrap.cc:492] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:34.783216 28484 raft_consensus.cc:385] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.783535 28484 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.784199 28484 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.786536 28484 ts_tablet_manager.cc:1434] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:16:34.787496 28484 tablet_bootstrap.cc:492] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.790618 28481 tablet_bootstrap.cc:654] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.792992 28484 tablet_bootstrap.cc:654] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.793725 28482 tablet_bootstrap.cc:492] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:34.794343 28482 ts_tablet_manager.cc:1403] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.173s	user 0.014s	sys 0.001s
I20260709 12:16:34.799175 28482 raft_consensus.cc:359] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.800022 28482 raft_consensus.cc:385] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.800299 28482 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.801312 28482 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.805312 28482 ts_tablet_manager.cc:1434] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:34.806625 28482 tablet_bootstrap.cc:492] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:34.809154 28544 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.809763 28544 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.812003 28544 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.813724 28482 tablet_bootstrap.cc:654] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.814608 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:34.815424 28439 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:34.817019 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.817104 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:34.817574 28481 tablet_bootstrap.cc:492] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:34.818132 28481 ts_tablet_manager.cc:1403] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.008s
I20260709 12:16:34.818277 28544 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.818725 28544 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:34.818761 28330 leader_election.cc:343] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.819217 28544 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.821787 28481 raft_consensus.cc:359] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.822619 28481 raft_consensus.cc:385] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.822914 28481 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.823715 28481 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.827329 28481 ts_tablet_manager.cc:1434] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:34.830749 28187 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.834390 28481 tablet_bootstrap.cc:492] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:34.835788 28484 tablet_bootstrap.cc:492] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.836305 28484 ts_tablet_manager.cc:1403] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.049s	user 0.026s	sys 0.013s
I20260709 12:16:34.839169 28484 raft_consensus.cc:359] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.839895 28484 raft_consensus.cc:385] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.840179 28484 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.840899 28484 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.842039 28481 tablet_bootstrap.cc:654] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.843475 28484 ts_tablet_manager.cc:1434] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:34.844523 28484 tablet_bootstrap.cc:492] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.851676 28482 tablet_bootstrap.cc:492] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:34.852066 28484 tablet_bootstrap.cc:654] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.852253 28482 ts_tablet_manager.cc:1403] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.002s
I20260709 12:16:34.855347 28482 raft_consensus.cc:359] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.857096 28544 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.857771 28484 tablet_bootstrap.cc:492] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.858358 28484 ts_tablet_manager.cc:1403] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 12:16:34.855880 28482 raft_consensus.cc:385] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.859372 28482 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.860376 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:34.861618 28481 tablet_bootstrap.cc:492] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:34.862196 28481 ts_tablet_manager.cc:1403] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.006s
W20260709 12:16:34.862742 28330 leader_election.cc:343] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.862355 28484 raft_consensus.cc:359] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.863513 28484 raft_consensus.cc:385] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.863502 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:34.864037 28484 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.864248 28439 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.864913 28481 raft_consensus.cc:359] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.864836 28484 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.865494 28481 raft_consensus.cc:385] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.865832 28481 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.866405 28481 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.868343 28481 ts_tablet_manager.cc:1434] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:34.869721 28481 tablet_bootstrap.cc:492] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:34.871451 28484 ts_tablet_manager.cc:1434] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:16:34.872474 28484 tablet_bootstrap.cc:492] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:34.874395 28544 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.877481 28439 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:34.879933 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:34.878321 28482 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.882393 28547 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.883158 28547 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.881203 28544 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.884475 28544 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:34.886976 28482 ts_tablet_manager.cc:1434] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:16:34.887796 28482 tablet_bootstrap.cc:492] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:34.885344 28544 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.890415 28481 tablet_bootstrap.cc:654] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.896018 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.897141 28289 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:34.898506 28547 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.898855 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:34.900650 28332 leader_election.cc:343] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.902212 28484 tablet_bootstrap.cc:654] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.906344 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:34.908185 28555 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.910084 28555 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.910496 28555 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.913468 28547 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.916657 28505 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.918557 28482 tablet_bootstrap.cc:654] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.922572 28517 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.923174 28517 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.925506 28517 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:34.927536 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:34.928288 28290 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:34.929708 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:34.930169 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:34.930712 28517 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.930868 28363 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:34.931109 28517 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.931456 28517 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.938169 28555 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.940785 28555 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:34.942589 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:34.943524 28290 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.945003 28481 tablet_bootstrap.cc:492] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:34.945863 28481 ts_tablet_manager.cc:1403] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.076s	user 0.024s	sys 0.018s
I20260709 12:16:34.948858 28481 raft_consensus.cc:359] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.949563 28481 raft_consensus.cc:385] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.949837 28481 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.951224 28290 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:34.952800 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:34.953614 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:16:34.957206 28332 leader_election.cc:343] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.958014 28287 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.958338 28517 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.962515 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:34.963129 28363 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.975864 28363 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:34.977286 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:34.978185 28610 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.978646 28610 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:34.978012 28188 catalog_manager.cc:5697] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.979425 28610 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.982789 28505 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.983304 28505 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:34.984124 28505 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.986624 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:34.987310 28290 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.993835 28484 tablet_bootstrap.cc:492] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:34.994374 28484 ts_tablet_manager.cc:1403] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.122s	user 0.017s	sys 0.008s
I20260709 12:16:34.950730 28481 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.998337 28484 raft_consensus.cc:359] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:34.999063 28484 raft_consensus.cc:385] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.002033 28484 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.002478 28589 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.002894 28601 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.002759 28484 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.004464 28644 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.010211 28608 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.010845 28608 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.013166 28608 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.014495 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:35.015296 28366 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:35.016120 28547 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.015592 28188 catalog_manager.cc:5697] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.016630 28547 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.019110 28547 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.020874 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:35.021528 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.021878 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:35.022689 28517 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.023360 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.024075 28288 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:35.024981 28187 catalog_manager.cc:5697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:35.025867 28330 leader_election.cc:343] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.027885 28482 tablet_bootstrap.cc:492] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:35.028386 28482 ts_tablet_manager.cc:1403] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.141s	user 0.030s	sys 0.015s
W20260709 12:16:35.028537 28332 leader_election.cc:343] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.029064 28332 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:35.029896 28589 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.031255 28482 raft_consensus.cc:359] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.032027 28482 raft_consensus.cc:385] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.032302 28482 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.023288 28517 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.032989 28482 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.033555 28517 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.033965 28517 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.041059 28482 ts_tablet_manager.cc:1434] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:16:35.040956 28517 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.046778 28482 tablet_bootstrap.cc:492] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:35.047142 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:35.047778 28288 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.049650 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:35.050318 28366 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.055150 28481 ts_tablet_manager.cc:1434] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.109s	user 0.006s	sys 0.001s
I20260709 12:16:35.055730 28484 ts_tablet_manager.cc:1434] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 12:16:35.056226 28481 tablet_bootstrap.cc:492] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:35.056686 28484 tablet_bootstrap.cc:492] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:35.057605 28366 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:35.058730 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:35.060210 28673 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.060796 28673 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:35.061702 28673 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.063478 28484 tablet_bootstrap.cc:654] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.063742 28517 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.070384 28610 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.072455 28482 tablet_bootstrap.cc:654] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.085326 28482 tablet_bootstrap.cc:492] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:35.085829 28482 ts_tablet_manager.cc:1403] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.003s
I20260709 12:16:35.088624 28482 raft_consensus.cc:359] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.089332 28482 raft_consensus.cc:385] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.090029 28481 tablet_bootstrap.cc:654] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.086651 28290 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:35.089602 28482 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.093843 28288 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:35.094444 28482 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.099656 28484 tablet_bootstrap.cc:492] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:35.100258 28484 ts_tablet_manager.cc:1403] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.003s
I20260709 12:16:35.104615 28482 ts_tablet_manager.cc:1434] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:16:35.117774 28482 tablet_bootstrap.cc:492] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:35.117275 28484 raft_consensus.cc:359] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.118357 28484 raft_consensus.cc:385] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.118731 28484 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.122215 28517 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.133958 28484 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.138480 28481 tablet_bootstrap.cc:492] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:35.139008 28481 ts_tablet_manager.cc:1403] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.083s	user 0.008s	sys 0.011s
I20260709 12:16:35.135448 28517 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.141072 28188 catalog_manager.cc:5697] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:35.145498 28517 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.145787 28539 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.146641 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.147125 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:35.148026 28484 ts_tablet_manager.cc:1434] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.006s
I20260709 12:16:35.149016 28484 tablet_bootstrap.cc:492] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:35.152006 28481 raft_consensus.cc:359] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.152673 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:35.152683 28481 raft_consensus.cc:385] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.154278 28481 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.153959 28608 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.155412 28482 tablet_bootstrap.cc:654] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.154999 28481 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.157434 28481 ts_tablet_manager.cc:1434] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:16:35.158358 28481 tablet_bootstrap.cc:492] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:35.164088 28484 tablet_bootstrap.cc:654] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.164472 28481 tablet_bootstrap.cc:654] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.170938 28504 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:35.192786 28484 tablet_bootstrap.cc:492] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:35.193382 28484 ts_tablet_manager.cc:1403] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.045s	user 0.006s	sys 0.014s
I20260709 12:16:35.204888 28555 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.205451 28555 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.207624 28555 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.209860 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:35.211730 28601 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.198581 28484 raft_consensus.cc:359] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:35.214515 28332 leader_election.cc:343] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.214768 28484 raft_consensus.cc:385] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.215065 28484 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.215731 28484 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.216607 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:16:35.218030 28330 leader_election.cc:343] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.218468 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:35.220783 28544 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.226965 28484 ts_tablet_manager.cc:1434] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 12:16:35.228006 28484 tablet_bootstrap.cc:492] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:35.234934 28484 tablet_bootstrap.cc:654] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.248769 28482 tablet_bootstrap.cc:492] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:35.249238 28482 ts_tablet_manager.cc:1403] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.132s	user 0.022s	sys 0.011s
I20260709 12:16:35.257206 28517 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:16:35.257833 28517 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.259390 28482 raft_consensus.cc:359] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.259964 28517 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.260095 28482 raft_consensus.cc:385] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.260425 28482 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.261117 28482 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.261911 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:35.262789 28673 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.268767 28363 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:16:35.268784 28673 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.270076 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:35.271054 28673 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.263366 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.271871 28482 ts_tablet_manager.cc:1434] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:16:35.265635 28439 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.272243 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.271579 28673 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:35.272868 28482 tablet_bootstrap.cc:492] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:35.273221 28673 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:16:35.273471 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:35.274133 28287 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:35.274233 28363 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:35.273622 28673 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:35.276436 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:35.272864 28289 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:35.277853 28517 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.278791 28517 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.279129 28517 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.279815 28484 tablet_bootstrap.cc:492] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:35.280323 28484 ts_tablet_manager.cc:1403] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.005s
I20260709 12:16:35.283176 28484 raft_consensus.cc:359] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.283866 28484 raft_consensus.cc:385] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.284149 28484 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.285522 28482 tablet_bootstrap.cc:654] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.285545 28484 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.288874 28673 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.293577 28517 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.299767 28517 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.300104 28484 ts_tablet_manager.cc:1434] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:16:35.301136 28484 tablet_bootstrap.cc:492] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:35.302922 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:35.303572 28363 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.294565 28481 tablet_bootstrap.cc:492] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:35.304667 28481 ts_tablet_manager.cc:1403] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.147s	user 0.017s	sys 0.048s
I20260709 12:16:35.307102 28484 tablet_bootstrap.cc:654] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.306934 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:35.307543 28481 raft_consensus.cc:359] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.308250 28481 raft_consensus.cc:385] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.307541 28289 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:35.297972 28238 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:35.313316 28481 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.313581 28484 tablet_bootstrap.cc:492] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:35.314294 28484 ts_tablet_manager.cc:1403] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 12:16:35.314196 28481 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.317366 28481 ts_tablet_manager.cc:1434] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 12:16:35.318383 28481 tablet_bootstrap.cc:492] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:35.319376 28484 raft_consensus.cc:359] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.320096 28484 raft_consensus.cc:385] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.320384 28484 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.320739 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:35.322610 28287 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:35.321164 28484 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.326691 28544 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.328621 28363 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:35.334573 28484 ts_tablet_manager.cc:1434] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 12:16:35.335564 28484 tablet_bootstrap.cc:492] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:35.329752 28289 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:35.339900 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:35.341008 28673 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.343447 28644 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.344607 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:35.345649 28482 tablet_bootstrap.cc:492] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:35.346251 28482 ts_tablet_manager.cc:1403] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.005s
I20260709 12:16:35.346755 28539 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.347657 28539 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.351506 28539 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.354208 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:35.355520 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:35.362818 28644 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:35.364286 28539 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:35.368799 28255 leader_election.cc:343] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:35.370669 28257 leader_election.cc:343] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.371147 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:35.366925 28539 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.375041 28601 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.375676 28539 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.376605 28484 tablet_bootstrap.cc:654] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.377882 28363 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:16:35.378677 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:35.378881 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:35.379874 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:35.379982 28482 raft_consensus.cc:359] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.380724 28517 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:35.381467 28517 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:35.382433 28517 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.383486 28481 tablet_bootstrap.cc:654] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.380720 28482 raft_consensus.cc:385] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.384899 28482 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.385898 28673 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.386359 28673 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.385599 28482 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.387737 28644 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.388548 28255 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:35.389285 28482 ts_tablet_manager.cc:1434] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 12:16:35.389787 28539 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.390359 28482 tablet_bootstrap.cc:492] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:35.391772 28673 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.392658 28289 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.416715 28482 tablet_bootstrap.cc:654] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.420703 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.424309 28608 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:35.426028 28287 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:16:35.426573 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:35.439137 28505 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:35.440835 28484 tablet_bootstrap.cc:492] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:35.477108 28484 ts_tablet_manager.cc:1403] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.142s	user 0.006s	sys 0.008s
I20260709 12:16:35.478881 28482 tablet_bootstrap.cc:492] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:35.479347 28482 ts_tablet_manager.cc:1403] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.089s	user 0.009s	sys 0.003s
I20260709 12:16:35.480527 28484 raft_consensus.cc:359] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.481397 28484 raft_consensus.cc:385] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.442538 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:35.481763 28484 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.482805 28484 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.483377 28517 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.485654 28484 ts_tablet_manager.cc:1434] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:35.482471 28482 raft_consensus.cc:359] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.486239 28482 raft_consensus.cc:385] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.486524 28482 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.486949 28484 tablet_bootstrap.cc:492] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:35.475854 28187 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:35.487169 28482 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.490123 28482 ts_tablet_manager.cc:1434] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:16:35.491000 28482 tablet_bootstrap.cc:492] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:35.491187 28481 tablet_bootstrap.cc:492] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:35.491619 28481 ts_tablet_manager.cc:1403] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.173s	user 0.014s	sys 0.035s
I20260709 12:16:35.494827 28484 tablet_bootstrap.cc:654] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.494982 28481 raft_consensus.cc:359] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.495661 28481 raft_consensus.cc:385] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.495944 28481 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.496592 28481 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.498647 28481 ts_tablet_manager.cc:1434] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:35.500166 28481 tablet_bootstrap.cc:492] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:35.507084 28484 tablet_bootstrap.cc:492] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:35.507695 28484 ts_tablet_manager.cc:1403] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.007s
I20260709 12:16:35.520010 28484 raft_consensus.cc:359] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.520740 28484 raft_consensus.cc:385] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.520995 28484 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.521865 28505 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.522396 28505 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.526436 28187 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 2, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 2 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:35.527571 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.528262 28287 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:35.530296 28693 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.535357 28482 tablet_bootstrap.cc:654] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.535852 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:35.536213 28517 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.537125 28544 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.537498 28544 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.538074 28544 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.538215 28481 tablet_bootstrap.cc:654] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.541290 28693 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.557112 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:35.558809 28332 leader_election.cc:343] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.559403 28693 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.560065 28693 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.560511 28589 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.562443 28693 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.559473 28505 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.575852 28505 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.575937 28544 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.576395 28505 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.578615 28544 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.583695 28505 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.583499 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:35.585160 28365 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:35.585568 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.586282 28292 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:35.586999 28484 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.587678 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:35.591012 28693 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.587924 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:35.591202 28484 ts_tablet_manager.cc:1434] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.000s
I20260709 12:16:35.591401 28693 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.592154 28292 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.592940 28484 tablet_bootstrap.cc:492] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:35.594167 28601 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.594321 28693 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.594658 28601 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.596688 28601 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.599895 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:35.602411 28255 leader_election.cc:343] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.605618 28673 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.605885 28292 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:35.600121 28484 tablet_bootstrap.cc:654] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.614120 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.614854 28288 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:35.618220 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:35.619205 28481 tablet_bootstrap.cc:492] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:35.619375 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:35.619724 28481 ts_tablet_manager.cc:1403] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.120s	user 0.010s	sys 0.000s
I20260709 12:16:35.620070 28555 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.620221 28589 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.620833 28589 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.620976 28555 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:35.621208 28589 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.622104 28555 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.626084 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:35.627744 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:35.628393 28440 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:35.630388 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:35.635195 28332 leader_election.cc:343] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.635653 28589 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:35.638266 28257 leader_election.cc:343] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.638646 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:35.642623 28693 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.640357 28504 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.642071 28544 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:16:35.658190 28481 raft_consensus.cc:359] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.658917 28481 raft_consensus.cc:385] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.659193 28481 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.659967 28481 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.667384 28482 tablet_bootstrap.cc:492] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:35.667537 28481 ts_tablet_manager.cc:1434] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.000s
I20260709 12:16:35.667915 28482 ts_tablet_manager.cc:1403] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.177s	user 0.024s	sys 0.052s
I20260709 12:16:35.672178 28693 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.672856 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:35.673476 28292 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.673943 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:35.661128 28589 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.674525 28366 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.673231 28482 raft_consensus.cc:359] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.677639 28482 raft_consensus.cc:385] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.677973 28482 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.676661 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:35.682786 28438 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.682541 28482 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.684897 28482 ts_tablet_manager.cc:1434] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:16:35.685881 28482 tablet_bootstrap.cc:492] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:35.686663 28292 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:35.673460 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:35.687737 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:35.689878 28481 tablet_bootstrap.cc:492] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:35.690608 28517 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.687652 28287 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.694446 28517 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:35.695252 28517 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.722285 28484 tablet_bootstrap.cc:492] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:35.722904 28484 ts_tablet_manager.cc:1403] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.130s	user 0.029s	sys 0.019s
I20260709 12:16:35.713475 28287 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:35.729005 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:35.713653 28366 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:35.722949 28438 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:35.743191 28709 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:35.743980 28709 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:35.747650 28481 tablet_bootstrap.cc:654] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.747798 28484 raft_consensus.cc:359] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.748364 28484 raft_consensus.cc:385] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.748540 28484 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.744844 28709 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.752173 28482 tablet_bootstrap.cc:654] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.755193 28484 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.757395 28188 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.761538 28481 tablet_bootstrap.cc:492] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:35.764375 28481 ts_tablet_manager.cc:1403] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.006s
I20260709 12:16:35.766196 28484 ts_tablet_manager.cc:1434] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.003s
I20260709 12:16:35.767747 28484 tablet_bootstrap.cc:492] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:35.776933 28187 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:35.778151 28484 tablet_bootstrap.cc:654] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.785938 28481 raft_consensus.cc:359] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.786769 28481 raft_consensus.cc:385] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.787062 28481 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.787804 28481 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.789419 28481 ts_tablet_manager.cc:1434] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.000s
I20260709 12:16:35.795156 28481 tablet_bootstrap.cc:492] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:35.800493 28366 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:35.804185 28673 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:35.807705 28188 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.813208 28482 tablet_bootstrap.cc:492] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:35.815037 28482 ts_tablet_manager.cc:1403] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.129s	user 0.010s	sys 0.005s
I20260709 12:16:35.819883 28484 tablet_bootstrap.cc:492] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:35.821700 28484 ts_tablet_manager.cc:1403] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.054s	user 0.020s	sys 0.011s
I20260709 12:16:35.820087 28481 tablet_bootstrap.cc:654] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.831127 28608 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.832628 28709 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.833158 28709 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.835631 28709 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.839511 28481 tablet_bootstrap.cc:492] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:35.840092 28481 ts_tablet_manager.cc:1403] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.007s
I20260709 12:16:35.840317 28288 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:35.841027 28482 raft_consensus.cc:359] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.839555 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.841766 28482 raft_consensus.cc:385] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.842128 28482 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.842800 28482 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.843075 28481 raft_consensus.cc:359] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.843711 28481 raft_consensus.cc:385] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.843977 28481 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.844668 28481 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.824510 28484 raft_consensus.cc:359] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.845535 28484 raft_consensus.cc:385] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.845906 28484 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.846668 28484 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.847574 28481 ts_tablet_manager.cc:1434] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:35.845124 28482 ts_tablet_manager.cc:1434] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.024s	user 0.001s	sys 0.003s
I20260709 12:16:35.848763 28482 tablet_bootstrap.cc:492] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:35.849001 28484 ts_tablet_manager.cc:1434] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.000s
I20260709 12:16:35.850023 28484 tablet_bootstrap.cc:492] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
W20260709 12:16:35.852754 28330 leader_election.cc:343] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.855316 28482 tablet_bootstrap.cc:654] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.856392 28484 tablet_bootstrap.cc:654] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.857760 28693 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:35.858645 28481 tablet_bootstrap.cc:492] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:35.866398 28481 tablet_bootstrap.cc:654] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.890350 28482 tablet_bootstrap.cc:492] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:35.890974 28482 ts_tablet_manager.cc:1403] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.004s
I20260709 12:16:35.893788 28482 raft_consensus.cc:359] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.891129 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:35.894519 28482 raft_consensus.cc:385] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.894815 28482 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.895500 28482 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.897953 28482 ts_tablet_manager.cc:1434] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
W20260709 12:16:35.899135 28332 leader_election.cc:343] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.915009 28482 tablet_bootstrap.cc:492] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:35.919476 28481 tablet_bootstrap.cc:492] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:35.920046 28481 ts_tablet_manager.cc:1403] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.014s
I20260709 12:16:35.921654 28693 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.922240 28693 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.922658 28481 raft_consensus.cc:359] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.923352 28481 raft_consensus.cc:385] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.923641 28481 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.924520 28693 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:35.924304 28481 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.926028 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.926894 28288 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:16:35.926842 28481 ts_tablet_manager.cc:1434] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:35.904414 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:35.930395 28544 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.939728 28481 tablet_bootstrap.cc:492] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:35.887984 28608 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.048s
I20260709 12:16:35.950865 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:35.954519 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:35.960877 28484 tablet_bootstrap.cc:492] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:35.961370 28484 ts_tablet_manager.cc:1403] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.112s	user 0.046s	sys 0.026s
I20260709 12:16:35.958055 28608 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.965111 28484 raft_consensus.cc:359] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.967393 28484 raft_consensus.cc:385] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:35.967785 28484 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:35.968724 28484 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.971135 28484 ts_tablet_manager.cc:1434] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 12:16:35.972256 28484 tablet_bootstrap.cc:492] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:35.979539 28484 tablet_bootstrap.cc:654] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.981403 28608 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:35.982151 28709 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.982772 28709 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:35.984647 28481 tablet_bootstrap.cc:654] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.986800 28482 tablet_bootstrap.cc:654] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:35.987599 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:35.989224 28288 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:35.990090 28709 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:35.990821 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:35.991561 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:16:35.993362 28332 leader_election.cc:343] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:35.994451 28709 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:35.994885 28709 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.995246 28709 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:35.999246 28517 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.999902 28517 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.002189 28517 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:36.002887 28484 tablet_bootstrap.cc:492] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:36.003522 28484 ts_tablet_manager.cc:1403] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.031s	user 0.021s	sys 0.007s
I20260709 12:16:36.004034 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.004868 28363 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:36.005456 28482 tablet_bootstrap.cc:492] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:36.006361 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:36.007328 28481 tablet_bootstrap.cc:492] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:36.007448 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:36.007838 28482 ts_tablet_manager.cc:1403] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.094s	user 0.013s	sys 0.003s
I20260709 12:16:36.008056 28484 raft_consensus.cc:359] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.008651 28517 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.008911 28484 raft_consensus.cc:385] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.008235 28481 ts_tablet_manager.cc:1403] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.073s	user 0.007s	sys 0.007s
I20260709 12:16:36.009459 28484 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.010435 28484 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.008186 28288 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:36.009055 28517 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.012300 28481 raft_consensus.cc:359] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.012956 28517 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.013029 28484 ts_tablet_manager.cc:1434] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 12:16:36.014252 28484 tablet_bootstrap.cc:492] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:36.015261 28482 raft_consensus.cc:359] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.015933 28482 raft_consensus.cc:385] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.016218 28482 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.016901 28482 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.019114 28482 ts_tablet_manager.cc:1434] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:16:36.013042 28481 raft_consensus.cc:385] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.019371 28709 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.019874 28481 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.020678 28481 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.021049 28517 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.021656 28709 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.023545 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:36.024209 28436 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.026119 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:36.026782 28288 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.032168 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:36.032979 28366 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.033306 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:36.037320 28292 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.040009 28484 tablet_bootstrap.cc:654] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.040932 28366 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:36.032989 28517 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:36.042456 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:36.043165 28481 ts_tablet_manager.cc:1434] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.005s
I20260709 12:16:36.044979 28288 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:36.047290 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:36.049937 28544 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:36.050505 28544 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:36.051405 28544 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.053120 28482 tablet_bootstrap.cc:492] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:36.055301 28481 tablet_bootstrap.cc:492] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:36.058302 28484 tablet_bootstrap.cc:492] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:36.058880 28484 ts_tablet_manager.cc:1403] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.002s
I20260709 12:16:36.060518 28436 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:36.062043 28484 raft_consensus.cc:359] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.062848 28484 raft_consensus.cc:385] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.063158 28484 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.063961 28484 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.065608 28517 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:36.066267 28484 ts_tablet_manager.cc:1434] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:36.066321 28517 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:36.067433 28484 tablet_bootstrap.cc:492] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:36.069165 28539 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.069943 28539 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.070914 28482 tablet_bootstrap.cc:654] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.072216 28539 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.073240 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.074007 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:36.075063 28255 leader_election.cc:343] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:36.077230 28257 leader_election.cc:343] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.077895 28481 tablet_bootstrap.cc:654] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.077776 28257 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:36.078540 28601 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:36.077291 28539 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.079068 28539 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.081914 28504 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.082633 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.067230 28517 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.082566 28504 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.083348 28366 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:36.084340 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.084538 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:36.084710 28504 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.085335 28504 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.085704 28504 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.086658 28504 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.086715 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.087445 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:36.087824 28257 leader_election.cc:343] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.088119 28363 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:36.089564 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:36.089836 28539 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.090531 28539 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.090899 28539 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.091226 28539 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.087476 28437 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:36.097432 28484 tablet_bootstrap.cc:654] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.124435 28292 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:36.113845 28589 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:16:36.133038 28504 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.134219 28539 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.135613 28504 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.136317 28539 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.138202 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:36.139000 28363 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.139129 28484 tablet_bootstrap.cc:492] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:36.139418 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:36.139798 28484 ts_tablet_manager.cc:1403] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.073s	user 0.018s	sys 0.003s
I20260709 12:16:36.140116 28366 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.144821 28484 raft_consensus.cc:359] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.155053 28484 raft_consensus.cc:385] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.155416 28484 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.156371 28484 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.159678 28484 ts_tablet_manager.cc:1434] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:16:36.147260 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:36.163061 28484 tablet_bootstrap.cc:492] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:36.147605 28589 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.162588 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:36.168341 28438 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.169394 28481 tablet_bootstrap.cc:492] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:36.169974 28481 ts_tablet_manager.cc:1403] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.115s	user 0.012s	sys 0.013s
I20260709 12:16:36.145715 28673 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
W20260709 12:16:36.174055 28257 leader_election.cc:343] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.173557 28693 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.177582 28693 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.180019 28693 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:36.179912 28481 raft_consensus.cc:359] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.180924 28481 raft_consensus.cc:385] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.181311 28481 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.181807 28363 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:36.182410 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.182284 28481 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.183387 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:36.184450 28481 ts_tablet_manager.cc:1434] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.003s
I20260709 12:16:36.193842 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:36.195082 28292 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:36.190618 28504 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:36.197260 28504 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:36.198557 28504 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.200155 28505 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:36.191582 28189 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:36.202826 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:36.165411 28589 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.205201 28608 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.214071 28608 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.214555 28608 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.215919 28366 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:36.217336 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:36.218011 28365 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:36.219336 28539 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:36.219856 28539 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:36.218026 28589 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.242079 28481 tablet_bootstrap.cc:492] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:36.247639 28482 tablet_bootstrap.cc:492] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:36.248184 28482 ts_tablet_manager.cc:1403] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.195s	user 0.033s	sys 0.020s
I20260709 12:16:36.224227 28438 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:36.220705 28539 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.263561 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.267118 28188 catalog_manager.cc:5697] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.266587 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:36.276837 28292 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:36.279453 28330 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:36.288275 28555 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.288973 28555 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.289417 28555 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.294962 28484 tablet_bootstrap.cc:654] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.294240 28187 catalog_manager.cc:5697] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.296710 28589 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.270989 28544 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:16:36.308688 28482 raft_consensus.cc:359] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.309623 28482 raft_consensus.cc:385] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:36.301344 28734 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:16:36.316349 28482 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.317166 28482 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.297318 28589 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.335047 28709 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:36.319036 28644 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:16:36.297972 28608 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.349872 28673 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.353497 28504 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.354230 28504 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.353647 28673 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.366657 28504 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.367689 28673 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:36.368505 28440 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:36.369405 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.370268 28366 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:36.369119 28555 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.376684 28482 ts_tablet_manager.cc:1434] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.128s	user 0.011s	sys 0.007s
I20260709 12:16:36.377965 28482 tablet_bootstrap.cc:492] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:36.378754 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.379515 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:36.374780 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:36.380180 28555 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.380520 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:36.381014 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.392259 28438 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:36.392395 28366 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:16:36.382100 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:36.394366 28287 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.384930 28589 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.370287 28608 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:36.386476 28481 tablet_bootstrap.cc:654] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.386291 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:36.397135 28364 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:36.396958 28288 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:36.387233 28189 catalog_manager.cc:5697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.399389 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:36.400250 28504 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.388108 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.401152 28436 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:36.402056 28504 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.402503 28504 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.402863 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:36.404003 28440 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:36.388800 28484 tablet_bootstrap.cc:492] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:36.406000 28484 ts_tablet_manager.cc:1403] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.243s	user 0.023s	sys 0.028s
I20260709 12:16:36.406068 28608 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:36.365685 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 58812.4us
W20260709 12:16:36.383002 28735 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:16:36.416673 28484 raft_consensus.cc:359] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.417488 28484 raft_consensus.cc:385] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.417814 28484 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.418526 28484 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.420815 28484 ts_tablet_manager.cc:1434] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:16:36.406513 28608 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.407128 28589 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.423264 28484 tablet_bootstrap.cc:492] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:36.424726 28608 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.434901 28504 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.438493 28481 tablet_bootstrap.cc:492] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:36.439019 28481 ts_tablet_manager.cc:1403] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.235s	user 0.008s	sys 0.006s
I20260709 12:16:36.440500 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:36.441226 28363 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.442687 28481 raft_consensus.cc:359] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.443439 28481 raft_consensus.cc:385] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.443738 28481 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.444625 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:36.444422 28481 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.446660 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:36.448760 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:36.449007 28481 ts_tablet_manager.cc:1434] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:16:36.449643 28289 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 1: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:36.450937 28539 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.451493 28539 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.454018 28608 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.456162 28481 tablet_bootstrap.cc:492] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:36.453502 28539 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.459389 28736 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.461623 28482 tablet_bootstrap.cc:654] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.462347 28440 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:36.464246 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.455418 28504 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.472886 28736 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:36.480855 28255 leader_election.cc:343] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.490701 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:36.491158 28287 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:36.490116 28608 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:36.493105 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.493662 28736 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.494879 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.495112 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:16:36.510094 28255 leader_election.cc:343] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.514804 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:16:36.516185 28257 leader_election.cc:343] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.510123 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.518188 28693 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:36.517923 28257 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:36.519560 28736 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:36.519723 28289 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:36.520336 28693 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:36.498363 28330 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:36.527027 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:16:36.527107 28257 leader_election.cc:343] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.527633 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:36.530453 28736 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:36.534188 28743 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:36.537519 28693 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.540165 28673 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.540688 28673 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:36.541321 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51409.2us
I20260709 12:16:36.542843 28673 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:36.534894 28644 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:36.552582 28484 tablet_bootstrap.cc:654] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.555554 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:36.557852 28517 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:36.557968 28404 leader_election.cc:343] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.565403 28482 tablet_bootstrap.cc:492] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:36.567740 28482 ts_tablet_manager.cc:1403] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.190s	user 0.011s	sys 0.010s
I20260709 12:16:36.574451 28482 raft_consensus.cc:359] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.575572 28482 raft_consensus.cc:385] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.576090 28482 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.576874 28482 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.583474 28482 ts_tablet_manager.cc:1434] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:16:36.585016 28482 tablet_bootstrap.cc:492] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:36.586488 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.605543 28481 tablet_bootstrap.cc:654] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:36.606506 28404 leader_election.cc:343] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.606900 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:36.605782 28189 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:36.607918 28644 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:36.631546 28743 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.608043 28363 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:36.634207 28589 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.635468 28589 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:36.636559 28484 tablet_bootstrap.cc:492] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:36.636974 28484 ts_tablet_manager.cc:1403] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.214s	user 0.012s	sys 0.007s
I20260709 12:16:36.641567 28482 tablet_bootstrap.cc:654] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:36.646905 28238 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:36.647228 28484 raft_consensus.cc:359] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.648180 28484 raft_consensus.cc:385] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.648540 28484 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.649384 28484 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.650993 28484 ts_tablet_manager.cc:1434] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 12:16:36.652148 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:36.652509 28484 tablet_bootstrap.cc:492] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:36.652987 28736 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:36.653483 28736 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:36.654359 28736 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.658736 28589 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.679607 28481 tablet_bootstrap.cc:492] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:36.672950 28673 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:16:36.682305 28481 ts_tablet_manager.cc:1403] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.232s	user 0.014s	sys 0.007s
I20260709 12:16:36.684141 28555 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.694813 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:36.695174 28484 tablet_bootstrap.cc:654] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.695591 28441 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:36.696352 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:36.700816 28589 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.731105 28481 raft_consensus.cc:359] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.732111 28481 raft_consensus.cc:385] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.732398 28481 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.733174 28481 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.736264 28481 ts_tablet_manager.cc:1434] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.003s
I20260709 12:16:36.741298 28441 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:16:36.742551 28481 tablet_bootstrap.cc:492] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:36.748097 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:36.748768 28482 tablet_bootstrap.cc:492] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:36.749537 28482 ts_tablet_manager.cc:1403] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.165s	user 0.021s	sys 0.011s
I20260709 12:16:36.748773 28589 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:36.752096 28589 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:36.752815 28482 raft_consensus.cc:359] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.753664 28482 raft_consensus.cc:385] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.753994 28482 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.753011 28589 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.754731 28482 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.756994 28482 ts_tablet_manager.cc:1434] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:16:36.757690 28737 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:36.757779 28484 tablet_bootstrap.cc:492] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:36.758699 28484 ts_tablet_manager.cc:1403] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.106s	user 0.008s	sys 0.027s
I20260709 12:16:36.761940 28484 raft_consensus.cc:359] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.762699 28484 raft_consensus.cc:385] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.762758 28737 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.767206 28482 tablet_bootstrap.cc:492] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:36.770874 28481 tablet_bootstrap.cc:654] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.773820 28484 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.774682 28484 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.776507 28743 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.777093 28743 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.779413 28484 ts_tablet_manager.cc:1434] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:16:36.780599 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:36.781239 28291 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:36.780968 28743 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.782497 28484 tablet_bootstrap.cc:492] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:36.783005 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.786216 28189 catalog_manager.cc:5697] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.789849 28484 tablet_bootstrap.cc:654] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.773766 28482 tablet_bootstrap.cc:654] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.782649 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:16:36.801828 28332 leader_election.cc:343] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.801541 28736 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:36.803090 28441 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.811033 28589 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.811527 28589 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.811856 28589 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.819517 28673 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.820104 28673 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.824985 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:36.826010 28673 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:36.830502 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.832574 28555 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.832703 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:36.833177 28555 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.834056 28517 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:36.835471 28555 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.837797 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:36.838112 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.838585 28291 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
W20260709 12:16:36.839776 28332 leader_election.cc:343] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.840351 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:36.841347 28555 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.841774 28555 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.842181 28555 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.848878 28504 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:36.859315 28484 tablet_bootstrap.cc:492] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:36.858883 28555 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.859990 28484 ts_tablet_manager.cc:1403] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.027s
I20260709 12:16:36.861815 28481 tablet_bootstrap.cc:492] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:36.862422 28481 ts_tablet_manager.cc:1403] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.120s	user 0.008s	sys 0.005s
I20260709 12:16:36.847985 28188 catalog_manager.cc:5697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 1 to 2, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.862874 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:36.863559 28291 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.864372 28484 raft_consensus.cc:359] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.872767 28291 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:36.874847 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:36.878957 28484 raft_consensus.cc:385] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.881703 28484 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.858817 28589 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.882761 28484 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.884653 28589 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.887025 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:36.877978 28481 raft_consensus.cc:359] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:36.880136 28332 leader_election.cc:343] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.888025 28481 raft_consensus.cc:385] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.888345 28481 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.889060 28481 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.889652 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:36.894326 28481 ts_tablet_manager.cc:1434] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:16:36.894865 28589 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:36.895457 28589 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:36.896003 28481 tablet_bootstrap.cc:492] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:36.896297 28589 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.875177 28555 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.880604 28482 tablet_bootstrap.cc:492] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:36.887665 28291 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.900462 28482 ts_tablet_manager.cc:1403] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.133s	user 0.034s	sys 0.011s
I20260709 12:16:36.903313 28482 raft_consensus.cc:359] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.904361 28504 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.904541 28482 raft_consensus.cc:385] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.905155 28484 ts_tablet_manager.cc:1434] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.003s
I20260709 12:16:36.904975 28504 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.905002 28482 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.906620 28482 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.907437 28504 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.907645 28291 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:36.908973 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:36.909365 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:36.916239 28367 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:36.904364 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:36.917327 28482 ts_tablet_manager.cc:1434] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
W20260709 12:16:36.918026 28332 leader_election.cc:343] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.918354 28482 tablet_bootstrap.cc:492] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:36.910272 28481 tablet_bootstrap.cc:654] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:36.917284 28257 leader_election.cc:343] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.919992 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:36.921154 28504 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:36.921520 28504 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.921885 28504 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.915333 28484 tablet_bootstrap.cc:492] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:36.926308 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:36.927289 28555 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:36.928010 28555 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:36.928874 28555 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.935477 28504 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.935979 28484 tablet_bootstrap.cc:654] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.936173 28482 tablet_bootstrap.cc:654] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.939105 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:36.939759 28367 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:36.946063 28481 tablet_bootstrap.cc:492] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:36.946645 28481 ts_tablet_manager.cc:1403] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.012s
I20260709 12:16:36.949126 28367 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:36.953784 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:36.952234 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:36.954779 28737 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:36.955677 28737 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:16:36.955713 28257 leader_election.cc:343] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:36.956566 28737 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.950048 28481 raft_consensus.cc:359] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.952127 28504 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:36.958586 28481 raft_consensus.cc:385] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.958920 28481 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.959621 28481 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.962072 28481 ts_tablet_manager.cc:1434] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:16:36.963091 28481 tablet_bootstrap.cc:492] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:36.969894 28190 catalog_manager.cc:5697] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.970866 28481 tablet_bootstrap.cc:654] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:36.980732 28482 tablet_bootstrap.cc:492] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:36.981376 28482 ts_tablet_manager.cc:1403] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.005s
I20260709 12:16:36.984022 28189 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.984563 28484 tablet_bootstrap.cc:492] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:36.985225 28484 ts_tablet_manager.cc:1403] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.070s	user 0.008s	sys 0.008s
I20260709 12:16:36.985662 28189 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.992833 28481 tablet_bootstrap.cc:492] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:36.993377 28481 ts_tablet_manager.cc:1403] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.019s
I20260709 12:16:36.989280 28484 raft_consensus.cc:359] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.994287 28484 raft_consensus.cc:385] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.994580 28484 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.995234 28484 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.992057 28482 raft_consensus.cc:359] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.996177 28481 raft_consensus.cc:359] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:36.996737 28482 raft_consensus.cc:385] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.996860 28481 raft_consensus.cc:385] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:36.997083 28482 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.997156 28481 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:36.997849 28481 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.000062 28481 ts_tablet_manager.cc:1434] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:37.001040 28481 tablet_bootstrap.cc:492] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:37.007256 28481 tablet_bootstrap.cc:654] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.007665 28484 ts_tablet_manager.cc:1434] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.005s
I20260709 12:16:37.008749 28484 tablet_bootstrap.cc:492] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:37.012992 28481 tablet_bootstrap.cc:492] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:37.013566 28481 ts_tablet_manager.cc:1403] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:16:37.016530 28481 raft_consensus.cc:359] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.017230 28481 raft_consensus.cc:385] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.017509 28481 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.018353 28481 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.025518 28481 ts_tablet_manager.cc:1434] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:16:37.026743 28481 tablet_bootstrap.cc:492] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:36.997854 28482 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.029893 28482 ts_tablet_manager.cc:1434] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 12:16:37.031394 28482 tablet_bootstrap.cc:492] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:37.037607 28482 tablet_bootstrap.cc:654] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.038519 28481 tablet_bootstrap.cc:654] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.041265 28484 tablet_bootstrap.cc:654] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.048787 28482 tablet_bootstrap.cc:492] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:37.049083 28481 tablet_bootstrap.cc:492] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:37.049326 28482 ts_tablet_manager.cc:1403] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.004s
I20260709 12:16:37.049579 28481 ts_tablet_manager.cc:1403] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.003s
I20260709 12:16:37.052452 28481 raft_consensus.cc:359] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.053197 28481 raft_consensus.cc:385] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.053514 28481 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.056401 28481 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.057528 28484 tablet_bootstrap.cc:492] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:37.059170 28484 ts_tablet_manager.cc:1403] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.051s	user 0.011s	sys 0.003s
I20260709 12:16:37.053690 28482 raft_consensus.cc:359] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.063385 28482 raft_consensus.cc:385] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.063687 28482 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.063846 28693 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:37.066952 28484 raft_consensus.cc:359] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.067715 28484 raft_consensus.cc:385] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.069157 28484 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.069825 28481 ts_tablet_manager.cc:1434] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 12:16:37.064476 28482 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.070904 28481 tablet_bootstrap.cc:492] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:37.073359 28484 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.075575 28482 ts_tablet_manager.cc:1434] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.002s
I20260709 12:16:37.076629 28482 tablet_bootstrap.cc:492] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:37.079145 28481 tablet_bootstrap.cc:654] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.089494 28484 ts_tablet_manager.cc:1434] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.001s
I20260709 12:16:37.090422 28504 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.091110 28504 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.095918 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:37.097613 28255 leader_election.cc:343] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:37.100677 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:37.102523 28441 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:37.093688 28504 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.114434 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:37.106006 28481 tablet_bootstrap.cc:492] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:37.115612 28481 ts_tablet_manager.cc:1403] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.045s	user 0.021s	sys 0.012s
I20260709 12:16:37.116436 28504 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:37.116927 28504 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.117321 28504 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.106664 28484 tablet_bootstrap.cc:492] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:37.119096 28481 raft_consensus.cc:359] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.119860 28481 raft_consensus.cc:385] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.120141 28481 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.122511 28481 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.124696 28484 tablet_bootstrap.cc:654] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.108136 28482 tablet_bootstrap.cc:654] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.112141 28608 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:37.125186 28481 ts_tablet_manager.cc:1434] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:16:37.126343 28481 tablet_bootstrap.cc:492] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:37.132767 28481 tablet_bootstrap.cc:654] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.124876 28504 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.137291 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:37.139029 28482 tablet_bootstrap.cc:492] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
W20260709 12:16:37.139001 28255 leader_election.cc:343] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:37.139671 28482 ts_tablet_manager.cc:1403] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.000s
I20260709 12:16:37.139842 28481 tablet_bootstrap.cc:492] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:37.140342 28481 ts_tablet_manager.cc:1403] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:16:37.141439 28504 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.142527 28482 raft_consensus.cc:359] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.142911 28481 raft_consensus.cc:359] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.143276 28482 raft_consensus.cc:385] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.143504 28481 raft_consensus.cc:385] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.143792 28481 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.144791 28482 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.146883 28482 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.148489 28482 ts_tablet_manager.cc:1434] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:16:37.151229 28481 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.154474 28481 ts_tablet_manager.cc:1434] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.002s
I20260709 12:16:37.155586 28481 tablet_bootstrap.cc:492] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:37.160889 28481 tablet_bootstrap.cc:654] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.162305 28482 tablet_bootstrap.cc:492] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:37.171234 28482 tablet_bootstrap.cc:654] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.185070 28481 tablet_bootstrap.cc:492] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:37.185633 28481 ts_tablet_manager.cc:1403] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.012s
I20260709 12:16:37.186455 28482 tablet_bootstrap.cc:492] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:37.187631 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:37.188280 28441 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.191746 28481 raft_consensus.cc:359] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.192706 28481 raft_consensus.cc:385] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.193080 28481 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.194084 28481 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.196271 28441 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:37.196790 28481 ts_tablet_manager.cc:1434] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:16:37.193172 28482 ts_tablet_manager.cc:1403] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.003s
I20260709 12:16:37.200345 28482 raft_consensus.cc:359] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.201161 28482 raft_consensus.cc:385] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.201453 28482 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.202391 28482 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.203989 28481 tablet_bootstrap.cc:492] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:37.204878 28482 ts_tablet_manager.cc:1434] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:37.206048 28482 tablet_bootstrap.cc:492] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:37.208003 28484 tablet_bootstrap.cc:492] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:37.208859 28484 ts_tablet_manager.cc:1403] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.102s	user 0.016s	sys 0.026s
I20260709 12:16:37.210664 28481 tablet_bootstrap.cc:654] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.212131 28482 tablet_bootstrap.cc:654] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.215373 28484 raft_consensus.cc:359] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.216393 28484 raft_consensus.cc:385] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.216754 28484 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.217470 28484 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.218266 28709 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.218767 28709 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.220162 28484 ts_tablet_manager.cc:1434] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:37.220997 28709 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.221381 28484 tablet_bootstrap.cc:492] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:37.223155 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:37.223852 28441 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:37.225437 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:37.226398 28709 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:37.226820 28709 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.227205 28709 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.227474 28693 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.228016 28693 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.230671 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:37.231621 28504 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:37.231956 28441 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:37.232275 28504 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:37.233180 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:37.233134 28504 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.233922 28287 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:37.235184 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.231076 28693 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:37.236580 28737 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:37.237222 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:37.238119 28287 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:16:37.237466 28737 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:37.240557 28737 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:37.241479 28737 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.242749 28737 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:37.244874 28736 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.245415 28736 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.247821 28736 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.249869 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.250614 28367 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:37.251240 28481 tablet_bootstrap.cc:492] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:37.251824 28481 ts_tablet_manager.cc:1403] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.048s	user 0.017s	sys 0.005s
I20260709 12:16:37.252029 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:37.253314 28737 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:37.254262 28693 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.254830 28693 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.255544 28481 raft_consensus.cc:359] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.256269 28481 raft_consensus.cc:385] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.256552 28481 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.257081 28693 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:37.257336 28481 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.259054 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.259682 28481 ts_tablet_manager.cc:1434] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:16:37.259814 28367 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:37.260737 28481 tablet_bootstrap.cc:492] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:37.261719 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:37.262603 28693 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:37.262967 28693 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.263309 28693 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.266956 28709 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.268110 28482 tablet_bootstrap.cc:492] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:37.268136 28484 tablet_bootstrap.cc:654] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.268765 28482 ts_tablet_manager.cc:1403] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.008s
I20260709 12:16:37.268648 28189 catalog_manager.cc:5697] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:37.253795 28737 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.271066 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:37.270102 28737 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.271759 28441 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:37.273008 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:37.273563 28504 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.274457 28504 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.275331 28481 tablet_bootstrap.cc:654] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.282176 28504 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.284595 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.285449 28367 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:37.287506 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:37.288430 28673 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.288904 28504 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:37.289343 28504 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.289772 28504 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.291095 28482 raft_consensus.cc:359] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.297345 28673 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:37.299860 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:37.300536 28287 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.308393 28287 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:37.316013 28291 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:16:37.323644 28291 pending_rounds.cc:85] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 0
I20260709 12:16:37.324002 28291 pending_rounds.cc:107] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:16:37.324316 28291 raft_consensus.cc:2889] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:37.318845 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:37.325102 28693 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.325610 28437 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.328640 28737 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.333238 28437 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:37.340608 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:37.341233 28363 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.341619 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:37.342520 28589 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.342994 28589 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.345127 28589 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.345630 28589 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:37.346338 28589 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:37.347088 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:37.347652 28484 tablet_bootstrap.cc:492] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:37.348127 28484 ts_tablet_manager.cc:1403] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.127s	user 0.013s	sys 0.033s
W20260709 12:16:37.348753 28332 leader_election.cc:343] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:37.348258 28363 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:37.350492 28437 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:37.320858 28555 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:37.356045 28743 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:37.291882 28482 raft_consensus.cc:385] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.333720 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:37.321635 28709 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.335995 28737 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.336899 28481 tablet_bootstrap.cc:492] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:37.358933 28364 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.359426 28481 ts_tablet_manager.cc:1403] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.099s	user 0.021s	sys 0.017s
I20260709 12:16:37.337935 28693 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:37.352761 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:37.361533 28737 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.362708 28693 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:37.362257 28737 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.364008 28693 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:37.364136 28481 raft_consensus.cc:359] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.364841 28481 raft_consensus.cc:385] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.365113 28481 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.366138 28481 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.366837 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.368005 28737 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.369216 28693 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.370049 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:37.322937 28504 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.373319 28504 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.374207 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:37.375015 28367 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:37.353021 28484 raft_consensus.cc:359] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.376428 28484 raft_consensus.cc:385] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.376741 28484 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.347199 28589 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.376955 28482 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.377388 28484 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.377696 28482 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.378597 28363 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:37.354277 28608 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.379766 28484 ts_tablet_manager.cc:1434] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 12:16:37.379791 28608 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.381228 28484 tablet_bootstrap.cc:492] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:37.382297 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:37.383184 28504 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:37.383553 28504 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.383863 28504 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.385505 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.386300 28363 raft_consensus.cc:2393] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:37.388567 28608 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:37.368165 28481 ts_tablet_manager.cc:1434] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:16:37.392877 28287 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:37.393575 28481 tablet_bootstrap.cc:492] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:37.380806 28482 ts_tablet_manager.cc:1434] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.112s	user 0.009s	sys 0.000s
I20260709 12:16:37.396059 28504 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.396440 28482 tablet_bootstrap.cc:492] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:37.396422 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:37.399749 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:37.400321 28363 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.405211 28673 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.405931 28673 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.407680 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:37.408776 28504 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.396920 28484 tablet_bootstrap.cc:654] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.409395 28363 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:37.410743 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:37.411604 28504 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:37.412065 28504 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:37.411936 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:37.412855 28504 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.373519 28364 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:37.415086 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:37.419734 28517 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.422400 28517 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.420604 28793 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:37.425899 28517 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:37.424562 28441 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.436379 28484 tablet_bootstrap.cc:492] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:37.436960 28484 ts_tablet_manager.cc:1403] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.000s
I20260709 12:16:37.422864 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:37.443250 28589 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:37.445547 28481 tablet_bootstrap.cc:654] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.444413 28484 raft_consensus.cc:359] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.449918 28484 raft_consensus.cc:385] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.450235 28484 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.450877 28484 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.461517 28791 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.461951 28484 ts_tablet_manager.cc:1434] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:16:37.464486 28484 tablet_bootstrap.cc:492] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:37.463760 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:37.462098 28791 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.429435 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:37.476135 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:37.504348 28441 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:37.464059 28793 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.464759 28673 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:37.464579 28438 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:37.490669 28791 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.504426 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:37.505003 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:37.510211 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:37.513401 28287 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:37.523360 28189 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:37.521095 28736 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:16:37.525753 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 61593.1us
I20260709 12:16:37.526319 28601 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:37.535568 28601 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:37.523118 28608 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.536561 28601 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.530347 28435 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:37.577842 28608 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.586808 28257 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:37.609202 28484 tablet_bootstrap.cc:654] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.522806 28793 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.591482 28736 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:37.592118 28608 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:37.598310 28439 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.622992 28736 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.637115 28189 catalog_manager.cc:5697] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:37.644583 28736 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.718178 28791 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.718904 28481 tablet_bootstrap.cc:492] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:37.718812 28791 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.719364 28481 ts_tablet_manager.cc:1403] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.326s	user 0.021s	sys 0.013s
I20260709 12:16:37.696533 28709 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:16:37.686894 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:37.734016 28791 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.736181 28482 tablet_bootstrap.cc:654] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:37.742092 28798 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 12:16:37.742653 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 320600us
I20260709 12:16:37.788452 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:37.735419 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.789189 28481 raft_consensus.cc:359] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.791278 28481 raft_consensus.cc:385] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.791584 28481 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.785898 28793 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.797719 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:37.788887 28709 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:37.797910 28737 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:37.793982 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 107198us
I20260709 12:16:37.799592 28481 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.800982 28799 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:37.802853 28484 tablet_bootstrap.cc:492] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:37.804571 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.835791 28484 ts_tablet_manager.cc:1403] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.372s	user 0.026s	sys 0.032s
I20260709 12:16:37.795389 28364 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:37.806298 28709 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.809263 28292 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:37.837487 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.843135 28709 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.844576 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:37.838469 28799 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.874343 28285 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:16:37.873343 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 75831.7us
I20260709 12:16:37.869431 28366 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:37.895646 28481 ts_tablet_manager.cc:1434] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.176s	user 0.013s	sys 0.010s
I20260709 12:16:37.896924 28481 tablet_bootstrap.cc:492] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:37.908914 28799 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:37.920344 28484 raft_consensus.cc:359] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.921187 28484 raft_consensus.cc:385] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:37.921465 28484 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:37.921856 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:37.922427 28484 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.928017 28484 ts_tablet_manager.cc:1434] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.092s	user 0.008s	sys 0.000s
I20260709 12:16:37.933147 28673 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:37.935844 28441 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.944247 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:37.944962 28440 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
W20260709 12:16:37.945513 28404 leader_election.cc:343] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:37.948125 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:37.947054 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:37.932492 28736 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:37.954411 28736 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:37.962299 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:37.963028 28367 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.969019 28482 tablet_bootstrap.cc:492] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:37.969705 28482 ts_tablet_manager.cc:1403] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.573s	user 0.025s	sys 0.024s
I20260709 12:16:37.973371 28813 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:37.973995 28813 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:37.974380 28813 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:37.975291 28481 tablet_bootstrap.cc:654] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:37.993533 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:38.013042 28404 leader_election.cc:343] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:38.016966 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:38.036634 28808 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:38.037173 28808 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:38.037523 28808 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:38.038486 28504 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:38.040228 28255 consensus_peers.cc:597] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.042364 28404 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:16:38.049309 28255 leader_election.cc:343] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:38.057778 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:38.062217 28482 raft_consensus.cc:359] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.063140 28482 raft_consensus.cc:385] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:38.063423 28482 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:38.085958 28793 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:38.087675 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:38.085809 28482 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.104678 28287 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:38.107254 28481 tablet_bootstrap.cc:492] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:38.107796 28481 ts_tablet_manager.cc:1403] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.211s	user 0.021s	sys 0.006s
I20260709 12:16:38.108596 28482 ts_tablet_manager.cc:1434] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.137s	user 0.013s	sys 0.003s
I20260709 12:16:38.109612 28482 tablet_bootstrap.cc:492] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:38.100325 28190 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:38.124907 28814 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:38.126950 28238 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:38.131475 28481 raft_consensus.cc:359] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.132272 28481 raft_consensus.cc:385] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:38.132556 28481 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:38.132609 28367 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:38.075260 28813 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.136438 28481 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.144342 28808 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.151535 28484 tablet_bootstrap.cc:492] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:38.146515 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:38.154480 28504 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:38.154958 28504 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:38.134994 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:38.155725 28504 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.157961 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:38.163594 28404 leader_election.cc:343] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:38.164116 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:38.173026 28815 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:38.176782 28481 ts_tablet_manager.cc:1434] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.002s
I20260709 12:16:38.182377 28517 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:38.169001 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:38.177582 28437 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:38.182997 28517 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.163026 28736 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:16:38.207448 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:38.208105 28292 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:38.209374 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:38.210421 28517 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:38.211158 28517 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:38.211503 28517 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:38.211805 28517 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:38.184183 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 55724.4us
I20260709 12:16:38.214869 28808 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:38.216710 28433 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:38.190151 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:38.221578 28517 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.224671 28440 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:38.226459 28290 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:38.210063 28813 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:38.231890 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:38.233829 28332 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:38.234037 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:38.235419 28482 tablet_bootstrap.cc:654] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:38.236701 28517 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:38.203599 28709 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.242455 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:38.243113 28289 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:38.244488 28709 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:38.218536 28644 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:16:38.248684 28825 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:16:38.258092 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:38.246690 28791 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:16:38.273720 28481 tablet_bootstrap.cc:492] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:38.266017 28812 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:16:38.279279 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:38.279266 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:38.283089 28435 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:38.299500 28809 raft_consensus.cc:2764] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:16:38.300864 28484 tablet_bootstrap.cc:654] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:38.302563 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:38.307435 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:38.309975 28365 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:38.311743 28481 tablet_bootstrap.cc:654] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:38.303447 28366 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:16:38.299597 28823 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:16:38.315557 28290 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
W20260709 12:16:38.322144 28404 leader_election.cc:343] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:38.324880 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:38.326049 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:38.306440 28809 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:38.323153 28601 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:38.333889 28289 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:38.360919 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:38.363986 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:38.329425 28709 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:38.372725 28808 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:38.345328 28608 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:38.362226 28644 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:38.362200 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:38.339079 28673 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
W20260709 12:16:38.384704 28830 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:16:38.374516 28693 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:38.348333 28809 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.389549 28693 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:38.373723 28589 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:16:38.387504 28808 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:38.391805 28693 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.405539 28364 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 1: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:16:38.405700 28831 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.052s
I20260709 12:16:38.388370 28644 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.429581 28482 tablet_bootstrap.cc:492] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:38.457449 28482 ts_tablet_manager.cc:1403] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.348s	user 0.016s	sys 0.016s
I20260709 12:16:38.454259 28808 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.427796 28737 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:38.476524 28644 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:16:38.477244 28826 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:16:38.485246 28481 tablet_bootstrap.cc:492] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:38.485936 28481 ts_tablet_manager.cc:1403] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.212s	user 0.026s	sys 0.006s
I20260709 12:16:38.436450 28360 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:38.482369 28482 raft_consensus.cc:359] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.489117 28482 raft_consensus.cc:385] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:38.489502 28482 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:38.504411 28482 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.490401 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:38.487922 28737 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.502501 28809 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:38.516469 28481 raft_consensus.cc:359] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.517262 28481 raft_consensus.cc:385] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:38.517534 28481 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:38.518313 28481 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.499428 28831 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:16:38.519958 28481 ts_tablet_manager.cc:1434] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 12:16:38.514719 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:38.532575 28288 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:38.540616 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:38.540230 28482 ts_tablet_manager.cc:1434] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.000s
I20260709 12:16:38.542256 28482 tablet_bootstrap.cc:492] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:38.540180 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:38.566516 28840 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:38.567193 28840 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:38.567580 28840 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:38.594877 28404 leader_election.cc:343] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:38.601986 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:38.589032 28737 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:38.693431 28482 tablet_bootstrap.cc:654] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:38.713078 28481 tablet_bootstrap.cc:492] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:38.713879 28817 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:38.714429 28817 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.757537 28484 tablet_bootstrap.cc:492] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:38.758252 28484 ts_tablet_manager.cc:1403] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.607s	user 0.066s	sys 0.043s
I20260709 12:16:38.798336 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:16:38.840095 28845 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:16:38.769274 28840 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:38.883991 28439 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:38.842621 28187 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 92) took 1119 ms. Trace:
I20260709 12:16:38.910521 28840 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:38.886968 28187 rpcz_store.cc:276] 0709 12:16:37.729679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:37.730452 (+   773us) service_pool.cc:224] Handling call
0709 12:16:37.921664 (+191212us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f1090 00007f85be88ff3a 00007f85c4645e9c 00007f85c474404e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc 00007f85c2c04392 00007f85bcbc248c 00007f85bcbc18c6
0709 12:16:38.842547 (+920883us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":264408,"reported_tablets":3,"spinlock_wait_cycles":43230080}
I20260709 12:16:39.130575 28817 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:39.133387 28484 raft_consensus.cc:359] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.134223 28484 raft_consensus.cc:385] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.134513 28484 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.142423 28484 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.146339 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:39.158747 28284 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.167400 28189 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:39.171630 28189 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:39.178735 28484 ts_tablet_manager.cc:1434] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.420s	user 0.016s	sys 0.005s
I20260709 12:16:39.182389 28481 tablet_bootstrap.cc:654] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.183529 28484 tablet_bootstrap.cc:492] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
W20260709 12:16:39.184820 28844 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.169s	user 0.003s	sys 0.016s
I20260709 12:16:39.210119 28802 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:39.231002 28238 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:39.240077 28802 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:39.251232 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 102812us
I20260709 12:16:39.252048 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:39.260056 28482 tablet_bootstrap.cc:492] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:39.260658 28482 ts_tablet_manager.cc:1403] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.719s	user 0.043s	sys 0.047s
I20260709 12:16:39.272253 28804 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:39.275017 28802 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:39.275687 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:39.277956 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:16:39.280450 28404 leader_election.cc:343] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.284749 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:39.290383 28255 leader_election.cc:343] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.291474 28484 tablet_bootstrap.cc:654] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.286813 28284 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:39.309440 28481 tablet_bootstrap.cc:492] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:39.310014 28481 ts_tablet_manager.cc:1403] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.598s	user 0.020s	sys 0.026s
I20260709 12:16:39.311287 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:16:39.314031 28504 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:39.315382 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54984us
I20260709 12:16:39.317927 28608 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:39.319967 28482 raft_consensus.cc:359] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.320750 28482 raft_consensus.cc:385] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.321069 28482 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.326395 28482 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.327943 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:39.319322 28812 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:39.328707 28435 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:39.340031 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:39.340824 28284 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:39.340966 28482 ts_tablet_manager.cc:1434] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.080s	user 0.013s	sys 0.000s
I20260709 12:16:39.342183 28482 tablet_bootstrap.cc:492] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:39.345002 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:39.346401 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:39.349512 28693 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:39.350179 28693 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:39.351135 28693 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.341923 28850 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:16:39.361820 28837 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:39.362365 28837 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:39.362756 28837 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.370292 28481 raft_consensus.cc:359] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.371351 28481 raft_consensus.cc:385] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.371637 28481 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.378808 28484 tablet_bootstrap.cc:492] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:39.379490 28484 ts_tablet_manager.cc:1403] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.200s	user 0.018s	sys 0.013s
I20260709 12:16:39.390945 28481 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.391839 28812 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:39.393767 28481 ts_tablet_manager.cc:1434] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.083s	user 0.005s	sys 0.002s
I20260709 12:16:39.395828 28481 tablet_bootstrap.cc:492] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
W20260709 12:16:39.412484 28404 leader_election.cc:343] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:39.416710 28404 consensus_peers.cc:597] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:39.419111 28484 raft_consensus.cc:359] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.419756 28484 raft_consensus.cc:385] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.419963 28484 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.422230 28852 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:39.440456 28484 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.446976 28482 tablet_bootstrap.cc:654] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:39.471284 28858 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.180s	user 0.005s	sys 0.000s
I20260709 12:16:39.484688 28484 ts_tablet_manager.cc:1434] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.105s	user 0.004s	sys 0.004s
I20260709 12:16:39.474130 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:39.486829 28292 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 0.
I20260709 12:16:39.521806 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:39.514412 28484 tablet_bootstrap.cc:492] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:39.533860 28852 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:39.535506 28852 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:39.539392 28852 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.523447 28810 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:39.554291 28481 tablet_bootstrap.cc:654] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.556417 28837 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.551108 28810 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.572229 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:39.572927 28441 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.572044 28850 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:39.573757 28837 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:39.581271 28441 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:39.573669 28850 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.582998 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:39.573637 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:39.585350 28286 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.583865 28709 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:39.589228 28709 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:39.598366 28709 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.607863 28286 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:39.608515 28852 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.620936 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:39.621824 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:39.615149 28850 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:39.623092 28482 tablet_bootstrap.cc:492] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
W20260709 12:16:39.623265 28404 leader_election.cc:343] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.626133 28482 ts_tablet_manager.cc:1403] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.284s	user 0.012s	sys 0.015s
W20260709 12:16:39.626807 28404 leader_election.cc:343] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.627250 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:39.627605 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:39.628203 28850 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:39.629702 28482 raft_consensus.cc:359] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.634394 28482 raft_consensus.cc:385] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.634725 28482 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.637144 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:39.639910 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:39.640765 28837 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:39.642644 28484 tablet_bootstrap.cc:654] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.630930 28810 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:39.646127 28482 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.649325 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:39.650168 28292 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.651597 28852 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:39.657073 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:16:39.659433 28404 leader_election.cc:343] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.662907 28482 ts_tablet_manager.cc:1434] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.001s
I20260709 12:16:39.664125 28482 tablet_bootstrap.cc:492] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:39.671906 28187 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:39.673677 28187 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:39.678442 28189 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 98) took 1101 ms. Trace:
I20260709 12:16:39.689911 28292 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:16:39.696795 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:39.678828 28189 rpcz_store.cc:276] 0709 12:16:38.586606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:38.586760 (+   154us) service_pool.cc:224] Handling call
0709 12:16:39.678375 (+1091615us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:39.260791 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:39.290867 (+ 30076us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:39.290902 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:16:39.290922 (+    20us) tablet.cc:662] Decoding operations
0709 12:16:39.291341 (+   419us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:39.291402 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:39.291431 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:16:39.315716 (+ 24285us) tablet.cc:701] Row locks acquired
0709 12:16:39.315734 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:16:39.315903 (+   169us) write_op.cc:270] Start()
0709 12:16:39.316016 (+   113us) write_op.cc:276] Timestamp: P: 1783599399315852 usec, L: 0
0709 12:16:39.316039 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:16:39.316715 (+   676us) log.cc:844] Serialized 922 byte log entry
0709 12:16:39.565514 (+248799us) op_driver.cc:464] REPLICATION: finished
0709 12:16:39.638669 (+ 73155us) write_op.cc:301] APPLY: starting
0709 12:16:39.638749 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:39.638870 (+   121us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:39.644982 (+  6112us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:39.645336 (+   354us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:39.645526 (+   190us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:16:39.645576 (+    50us) write_op.cc:312] APPLY: finished
0709 12:16:39.655492 (+  9916us) log.cc:844] Serialized 28 byte log entry
0709 12:16:39.658690 (+  3198us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:39.660233 (+  1543us) write_op.cc:454] Released schema lock
0709 12:16:39.661263 (+  1030us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:39.661878 (+   615us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:39.661905 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":254275,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":72936,"apply.run_cpu_time_us":2285,"apply.run_wall_time_us":23191,"num_ops":2,"prepare.queue_time_us":57603,"prepare.run_cpu_time_us":26863,"prepare.run_wall_time_us":143766,"raft.queue_time_us":114531,"raft.run_cpu_time_us":6663,"raft.run_wall_time_us":33893,"replication_time_us":249328,"thread_start_us":196212,"threads_started":4,"wal-append.queue_time_us":130728}]]}
I20260709 12:16:39.699169 28481 tablet_bootstrap.cc:492] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:39.699736 28481 ts_tablet_manager.cc:1403] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.304s	user 0.015s	sys 0.004s
I20260709 12:16:39.701071 28850 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:39.701684 28850 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:16:39.702594 28850 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.708370 28481 raft_consensus.cc:359] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.709252 28481 raft_consensus.cc:385] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.709539 28481 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.712769 28481 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.715690 28481 ts_tablet_manager.cc:1434] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:16:39.716809 28481 tablet_bootstrap.cc:492] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:39.724666 28484 tablet_bootstrap.cc:492] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:39.725337 28484 ts_tablet_manager.cc:1403] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.211s	user 0.015s	sys 0.005s
I20260709 12:16:39.730074 28484 raft_consensus.cc:359] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.730860 28484 raft_consensus.cc:385] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.731151 28484 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.731912 28484 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.734517 28484 ts_tablet_manager.cc:1434] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:16:39.745605 28484 tablet_bootstrap.cc:492] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:39.756314 28812 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:39.756935 28812 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.760640 28812 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:39.763271 28482 tablet_bootstrap.cc:654] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.764053 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:39.764695 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:39.766450 28439 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
W20260709 12:16:39.766516 28255 leader_election.cc:343] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.770081 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:39.774423 28812 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:39.783149 28812 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:39.795755 28812 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.803308 28358 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.815148 28484 tablet_bootstrap.cc:654] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.807634 28187 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:39.823881 28812 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.826362 28812 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:39.886911 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:39.889468 28481 tablet_bootstrap.cc:654] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.890882 28555 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:39.894191 28255 leader_election.cc:343] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.897047 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:39.901193 28441 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.905095 28484 tablet_bootstrap.cc:492] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:39.896092 28555 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.908494 28484 ts_tablet_manager.cc:1403] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.163s	user 0.012s	sys 0.004s
I20260709 12:16:39.920516 28484 raft_consensus.cc:359] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.921445 28484 raft_consensus.cc:385] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.921811 28484 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.932869 28484 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.941998 28484 ts_tablet_manager.cc:1434] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.003s
I20260709 12:16:39.944072 28484 tablet_bootstrap.cc:492] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:39.950423 28360 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:39.950677 28482 tablet_bootstrap.cc:492] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:39.951414 28482 ts_tablet_manager.cc:1403] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.288s	user 0.012s	sys 0.004s
I20260709 12:16:39.948333 28189 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:39.958622 28791 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:39.959304 28791 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.963696 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:39.964742 28791 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:39.964978 28292 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 1: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:16:39.966660 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:39.968147 28482 raft_consensus.cc:359] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:39.969048 28482 raft_consensus.cc:385] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:39.969380 28482 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.975842 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:39.977335 28189 catalog_manager.cc:5697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:39.984480 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:39.984856 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:39.985793 28439 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:39.974079 28555 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:39.988198 28555 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:39.971637 28255 leader_election.cc:343] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:39.995015 28813 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:40.000936 28482 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.002941 28482 ts_tablet_manager.cc:1434] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.051s	user 0.001s	sys 0.007s
I20260709 12:16:40.004408 28482 tablet_bootstrap.cc:492] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:40.012015 28481 tablet_bootstrap.cc:492] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:40.012766 28481 ts_tablet_manager.cc:1403] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.296s	user 0.007s	sys 0.010s
I20260709 12:16:40.013302 28804 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:40.016571 28481 raft_consensus.cc:359] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.017455 28481 raft_consensus.cc:385] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.023104 28484 tablet_bootstrap.cc:654] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:39.997550 28441 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:40.033809 28481 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.036190 28481 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.038242 28608 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:40.039613 28481 ts_tablet_manager.cc:1434] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.007s
I20260709 12:16:40.041124 28481 tablet_bootstrap.cc:492] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:40.090950 28481 tablet_bootstrap.cc:654] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.092779 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:40.094820 28791 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.095332 28791 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.095754 28791 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.104146 28482 tablet_bootstrap.cc:654] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.100339 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:40.108768 28865 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.113924 28865 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:40.115016 28791 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.115046 28865 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.117424 28791 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:40.132830 28709 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:16:40.158788 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:40.159370 28441 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.152865 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:40.172034 28481 tablet_bootstrap.cc:492] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:40.172652 28481 ts_tablet_manager.cc:1403] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.132s	user 0.016s	sys 0.014s
I20260709 12:16:40.169566 28360 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.183873 28481 raft_consensus.cc:359] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.186333 28481 raft_consensus.cc:385] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.186573 28481 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.215750 28484 tablet_bootstrap.cc:492] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:40.216270 28484 ts_tablet_manager.cc:1403] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.273s	user 0.019s	sys 0.011s
I20260709 12:16:40.217846 28481 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.220094 28441 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:40.223409 28484 raft_consensus.cc:359] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.224153 28484 raft_consensus.cc:385] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.224452 28484 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.229929 28481 ts_tablet_manager.cc:1434] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.002s
I20260709 12:16:40.241014 28484 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.244092 28190 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:40.252183 28484 ts_tablet_manager.cc:1434] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.001s
I20260709 12:16:40.266853 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:40.234359 28360 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:40.270262 28865 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.272472 28481 tablet_bootstrap.cc:492] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:40.283236 28484 tablet_bootstrap.cc:492] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:40.276335 28865 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:40.301066 28865 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:40.302927 28404 consensus_peers.cc:597] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:40.306350 28482 tablet_bootstrap.cc:492] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:40.306907 28482 ts_tablet_manager.cc:1403] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.303s	user 0.042s	sys 0.032s
I20260709 12:16:40.311172 28813 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.311743 28813 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.322685 28813 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:40.323552 28482 raft_consensus.cc:359] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.324452 28482 raft_consensus.cc:385] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.324743 28482 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.325489 28482 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:40.326575 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49645.3us
I20260709 12:16:40.336364 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:40.339519 28484 tablet_bootstrap.cc:654] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.329280 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:40.383294 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 54019.1us
I20260709 12:16:40.386282 28482 ts_tablet_manager.cc:1434] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.000s
I20260709 12:16:40.388783 28897 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.389353 28897 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.391695 28897 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:16:40.387940 28404 leader_election.cc:343] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:40.392390 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 62643us
I20260709 12:16:40.393316 28481 tablet_bootstrap.cc:654] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.394430 28482 tablet_bootstrap.cc:492] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
W20260709 12:16:40.406836 28404 leader_election.cc:343] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.414183 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:40.419886 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:40.420691 28291 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:40.423529 28330 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:40.437354 28896 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.440780 28484 tablet_bootstrap.cc:492] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:40.441380 28484 ts_tablet_manager.cc:1403] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.169s	user 0.009s	sys 0.010s
I20260709 12:16:40.441987 28808 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:40.437873 28896 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.447078 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:40.447623 28517 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:16:40.447796 28437 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 0.
I20260709 12:16:40.445597 28896 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:40.471141 28238 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:40.474632 28896 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.475876 28482 tablet_bootstrap.cc:654] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.479257 28896 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:40.480122 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:40.480620 28435 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.487461 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:16:40.488158 28289 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.496948 28484 raft_consensus.cc:359] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.497707 28484 raft_consensus.cc:385] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.498018 28484 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.498672 28484 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.501039 28484 ts_tablet_manager.cc:1434] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.059s	user 0.002s	sys 0.003s
I20260709 12:16:40.495568 28435 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:40.532024 28481 tablet_bootstrap.cc:492] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:40.532614 28481 ts_tablet_manager.cc:1403] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.269s	user 0.008s	sys 0.016s
I20260709 12:16:40.535753 28481 raft_consensus.cc:359] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.536564 28481 raft_consensus.cc:385] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.536847 28481 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.537559 28481 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.531916 28289 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:16:40.540649 28481 ts_tablet_manager.cc:1434] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:16:40.544847 28893 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.554262 28893 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.549700 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:16:40.556670 28484 tablet_bootstrap.cc:492] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:40.557161 28896 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.562507 28896 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:16:40.563416 28896 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.565972 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:40.567714 28255 leader_election.cc:343] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.575695 28481 tablet_bootstrap.cc:492] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:40.576586 28893 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:40.577849 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:40.578531 28437 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:40.591157 28482 tablet_bootstrap.cc:492] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:40.591720 28482 ts_tablet_manager.cc:1403] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.198s	user 0.019s	sys 0.034s
I20260709 12:16:40.594781 28190 catalog_manager.cc:5697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:40.589912 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:40.596689 28893 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.600124 28893 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.600554 28893 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.603994 28482 raft_consensus.cc:359] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.604987 28482 raft_consensus.cc:385] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.605273 28482 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.606009 28482 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.602695 28791 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:16:40.623005 28187 catalog_manager.cc:5697] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:40.625633 28255 consensus_peers.cc:597] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:40.628551 28482 ts_tablet_manager.cc:1434] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.002s
I20260709 12:16:40.643788 28482 tablet_bootstrap.cc:492] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:40.657444 28481 tablet_bootstrap.cc:654] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.661664 28484 tablet_bootstrap.cc:654] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.666453 28482 tablet_bootstrap.cc:654] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.670622 28893 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.692417 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:40.693063 28363 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.700040 28893 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:40.717621 28363 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:40.718839 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:40.721665 28893 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.722199 28893 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:40.722250 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:40.722997 28437 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.723111 28893 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.745144 28855 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.741529 28437 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:40.749339 28855 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.751763 28855 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:40.752652 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:40.753964 28404 leader_election.cc:343] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.756790 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:40.759155 28481 tablet_bootstrap.cc:492] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
W20260709 12:16:40.759272 28404 leader_election.cc:343] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.759811 28481 ts_tablet_manager.cc:1403] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.198s	user 0.050s	sys 0.036s
I20260709 12:16:40.759685 28404 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:40.760546 28850 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:40.763305 28481 raft_consensus.cc:359] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.764029 28481 raft_consensus.cc:385] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.764343 28481 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.764704 28484 tablet_bootstrap.cc:492] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:40.765152 28484 ts_tablet_manager.cc:1403] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.237s	user 0.011s	sys 0.000s
I20260709 12:16:40.765173 28481 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.767397 28481 ts_tablet_manager.cc:1434] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:40.768960 28481 tablet_bootstrap.cc:492] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:40.774968 28481 tablet_bootstrap.cc:654] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.774937 28484 raft_consensus.cc:359] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.778309 28484 raft_consensus.cc:385] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.778656 28484 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.779338 28482 tablet_bootstrap.cc:492] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:40.779817 28482 ts_tablet_manager.cc:1403] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.136s	user 0.012s	sys 0.000s
I20260709 12:16:40.781250 28481 tablet_bootstrap.cc:492] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:40.781796 28481 ts_tablet_manager.cc:1403] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 12:16:40.782580 28482 raft_consensus.cc:359] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.783252 28482 raft_consensus.cc:385] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.779353 28484 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.783524 28482 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.784353 28482 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.786501 28484 ts_tablet_manager.cc:1434] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:16:40.788190 28190 catalog_manager.cc:5697] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:40.789024 28484 tablet_bootstrap.cc:492] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:40.786692 28482 ts_tablet_manager.cc:1434] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:16:40.791597 28482 tablet_bootstrap.cc:492] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:40.792064 28850 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.792577 28850 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.787634 28481 raft_consensus.cc:359] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.796403 28481 raft_consensus.cc:385] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.796478 28484 tablet_bootstrap.cc:654] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.796754 28481 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.797505 28481 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.797928 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:40.797928 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:16:40.799670 28404 leader_election.cc:343] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.799996 28481 ts_tablet_manager.cc:1434] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
W20260709 12:16:40.800331 28404 leader_election.cc:343] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.800769 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:40.801157 28481 tablet_bootstrap.cc:492] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Bootstrap starting.
I20260709 12:16:40.794714 28850 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:40.802085 28850 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:40.808670 28482 tablet_bootstrap.cc:654] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.811589 28484 tablet_bootstrap.cc:492] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:40.812186 28484 ts_tablet_manager.cc:1403] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.002s
I20260709 12:16:40.815232 28484 raft_consensus.cc:359] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.815948 28484 raft_consensus.cc:385] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.816243 28484 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.816867 28484 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.819407 28484 ts_tablet_manager.cc:1434] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 12:16:40.820693 28481 tablet_bootstrap.cc:654] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.822925 28484 tablet_bootstrap.cc:492] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:40.826511 28482 tablet_bootstrap.cc:492] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:40.827031 28482 ts_tablet_manager.cc:1403] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.004s
I20260709 12:16:40.829969 28484 tablet_bootstrap.cc:654] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.829949 28482 raft_consensus.cc:359] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.830770 28482 raft_consensus.cc:385] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.831069 28482 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.831748 28482 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.834079 28482 ts_tablet_manager.cc:1434] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:16:40.835146 28482 tablet_bootstrap.cc:492] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:40.845595 28791 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:40.862859 28363 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.869452 28893 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:40.871523 28482 tablet_bootstrap.cc:654] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.885259 28482 tablet_bootstrap.cc:492] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:40.885815 28482 ts_tablet_manager.cc:1403] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.001s
I20260709 12:16:40.888700 28482 raft_consensus.cc:359] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.889406 28482 raft_consensus.cc:385] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.889669 28482 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.896852 28482 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.899523 28482 ts_tablet_manager.cc:1434] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:16:40.900602 28482 tablet_bootstrap.cc:492] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:40.903855 28484 tablet_bootstrap.cc:492] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:40.904474 28484 ts_tablet_manager.cc:1403] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.084s	user 0.006s	sys 0.011s
I20260709 12:16:40.907538 28482 tablet_bootstrap.cc:654] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.907541 28484 raft_consensus.cc:359] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.908685 28484 raft_consensus.cc:385] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.908978 28484 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.909891 28484 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.920220 28484 ts_tablet_manager.cc:1434] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:16:40.921378 28484 tablet_bootstrap.cc:492] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: Bootstrap starting.
I20260709 12:16:40.925454 28865 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.926079 28865 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.927460 28484 tablet_bootstrap.cc:654] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:40.928545 28865 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:40.929700 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:40.930521 28439 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:40.931654 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:40.933322 28257 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:16:40.934934 28255 leader_election.cc:343] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.939150 28865 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.939695 28865 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.940267 28865 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.948324 28484 tablet_bootstrap.cc:492] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: No bootstrap required, opened a new log
I20260709 12:16:40.948940 28484 ts_tablet_manager.cc:1403] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.004s
I20260709 12:16:40.951850 28812 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:40.952167 28484 raft_consensus.cc:359] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.952345 28812 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.953040 28484 raft_consensus.cc:385] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.953387 28484 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.954046 28482 tablet_bootstrap.cc:492] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:40.954598 28482 ts_tablet_manager.cc:1403] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.004s
I20260709 12:16:40.955305 28812 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:40.956350 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:40.957141 28439 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:40.958412 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:40.959025 28812 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:40.958868 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:40.962199 28812 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:40.962634 28812 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:40.965010 28255 leader_election.cc:343] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.954193 28484 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.961810 28482 raft_consensus.cc:359] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.966634 28482 raft_consensus.cc:385] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:40.966951 28482 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:40.968478 28482 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.971057 28482 ts_tablet_manager.cc:1434] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:16:40.971558 28812 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.972330 28482 tablet_bootstrap.cc:492] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:40.974464 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:16:40.975891 28255 leader_election.cc:343] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:40.977193 28812 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:40.978994 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:40.979749 28439 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:40.988435 28439 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:40.989943 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:16:40.990742 28893 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:40.991022 28484 ts_tablet_manager.cc:1434] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.001s
I20260709 12:16:40.991240 28893 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:40.992187 28893 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:40.993423 28482 tablet_bootstrap.cc:654] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.008864 28482 tablet_bootstrap.cc:492] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:41.009459 28482 ts_tablet_manager.cc:1403] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.037s	user 0.006s	sys 0.011s
I20260709 12:16:41.012727 28482 raft_consensus.cc:359] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.013485 28482 raft_consensus.cc:385] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.013995 28482 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.014724 28482 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.016804 28482 ts_tablet_manager.cc:1434] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:41.018896 28482 tablet_bootstrap.cc:492] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284: Bootstrap starting.
I20260709 12:16:41.026482 28482 tablet_bootstrap.cc:654] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:41.028690 28896 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:41.041342 28482 tablet_bootstrap.cc:492] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284: No bootstrap required, opened a new log
I20260709 12:16:41.041968 28482 ts_tablet_manager.cc:1403] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.001s
I20260709 12:16:41.045069 28482 raft_consensus.cc:359] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.045883 28482 raft_consensus.cc:385] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.046181 28482 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.046931 28482 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.049522 28482 ts_tablet_manager.cc:1434] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:16:41.054394 28736 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.054713 28709 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.054998 28736 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.057427 28736 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:41.058535 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:41.061228 28367 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:41.059026 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:41.062357 28437 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:41.062754 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:41.063750 28736 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.064328 28736 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.064711 28736 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.065053 28367 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.072598 28736 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.076206 28736 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:41.076476 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:41.077328 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:41.078109 28360 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.078199 28437 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.086881 28437 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:41.089489 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:41.090690 28791 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.091240 28791 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:41.092103 28791 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.086392 28360 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:41.108220 28865 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:16:41.113818 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 74831.1us
I20260709 12:16:41.113078 28481 tablet_bootstrap.cc:492] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: No bootstrap required, opened a new log
I20260709 12:16:41.114841 28481 ts_tablet_manager.cc:1403] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Time spent bootstrapping tablet: real 0.314s	user 0.091s	sys 0.134s
I20260709 12:16:41.119165 28481 raft_consensus.cc:359] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.119978 28481 raft_consensus.cc:385] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:41.120275 28481 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Initialized, Role: FOLLOWER
I20260709 12:16:41.121140 28481 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.119246 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:41.122198 28365 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.123891 28481 ts_tablet_manager.cc:1434] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:41.134725 28865 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:41.140791 28365 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:41.130885 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:41.143326 28437 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.153481 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:41.154517 28893 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.155992 28437 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:41.155104 28893 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:41.159811 28893 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.185004 28187 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.186530 28187 catalog_manager.cc:5697] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.187975 28187 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.231530 28900 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.232041 28900 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.234208 28900 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:41.234408 28893 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.234827 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:41.235212 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:41.235993 28439 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 1: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:16:41.237767 28332 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:41.240115 28900 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.249439 28791 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:41.261165 28900 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:41.283020 28791 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:41.284945 28791 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.287082 28791 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:41.288056 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:41.288689 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:16:41.288893 28360 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:41.289371 28437 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 0.
I20260709 12:16:41.290457 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:41.291244 28791 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:41.291567 28791 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:41.291890 28791 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.298906 28791 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.300927 28791 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:41.301539 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:16:41.302220 28360 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.302326 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:16:41.302927 28437 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:41.311280 28437 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:41.312455 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:16:41.313611 28360 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:16:41.314612 28791 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:41.315178 28791 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:16:41.316001 28791 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:41.326414 28190 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:41.482097 28893 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:41.509217 28812 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.571334 28791 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.583174 28736 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:41.600798 28791 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:41.617012 28865 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.775672 28893 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:41.787606 28865 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:42.936509 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:42.975957 28945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:42.975994 28948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:42.975957 28946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:42.977596 27116 server_base.cc:1061] running on GCE node
I20260709 12:16:42.979132 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:42.979424 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:42.979664 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599402979640 us; error 0 us; skew 500 ppm
I20260709 12:16:42.980322 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:42.984390 27116 webserver.cc:533] Webserver started at http://127.26.123.4:41307/ using document root <none> and password file <none>
I20260709 12:16:42.985033 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:42.985363 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:42.985831 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:42.987341 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-3-root/instance:
uuid: "f526369c0d0743b192be6e205540a059"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-9q7n"
I20260709 12:16:42.993834 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 12:16:42.998144 28953 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:42.999302 27116 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 12:16:42.999660 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-3-root
uuid: "f526369c0d0743b192be6e205540a059"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-9q7n"
I20260709 12:16:42.999958 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:43.024818 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:43.026484 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:43.028501 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:43.031258 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:43.031487 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:43.031731 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:43.031903 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:43.127851 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.4:39985
I20260709 12:16:43.128072 29015 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.4:39985 every 8 connection(s)
I20260709 12:16:43.132153 27116 auto_leader_rebalancer-test.cc:439] add a tserver: f526369c0d0743b192be6e205540a059
I20260709 12:16:43.147856 29016 heartbeater.cc:344] Connected to a master server at 127.26.123.62:36149
I20260709 12:16:43.148406 29016 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:43.149219 29016 heartbeater.cc:507] Master 127.26.123.62:36149 requested a full tablet report, sending...
I20260709 12:16:43.151335 28189 ts_manager.cc:194] Registered new tserver with Master: f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:16:43.153447 28189 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46508
I20260709 12:16:43.712209 28439 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:43.720626 28360 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:43.720712 28286 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:43.722311 28944 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:43.723240 29021 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:43.733915 28404 consensus_peers.cc:597] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d63db98cb72412194c214fa082eee94. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:43.736271 28808 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:43.737895 28367 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:43.739236 28286 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:43.756220 28439 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:43.759575 28189 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:43.775123 28404 consensus_peers.cc:597] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 255e0140f2a64433a6c2dd64fe24e6b3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:43.775837 28367 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:43.777444 28286 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:43.777320 28808 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:43.780108 28808 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:43.792306 28944 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:43.800441 28287 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:43.804452 28367 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:43.819914 28190 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:43.868886 28367 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:43.877535 28439 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:43.878226 28287 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:43.879580 28896 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:43.881690 28896 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:43.905429 28896 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:43.919237 28287 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:43.922724 28437 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:16:43.934394 28330 consensus_peers.cc:597] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab6d72f1fb474adfafdd159090dc63b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:43.936916 28360 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:43.966148 28187 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:16:43.987649 28330 consensus_peers.cc:597] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a388b90c9b140649cada36c135d7080. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:43.988740 28287 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:43.990569 28437 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:43.990901 29045 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:16:43.993170 29039 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:44.012991 29045 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:44.024685 28287 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:44.026204 28439 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:44.043015 28189 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:44.156416 29016 heartbeater.cc:499] Master 127.26.123.62:36149 was elected leader, sending a full tablet report...
I20260709 12:16:44.285935 29048 ts_tablet_manager.cc:933] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:44.289152 29048 tablet_copy_client.cc:323] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:16:44.309362 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 8d63db98cb72412194c214fa082eee94 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:16:44.310058 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 8d63db98cb72412194c214fa082eee94 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-8d63db98cb72412194c214fa082eee94
I20260709 12:16:44.319096 28451 tablet_copy_source_session.cc:215] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:44.324666 29048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d63db98cb72412194c214fa082eee94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.337867 29052 ts_tablet_manager.cc:933] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:44.343044 29052 tablet_copy_client.cc:323] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:16:44.343842 29048 tablet_copy_client.cc:806] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:44.344529 29048 tablet_copy_client.cc:670] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:44.344812 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 255e0140f2a64433a6c2dd64fe24e6b3 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:16:44.345351 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 255e0140f2a64433a6c2dd64fe24e6b3 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-255e0140f2a64433a6c2dd64fe24e6b3
I20260709 12:16:44.351094 29048 tablet_copy_client.cc:538] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:44.352017 28451 tablet_copy_source_session.cc:215] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:44.355499 29052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255e0140f2a64433a6c2dd64fe24e6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.364372 29048 tablet_bootstrap.cc:492] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:44.371927 29052 tablet_copy_client.cc:806] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:44.372712 29052 tablet_copy_client.cc:670] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:44.379560 29052 tablet_copy_client.cc:538] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:44.390985 29052 tablet_bootstrap.cc:492] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:44.409816 29048 tablet_bootstrap.cc:492] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:44.411166 29048 tablet_bootstrap.cc:492] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:44.412052 29048 ts_tablet_manager.cc:1403] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.048s	user 0.026s	sys 0.001s
I20260709 12:16:44.416344 29048 raft_consensus.cc:359] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:44.417349 29048 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:44.418169 29048 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:44.439503 29052 tablet_bootstrap.cc:492] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:44.441998 29055 ts_tablet_manager.cc:933] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:44.446151 29052 tablet_bootstrap.cc:492] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:44.449106 29052 ts_tablet_manager.cc:1403] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.058s	user 0.032s	sys 0.001s
I20260709 12:16:44.450037 29048 ts_tablet_manager.cc:1434] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.024s
I20260709 12:16:44.452328 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-8d63db98cb72412194c214fa082eee94 received from {username='slave'} at 127.0.0.1:53280
I20260709 12:16:44.452881 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-8d63db98cb72412194c214fa082eee94 on tablet 8d63db98cb72412194c214fa082eee94 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:44.455235 29052 raft_consensus.cc:359] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:44.456256 29052 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:44.457472 29055 tablet_copy_client.cc:323] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:16:44.457269 29052 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:44.462432 29052 ts_tablet_manager.cc:1434] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.004s
I20260709 12:16:44.469985 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-255e0140f2a64433a6c2dd64fe24e6b3 received from {username='slave'} at 127.0.0.1:53280
I20260709 12:16:44.470495 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-255e0140f2a64433a6c2dd64fe24e6b3 on tablet 255e0140f2a64433a6c2dd64fe24e6b3 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:44.490748 28377 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 9a388b90c9b140649cada36c135d7080 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:16:44.491418 28377 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 9a388b90c9b140649cada36c135d7080 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-9a388b90c9b140649cada36c135d7080
I20260709 12:16:44.502410 29052 ts_tablet_manager.cc:933] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:44.501801 28377 tablet_copy_source_session.cc:215] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:44.514629 29052 tablet_copy_client.cc:323] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:16:44.547415 28377 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet ab6d72f1fb474adfafdd159090dc63b5 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:16:44.548053 28377 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet ab6d72f1fb474adfafdd159090dc63b5 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-ab6d72f1fb474adfafdd159090dc63b5
I20260709 12:16:44.556687 29055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a388b90c9b140649cada36c135d7080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.557044 28377 tablet_copy_source_session.cc:215] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:44.576470 29052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6d72f1fb474adfafdd159090dc63b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.596889 29055 tablet_copy_client.cc:806] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:44.597517 29055 tablet_copy_client.cc:670] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:44.607187 29052 tablet_copy_client.cc:806] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:44.608402 29052 tablet_copy_client.cc:670] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:44.614842 29055 tablet_copy_client.cc:538] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:44.617256 29052 tablet_copy_client.cc:538] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:44.638154 29052 tablet_bootstrap.cc:492] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:44.640724 29055 tablet_bootstrap.cc:492] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:44.662443 29055 tablet_bootstrap.cc:492] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:44.663064 29052 tablet_bootstrap.cc:492] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:44.663502 29055 tablet_bootstrap.cc:492] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:44.664497 29055 ts_tablet_manager.cc:1403] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.024s	user 0.021s	sys 0.001s
I20260709 12:16:44.668408 29055 raft_consensus.cc:359] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:44.669338 29055 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:44.670101 29055 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:44.672679 29055 ts_tablet_manager.cc:1434] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:44.675434 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-9a388b90c9b140649cada36c135d7080 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:16:44.675895 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-9a388b90c9b140649cada36c135d7080 on tablet 9a388b90c9b140649cada36c135d7080 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:44.680475 29052 tablet_bootstrap.cc:492] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:44.681272 29052 ts_tablet_manager.cc:1403] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.005s
I20260709 12:16:44.684336 29052 raft_consensus.cc:359] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:44.685272 29052 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:44.686177 29052 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:44.689090 29052 ts_tablet_manager.cc:1434] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:44.691886 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-ab6d72f1fb474adfafdd159090dc63b5 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:16:44.692334 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-ab6d72f1fb474adfafdd159090dc63b5 on tablet ab6d72f1fb474adfafdd159090dc63b5 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:44.779984 28991 raft_consensus.cc:1217] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:44.797288 28991 raft_consensus.cc:1217] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:44.871663 28991 raft_consensus.cc:1217] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:45.018584 28991 raft_consensus.cc:1217] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:45.220928 29067 raft_consensus.cc:1064] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:45.223901 29067 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:45.232749 28991 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.233521 28358 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.233726 28287 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.234579 29069 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:45.235561 29067 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:45.236235 29021 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:45.251991 29021 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.254966 28287 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.256186 28358 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.266088 28991 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.273180 28187 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:45.311719 28439 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:45.314524 29061 raft_consensus.cc:1064] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:45.318791 28991 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:45.317194 29061 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:45.321619 29021 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:45.327737 28287 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:45.329433 29067 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:45.333310 29069 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.347002 28289 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.352311 28990 raft_consensus.cc:2955] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.353595 28437 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.355225 29039 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:45.359875 28991 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.368206 29039 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:45.384641 28189 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.406389 28287 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.415192 29039 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:45.425702 29067 raft_consensus.cc:1064] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:45.431005 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 255e0140f2a64433a6c2dd64fe24e6b3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:45.444821 29067 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:45.473500 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet 255e0140f2a64433a6c2dd64fe24e6b3 with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55934
I20260709 12:16:45.481943 29039 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.490712 28437 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.505697 28367 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.505686 28991 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.511201 29021 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:45.513123 29021 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:45.522840 28990 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.552707 28189 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.563381 28289 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.568493 29082 tablet_replica.cc:333] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:16:45.623070 29092 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:45.616104 29082 raft_consensus.cc:2243] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.662078 29082 raft_consensus.cc:2272] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:45.663024 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 56158.4us
I20260709 12:16:45.685973 28358 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:45.686247 29082 ts_tablet_manager.cc:1916] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:45.691670 29096 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.693837 28991 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:16:45.706280 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 88415.4us
I20260709 12:16:45.709800 28286 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.717111 29082 ts_tablet_manager.cc:1929] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:45.717769 29082 log.cc:1199] T 255e0140f2a64433a6c2dd64fe24e6b3 P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/255e0140f2a64433a6c2dd64fe24e6b3
I20260709 12:16:45.719720 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet 255e0140f2a64433a6c2dd64fe24e6b3 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:45.721227 29085 raft_consensus.cc:1064] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:45.721520 28990 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:45.724252 29085 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:45.728937 28286 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:45.739534 28358 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.742405 29088 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:45.744948 29039 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:45.750643 28286 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.756640 28189 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.758324 29039 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.786607 28286 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.787339 28991 raft_consensus.cc:2955] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.804371 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9a388b90c9b140649cada36c135d7080 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:45.815932 28187 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:45.832086 28439 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:45.842281 28360 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:45.842510 28991 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:45.843814 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a388b90c9b140649cada36c135d7080 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33090
I20260709 12:16:45.850904 29067 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:16:45.873668 29100 tablet_replica.cc:333] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:16:45.878290 29069 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:45.879602 29100 raft_consensus.cc:2243] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.876132 28286 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.880398 29100 raft_consensus.cc:2272] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.881456 28437 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.883180 29078 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:16:45.886086 29099 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:16:45.888610 29100 ts_tablet_manager.cc:1916] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:45.905927 28991 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:45.912407 29078 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:45.911600 29100 ts_tablet_manager.cc:1929] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:45.913450 29088 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:45.913888 29100 log.cc:1199] T 9a388b90c9b140649cada36c135d7080 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/9a388b90c9b140649cada36c135d7080
I20260709 12:16:45.920291 29069 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.923552 28990 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.931707 28358 raft_consensus.cc:2955] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.935175 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 9a388b90c9b140649cada36c135d7080 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:45.948210 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8d63db98cb72412194c214fa082eee94 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:45.941505 28188 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.961956 29090 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:45.989636 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d63db98cb72412194c214fa082eee94 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59772
I20260709 12:16:45.999540 29104 tablet_replica.cc:333] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:16:46.000813 29104 raft_consensus.cc:2243] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.001722 29104 raft_consensus.cc:2272] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.996455 28991 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:46.010599 28437 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:46.021764 28287 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:46.029557 29104 ts_tablet_manager.cc:1916] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:46.048918 28188 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:46.069262 28358 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:46.071852 29104 ts_tablet_manager.cc:1929] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:46.072309 29104 log.cc:1199] T 8d63db98cb72412194c214fa082eee94 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/8d63db98cb72412194c214fa082eee94
I20260709 12:16:46.077569 28991 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:46.080008 28287 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:46.081681 29088 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:46.082844 29090 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:46.092542 29088 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:46.095588 28287 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:46.095988 28991 raft_consensus.cc:2955] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:46.100283 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 8d63db98cb72412194c214fa082eee94 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:46.111019 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ab6d72f1fb474adfafdd159090dc63b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:46.126063 28187 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:46.151908 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet ab6d72f1fb474adfafdd159090dc63b5 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33090
I20260709 12:16:46.152933 29100 tablet_replica.cc:333] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:16:46.154086 29100 raft_consensus.cc:2243] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:46.154973 29100 raft_consensus.cc:2272] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:46.159322 29100 ts_tablet_manager.cc:1916] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:46.173110 29100 ts_tablet_manager.cc:1929] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:46.173473 29100 log.cc:1199] T ab6d72f1fb474adfafdd159090dc63b5 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/ab6d72f1fb474adfafdd159090dc63b5
I20260709 12:16:46.175060 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet ab6d72f1fb474adfafdd159090dc63b5 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:47.226922 28360 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:47.234313 28439 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:47.234880 28330 consensus_peers.cc:597] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c59ee067797b40d396f97683aa7eca81. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:47.235733 29039 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:47.236521 28287 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:47.238440 29090 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:47.243762 29039 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.246872 28439 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.253194 28287 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.273473 28187 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.314221 28289 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:47.328905 28439 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:47.330936 29064 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:47.338472 28367 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:47.341085 29064 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:47.348331 28255 consensus_peers.cc:597] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6fd84f874b14098badd4d77b5703109. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:47.352946 29093 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.355028 28437 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.368216 28367 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.379739 28435 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:47.381824 28188 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.389246 28367 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:47.391535 28289 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:47.391570 28404 consensus_peers.cc:597] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 775a5161e47944f5ad46f1616a48b2fe. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:47.394410 29069 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:47.401258 29069 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:47.470028 29114 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.471648 28367 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.474417 28289 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.493386 28188 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.522117 28367 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:47.533824 28437 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:47.534521 28330 consensus_peers.cc:597] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 189fdd6055df4826a42af9c225596e34. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:47.533872 28289 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:47.535568 29127 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:47.537143 29090 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:47.544533 29108 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.547302 28437 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.554428 28287 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:47.567106 28188 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.706347 29133 ts_tablet_manager.cc:933] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:47.708585 29133 tablet_copy_client.cc:323] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:16:47.710904 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet c59ee067797b40d396f97683aa7eca81 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:16:47.711529 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet c59ee067797b40d396f97683aa7eca81 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-c59ee067797b40d396f97683aa7eca81
I20260709 12:16:47.720101 28376 tablet_copy_source_session.cc:215] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:47.724249 29133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59ee067797b40d396f97683aa7eca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:47.742700 29133 tablet_copy_client.cc:806] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:47.743412 29133 tablet_copy_client.cc:670] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:47.747529 29133 tablet_copy_client.cc:538] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:47.759816 29133 tablet_bootstrap.cc:492] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:47.828531 29133 tablet_bootstrap.cc:492] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:47.829620 29133 tablet_bootstrap.cc:492] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:47.830461 29133 ts_tablet_manager.cc:1403] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.071s	user 0.042s	sys 0.025s
I20260709 12:16:47.833537 29133 raft_consensus.cc:359] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:47.834604 29133 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:47.835392 29133 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:47.852406 29133 ts_tablet_manager.cc:1434] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.003s
I20260709 12:16:47.854442 29136 ts_tablet_manager.cc:933] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:16:47.859097 29136 tablet_copy_client.cc:323] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:16:47.868923 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-c59ee067797b40d396f97683aa7eca81 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:16:47.869414 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-c59ee067797b40d396f97683aa7eca81 on tablet c59ee067797b40d396f97683aa7eca81 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:47.890298 28302 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet d6fd84f874b14098badd4d77b5703109 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:16:47.890936 28302 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet d6fd84f874b14098badd4d77b5703109 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-d6fd84f874b14098badd4d77b5703109
I20260709 12:16:47.899430 28302 tablet_copy_source_session.cc:215] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:47.903578 29136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6fd84f874b14098badd4d77b5703109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:47.915143 29133 ts_tablet_manager.cc:933] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:47.921655 29136 tablet_copy_client.cc:806] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:47.922645 29136 tablet_copy_client.cc:670] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:47.927392 29136 tablet_copy_client.cc:538] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:47.940033 29136 tablet_bootstrap.cc:492] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:47.942553 29133 tablet_copy_client.cc:323] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:16:47.944537 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 775a5161e47944f5ad46f1616a48b2fe from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:16:47.945147 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 775a5161e47944f5ad46f1616a48b2fe from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-775a5161e47944f5ad46f1616a48b2fe
I20260709 12:16:47.953460 28451 tablet_copy_source_session.cc:215] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:47.957320 29133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775a5161e47944f5ad46f1616a48b2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:47.961535 29136 tablet_bootstrap.cc:492] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:47.966055 29136 tablet_bootstrap.cc:492] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:47.966873 29136 ts_tablet_manager.cc:1403] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.013s
I20260709 12:16:47.972259 29136 raft_consensus.cc:359] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:47.973343 29136 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:47.974154 29136 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:47.977324 29136 ts_tablet_manager.cc:1434] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 12:16:47.979347 28302 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-d6fd84f874b14098badd4d77b5703109 received from {username='slave'} at 127.0.0.1:39592
I20260709 12:16:47.979756 28302 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-d6fd84f874b14098badd4d77b5703109 on tablet d6fd84f874b14098badd4d77b5703109 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:47.995599 29133 tablet_copy_client.cc:806] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:47.996270 29133 tablet_copy_client.cc:670] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:48.002460 29133 tablet_copy_client.cc:538] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:48.013316 29133 tablet_bootstrap.cc:492] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:48.037528 29133 tablet_bootstrap.cc:492] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:48.038655 29133 tablet_bootstrap.cc:492] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:48.039502 29133 ts_tablet_manager.cc:1403] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.026s	user 0.021s	sys 0.005s
I20260709 12:16:48.043581 29133 raft_consensus.cc:359] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:48.044526 29133 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:48.045141 29133 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:48.047331 29133 ts_tablet_manager.cc:1434] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 12:16:48.049031 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-775a5161e47944f5ad46f1616a48b2fe received from {username='slave'} at 127.0.0.1:53280
I20260709 12:16:48.049467 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-775a5161e47944f5ad46f1616a48b2fe on tablet 775a5161e47944f5ad46f1616a48b2fe with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:48.068503 29133 ts_tablet_manager.cc:933] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:48.070188 29133 tablet_copy_client.cc:323] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:16:48.071587 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 189fdd6055df4826a42af9c225596e34 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:16:48.072031 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 189fdd6055df4826a42af9c225596e34 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-189fdd6055df4826a42af9c225596e34
I20260709 12:16:48.078048 28376 tablet_copy_source_session.cc:215] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:48.083067 29133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189fdd6055df4826a42af9c225596e34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:48.101202 29133 tablet_copy_client.cc:806] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:48.101924 29133 tablet_copy_client.cc:670] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:48.106971 29133 tablet_copy_client.cc:538] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:48.122838 28991 raft_consensus.cc:1217] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:48.129220 29133 tablet_bootstrap.cc:492] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:48.149974 29133 tablet_bootstrap.cc:492] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:48.150769 29133 tablet_bootstrap.cc:492] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:48.151299 29133 ts_tablet_manager.cc:1403] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 12:16:48.153399 29133 raft_consensus.cc:359] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:48.154031 29133 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:48.154544 29133 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:48.156312 29133 ts_tablet_manager.cc:1434] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:16:48.158011 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-189fdd6055df4826a42af9c225596e34 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:16:48.158416 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-189fdd6055df4826a42af9c225596e34 on tablet 189fdd6055df4826a42af9c225596e34 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:48.369357 28991 raft_consensus.cc:1217] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:48.386727 28991 raft_consensus.cc:1217] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:48.534794 29039 raft_consensus.cc:1064] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:48.537245 29039 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:48.544057 28439 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:48.544178 28991 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:48.545828 28285 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:48.545986 29108 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:48.547276 29039 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:48.547950 29109 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:48.574185 29039 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.577514 28286 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.577514 28991 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.596515 28187 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.603259 28439 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.645921 28367 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:48.650578 28991 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:48.652863 29039 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:48.652886 28439 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:48.659083 29109 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:48.680831 28990 raft_consensus.cc:1217] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:48.699601 29039 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.703161 28435 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.720791 28991 raft_consensus.cc:2955] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.723688 28190 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.730341 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c59ee067797b40d396f97683aa7eca81 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:48.744766 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet c59ee067797b40d396f97683aa7eca81 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59772
I20260709 12:16:48.752070 29156 tablet_replica.cc:333] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:16:48.752924 29156 raft_consensus.cc:2243] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:48.753518 29156 raft_consensus.cc:2272] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.756584 29156 ts_tablet_manager.cc:1916] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:48.769719 29067 raft_consensus.cc:1064] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:48.772379 29067 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:48.772941 29156 ts_tablet_manager.cc:1929] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:48.773471 29156 log.cc:1199] T c59ee067797b40d396f97683aa7eca81 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/c59ee067797b40d396f97683aa7eca81
I20260709 12:16:48.780153 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet c59ee067797b40d396f97683aa7eca81 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:48.791052 28991 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:48.791718 28360 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:48.793941 29067 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:48.794284 28286 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:48.795361 29129 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:48.796295 29114 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:48.818948 29067 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.823421 28289 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.833936 28367 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.842224 28991 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.861706 28189 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.890484 28439 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:48.910843 28289 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:48.911774 28990 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:48.923913 29161 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:48.932636 29146 raft_consensus.cc:1064] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:48.945670 29161 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:16:48.956830 29162 consensus_queue.cc:1175] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 1 status { last_received { term: 1 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 1 index: 2 } }
I20260709 12:16:48.978092 29161 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.982614 28289 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:48.983444 28991 raft_consensus.cc:2955] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.029145 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 775a5161e47944f5ad46f1616a48b2fe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:49.031682 29146 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:49.039444 28991 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:49.040879 29162 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:49.036098 28187 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.049577 28367 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:49.051622 29146 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:49.053766 28439 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:49.060322 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet 775a5161e47944f5ad46f1616a48b2fe with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55934
I20260709 12:16:49.065284 29169 tablet_replica.cc:333] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:16:49.066326 29169 raft_consensus.cc:2243] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.070384 29164 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:49.071672 29169 raft_consensus.cc:2272] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.073559 29162 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.076709 29169 ts_tablet_manager.cc:1916] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:49.077081 28367 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.109275 28991 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.113615 28187 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.134480 28439 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.149382 29169 ts_tablet_manager.cc:1929] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:49.150051 29169 log.cc:1199] T 775a5161e47944f5ad46f1616a48b2fe P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/775a5161e47944f5ad46f1616a48b2fe
I20260709 12:16:49.153136 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet 775a5161e47944f5ad46f1616a48b2fe (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:49.206393 29039 raft_consensus.cc:1064] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:49.209048 29039 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:49.216176 28435 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:49.218521 28286 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:49.218900 29155 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:49.216176 28991 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:49.222184 29109 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:49.228911 29039 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:49.258973 29108 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.272529 28990 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.282372 28289 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.296753 28187 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.305406 28439 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.339586 28360 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:49.345151 28439 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:49.345188 28990 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:49.346524 29108 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:49.347345 29109 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:49.354576 29039 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.356133 28990 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.365203 28432 raft_consensus.cc:2955] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.370601 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 189fdd6055df4826a42af9c225596e34 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:49.384079 28189 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:49.607393 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 75145.1us
I20260709 12:16:49.660471 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 189fdd6055df4826a42af9c225596e34 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59772
I20260709 12:16:49.662680 29176 tablet_replica.cc:333] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:16:49.666646 29176 raft_consensus.cc:2243] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.668165 29176 raft_consensus.cc:2272] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:49.736723 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 203746us
I20260709 12:16:49.765076 29176 ts_tablet_manager.cc:1916] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:49.829226 28286 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:49.852895 28990 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:49.855646 29165 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:49.860091 28439 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:49.870741 29146 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.893414 28990 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:49.913906 29176 ts_tablet_manager.cc:1929] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:49.921665 29175 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:49.965197 29176 log.cc:1199] T 189fdd6055df4826a42af9c225596e34 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/189fdd6055df4826a42af9c225596e34
I20260709 12:16:49.967356 28189 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:50.011574 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d6fd84f874b14098badd4d77b5703109 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:50.072433 28439 raft_consensus.cc:2955] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:50.082908 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 189fdd6055df4826a42af9c225596e34 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:50.183813 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet d6fd84f874b14098badd4d77b5703109 with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55934
I20260709 12:16:50.203459 29185 tablet_replica.cc:333] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:16:50.205049 29185 raft_consensus.cc:2243] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:50.206180 29185 raft_consensus.cc:2272] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:50.210515 29185 ts_tablet_manager.cc:1916] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:50.246227 29185 ts_tablet_manager.cc:1929] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:50.246982 29185 log.cc:1199] T d6fd84f874b14098badd4d77b5703109 P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/d6fd84f874b14098badd4d77b5703109
I20260709 12:16:50.249657 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet d6fd84f874b14098badd4d77b5703109 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
W20260709 12:16:50.658085 28313 debug-util.cc:398] Leaking SignalData structure 0x7b080005fc00 after lost signal to thread 27119
W20260709 12:16:50.659276 28313 debug-util.cc:398] Leaking SignalData structure 0x7b080011da60 after lost signal to thread 28224
W20260709 12:16:50.660710 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec880 after lost signal to thread 28316
W20260709 12:16:50.661753 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002ecaa0 after lost signal to thread 28391
W20260709 12:16:50.662840 28313 debug-util.cc:398] Leaking SignalData structure 0x7b080030f020 after lost signal to thread 28465
W20260709 12:16:50.664054 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5140 after lost signal to thread 29015
I20260709 12:16:51.290939 28291 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:51.305893 28432 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:51.307312 28255 consensus_peers.cc:597] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 777967d6330c4654813298b12226a560. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:51.307613 28358 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.308022 29175 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:51.310127 29146 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.330394 29146 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.332468 28432 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.351728 28439 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:51.355238 28188 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.356916 28358 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:16:51.379690 28404 consensus_peers.cc:597] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23d6b97dd9e9466d9581f6088e681a11. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:51.381762 28367 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.383895 29129 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:51.434078 29067 raft_consensus.cc:2955] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.444995 28291 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.449208 28367 raft_consensus.cc:2955] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.462296 29178 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:51.479957 28189 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.503455 28358 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:51.515476 28287 raft_consensus.cc:2955] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.521591 28432 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:51.522665 28330 consensus_peers.cc:597] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1410767867a5470e81621556e44c6560. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:51.524308 29155 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:51.544171 28287 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.547410 29199 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.551745 29109 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:51.562904 28289 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.571230 28358 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:51.570688 28432 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.586704 28189 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:16:51.598513 28330 consensus_peers.cc:597] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5fb143bfa1d44653b638d40442c7e955. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:51.600085 28432 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.601787 29109 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.604496 28289 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:51.606233 29109 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.622963 29109 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.634593 28439 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.648211 28291 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:51.654285 28189 catalog_manager.cc:5697] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.761566 29208 ts_tablet_manager.cc:933] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:51.772761 29208 tablet_copy_client.cc:323] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:16:51.775029 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 23d6b97dd9e9466d9581f6088e681a11 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:16:51.775652 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 23d6b97dd9e9466d9581f6088e681a11 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-23d6b97dd9e9466d9581f6088e681a11
I20260709 12:16:51.784524 28451 tablet_copy_source_session.cc:215] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:51.788359 29208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23d6b97dd9e9466d9581f6088e681a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.822463 29208 tablet_copy_client.cc:806] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:51.823252 29208 tablet_copy_client.cc:670] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:51.839949 29208 tablet_copy_client.cc:538] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:51.853027 29208 tablet_bootstrap.cc:492] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:51.902252 29208 tablet_bootstrap.cc:492] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:51.903512 29208 tablet_bootstrap.cc:492] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:51.904373 29208 ts_tablet_manager.cc:1403] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.052s	user 0.020s	sys 0.012s
I20260709 12:16:51.908430 29208 raft_consensus.cc:359] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:51.909446 29208 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:51.910293 29208 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:51.923409 29208 ts_tablet_manager.cc:1434] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.006s
I20260709 12:16:51.925457 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-23d6b97dd9e9466d9581f6088e681a11 received from {username='slave'} at 127.0.0.1:53280
I20260709 12:16:51.926012 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-23d6b97dd9e9466d9581f6088e681a11 on tablet 23d6b97dd9e9466d9581f6088e681a11 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:51.933184 29208 ts_tablet_manager.cc:933] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:16:51.942674 29208 tablet_copy_client.cc:323] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:16:51.945271 28302 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet 777967d6330c4654813298b12226a560 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:16:51.945951 28302 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet 777967d6330c4654813298b12226a560 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-777967d6330c4654813298b12226a560
I20260709 12:16:51.963047 28302 tablet_copy_source_session.cc:215] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:51.977996 29208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 777967d6330c4654813298b12226a560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.003808 29208 tablet_copy_client.cc:806] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.004513 29208 tablet_copy_client.cc:670] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.008134 29208 tablet_copy_client.cc:538] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.018121 29208 tablet_bootstrap.cc:492] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:52.038192 29208 tablet_bootstrap.cc:492] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.039247 29208 tablet_bootstrap.cc:492] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:52.039892 29208 ts_tablet_manager.cc:1403] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 12:16:52.042764 29208 raft_consensus.cc:359] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:52.043674 29208 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:52.044422 29208 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:52.049598 29208 ts_tablet_manager.cc:1434] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:16:52.051604 28302 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-777967d6330c4654813298b12226a560 received from {username='slave'} at 127.0.0.1:39592
I20260709 12:16:52.052106 28302 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-777967d6330c4654813298b12226a560 on tablet 777967d6330c4654813298b12226a560 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:52.068331 29208 ts_tablet_manager.cc:933] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:52.175511 29208 tablet_copy_client.cc:323] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:16:52.178285 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 1410767867a5470e81621556e44c6560 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:16:52.178923 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 1410767867a5470e81621556e44c6560 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-1410767867a5470e81621556e44c6560
I20260709 12:16:52.196879 28376 tablet_copy_source_session.cc:215] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.201829 29208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1410767867a5470e81621556e44c6560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.213850 29210 ts_tablet_manager.cc:933] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:52.226776 29210 tablet_copy_client.cc:323] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:16:52.228983 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 5fb143bfa1d44653b638d40442c7e955 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:16:52.229535 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 5fb143bfa1d44653b638d40442c7e955 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-5fb143bfa1d44653b638d40442c7e955
I20260709 12:16:52.244015 28376 tablet_copy_source_session.cc:215] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:52.258909 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fb143bfa1d44653b638d40442c7e955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.268818 29208 tablet_copy_client.cc:806] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.269531 29208 tablet_copy_client.cc:670] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.281185 29208 tablet_copy_client.cc:538] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.290432 29208 tablet_bootstrap.cc:492] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:52.290988 29210 tablet_copy_client.cc:806] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:52.291610 29210 tablet_copy_client.cc:670] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:52.297878 29210 tablet_copy_client.cc:538] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:52.313781 29210 tablet_bootstrap.cc:492] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:52.343116 29208 tablet_bootstrap.cc:492] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.344252 29208 tablet_bootstrap.cc:492] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:52.345027 29208 ts_tablet_manager.cc:1403] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.055s	user 0.025s	sys 0.003s
I20260709 12:16:52.350620 29208 raft_consensus.cc:359] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:52.351481 29208 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:52.352149 29208 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:52.361212 28990 raft_consensus.cc:1217] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:52.367599 28990 raft_consensus.cc:1217] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:52.380090 29208 ts_tablet_manager.cc:1434] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:16:52.386039 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-1410767867a5470e81621556e44c6560 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:16:52.386579 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-1410767867a5470e81621556e44c6560 on tablet 1410767867a5470e81621556e44c6560 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:52.403636 29210 tablet_bootstrap.cc:492] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:52.404816 29210 tablet_bootstrap.cc:492] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:52.405673 29210 ts_tablet_manager.cc:1403] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.012s
I20260709 12:16:52.409276 29210 raft_consensus.cc:359] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:52.410531 29210 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:52.411273 29210 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:52.413700 29210 ts_tablet_manager.cc:1434] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:52.415750 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-5fb143bfa1d44653b638d40442c7e955 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:16:52.416203 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-5fb143bfa1d44653b638d40442c7e955 on tablet 5fb143bfa1d44653b638d40442c7e955 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:52.675035 28990 raft_consensus.cc:1217] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:52.719084 28990 raft_consensus.cc:1217] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:52.802834 29192 raft_consensus.cc:1064] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:52.805799 29192 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:52.858189 28432 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:52.868597 29175 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:52.898533 28367 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:52.904558 28991 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:52.905097 29222 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:52.921036 29221 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:52.925899 29221 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:52.942523 29067 raft_consensus.cc:1064] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:52.945217 29067 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:52.949154 28358 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:52.960423 28991 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:52.980733 28990 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:52.988296 28285 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:52.988197 28367 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:52.991528 28187 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:52.988236 29178 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:16:52.995419 28439 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:53.000828 29067 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:53.008595 29067 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:16:53.614867 28991 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.014s	sys 0.031s
W20260709 12:16:53.615330 28991 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.014s	sys 0.031s
W20260709 12:16:53.611742 29227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.032s	sys 0.032s
W20260709 12:16:53.617125 29227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.033s	sys 0.033s
I20260709 12:16:53.630446 29200 raft_consensus.cc:2955] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:53.661267 29236 raft_consensus.cc:1064] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:53.663115 28988 raft_consensus.cc:2955] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:53.664762 29236 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:53.714391 28991 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:16:53.782099 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 66621.5us
W20260709 12:16:53.855566 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51604.6us
I20260709 12:16:53.925000 28284 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:16:53.927323 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 211667us
I20260709 12:16:53.929240 29236 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:53.940435 28292 raft_consensus.cc:2955] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:53.959831 28187 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:53.980316 29186 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:16:54.017446 29219 raft_consensus.cc:1064] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:54.020150 29219 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:16:54.045629 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:54.046528 28188 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 160) took 1048 ms. Trace:
I20260709 12:16:54.061553 28284 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:54.064463 29248 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:16:54.066483 28188 rpcz_store.cc:276] 0709 12:16:52.997792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:52.998861 (+  1069us) service_pool.cc:224] Handling call
0709 12:16:54.046456 (+1047595us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17555,"reported_tablets":1}
I20260709 12:16:54.066922 29248 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:16:54.077214 29248 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:54.094215 28988 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:54.108165 29244 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:54.111227 28290 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:16:54.114580 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:54.122839 28436 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:54.125140 29219 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:54.129809 28989 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.132114 29234 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:54.141511 28438 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:54.142000 29186 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.146747 28440 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:16:54.142530 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 58882.1us
I20260709 12:16:54.153636 29244 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:54.155906 29249 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:54.186275 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:54.190403 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:54.210855 28364 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.212442 28988 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.219076 29234 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:54.240301 28188 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.227824 28436 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.227145 29242 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.263877 28362 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:54.273636 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:16:54.291468 29251 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:54.297906 28991 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.294834 28434 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.298113 28987 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.292567 29235 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.304729 28290 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.307618 28283 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.199256 28284 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.328569 29200 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:16:54.334172 29251 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:54.345065 28989 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.344378 28991 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.362694 29242 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:16:54.398240 28288 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.376587 28364 raft_consensus.cc:2955] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.407538 28190 catalog_manager.cc:5697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.414769 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 777967d6330c4654813298b12226a560 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:54.411769 29242 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.432552 29249 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:54.438851 28362 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:54.443210 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1410767867a5470e81621556e44c6560 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:54.449090 28987 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.464529 28187 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.478045 28434 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.471431 28190 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.479853 29249 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:54.487475 29263 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.490880 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 777967d6330c4654813298b12226a560 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33090
I20260709 12:16:54.471658 28284 raft_consensus.cc:2955] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.475455 28988 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:54.474390 29251 raft_consensus.cc:2955] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.494493 29186 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:54.500488 28436 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.504207 28988 raft_consensus.cc:2955] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.506681 29254 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:16:54.512307 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5fb143bfa1d44653b638d40442c7e955 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:54.507380 29254 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:54.516058 29254 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:16:54.516595 29265 tablet_replica.cc:333] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:16:54.524221 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 1410767867a5470e81621556e44c6560 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33090
I20260709 12:16:54.524726 29265 raft_consensus.cc:2243] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:54.531925 29265 raft_consensus.cc:2272] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:54.535399 28190 catalog_manager.cc:5697] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.536460 29265 ts_tablet_manager.cc:1916] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:54.544416 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 23d6b97dd9e9466d9581f6088e681a11 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:54.559180 28186 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.570917 28290 raft_consensus.cc:2955] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.586206 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet 23d6b97dd9e9466d9581f6088e681a11 with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55934
I20260709 12:16:54.611996 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fb143bfa1d44653b638d40442c7e955 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59772
I20260709 12:16:54.624920 29268 tablet_replica.cc:333] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:16:54.626423 29268 raft_consensus.cc:2243] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:54.628353 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:16:54.632711 29268 raft_consensus.cc:2272] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:54.627118 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:16:54.626854 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:16:54.640007 29265 ts_tablet_manager.cc:1929] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:54.644217 29265 log.cc:1199] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/777967d6330c4654813298b12226a560
I20260709 12:16:54.638983 28987 raft_consensus.cc:2955] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:54.647665 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 777967d6330c4654813298b12226a560 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:54.644977 29268 ts_tablet_manager.cc:1916] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:54.647830 29265 tablet_replica.cc:333] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:16:54.653599 29265 raft_consensus.cc:2243] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:54.656991 29265 raft_consensus.cc:2272] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:54.671411 29265 ts_tablet_manager.cc:1916] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:16:54.684013 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08002dd340 after lost signal to thread 27119
W20260709 12:16:54.685418 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08002dda20 after lost signal to thread 28224
W20260709 12:16:54.687143 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800310f20 after lost signal to thread 28316
I20260709 12:16:54.678920 28404 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:16:54.693406 28404 raft_consensus.cc:2686] T 777967d6330c4654813298b12226a560 P 9aea2391841b4888b4e195e5a1f4623b: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:16:54.698758 29012 debug-util.cc:398] Leaking SignalData structure 0x7b080031af40 after lost signal to thread 28391
W20260709 12:16:54.700039 29012 debug-util.cc:398] Leaking SignalData structure 0x7b080031e880 after lost signal to thread 28465
W20260709 12:16:54.701284 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800274b80 after lost signal to thread 29015
I20260709 12:16:54.720113 29266 tablet_replica.cc:333] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:16:54.721354 29266 raft_consensus.cc:2243] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:54.722021 29266 pending_rounds.cc:70] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: Trying to abort 1 pending ops.
I20260709 12:16:54.722328 29266 pending_rounds.cc:77] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305623195417153536 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "23d6b97dd9e9466d9581f6088e681a11" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } } }
I20260709 12:16:54.723902 29266 raft_consensus.cc:2917] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 12:16:54.724565 29266 raft_consensus.cc:2889] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:16:54.737923 29266 raft_consensus.cc:2272] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:54.741328 29266 ts_tablet_manager.cc:1916] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:54.747413 29268 ts_tablet_manager.cc:1929] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:54.748106 29268 log.cc:1199] T 5fb143bfa1d44653b638d40442c7e955 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/5fb143bfa1d44653b638d40442c7e955
I20260709 12:16:54.750372 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 5fb143bfa1d44653b638d40442c7e955 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:54.769138 29266 ts_tablet_manager.cc:1929] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:54.769850 29266 log.cc:1199] T 23d6b97dd9e9466d9581f6088e681a11 P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/23d6b97dd9e9466d9581f6088e681a11
I20260709 12:16:54.772047 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet 23d6b97dd9e9466d9581f6088e681a11 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:54.777209 29265 ts_tablet_manager.cc:1929] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:54.777848 29265 log.cc:1199] T 1410767867a5470e81621556e44c6560 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/1410767867a5470e81621556e44c6560
I20260709 12:16:54.779654 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 1410767867a5470e81621556e44c6560 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:55.788892 28366 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:55.800982 28434 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:55.801355 28330 consensus_peers.cc:597] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23f7ed53101a4cc3b4e19149d5a4a1a8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:55.802888 29232 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:55.806103 28291 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.808012 29232 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.829835 29263 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:55.856943 28434 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:55.858315 28436 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:55.889458 28291 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:55.895270 28187 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:16:55.899055 28404 consensus_peers.cc:597] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8fa1ea35643e43ea8ab0f71a59be9e29. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:55.923704 28284 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.929788 28366 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:55.932161 29067 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.967886 29067 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:55.979017 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44024.8us
I20260709 12:16:55.991120 28367 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:55.989213 29067 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:56.067726 28187 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:56.069726 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 54518.5us
I20260709 12:16:56.082662 28291 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:56.117327 28290 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:56.127686 28434 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:56.128875 28366 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:16:56.127562 28255 consensus_peers.cc:597] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e77b6d26dd064626ab3b950e5a6c272f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:56.129963 29243 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:56.131060 29276 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:56.156750 29235 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:56.165230 28366 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:56.172791 28290 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:56.168639 28434 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:56.176858 28190 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:56.201399 28436 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:56.202289 28366 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:56.204164 29235 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:56.207329 28255 consensus_peers.cc:597] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 448f610d25494d2c875c5042ec162f71. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:56.208098 29235 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:56.250371 29243 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:56.254541 28367 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:56.256569 28436 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:16:56.277493 28190 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:56.359654 29288 ts_tablet_manager.cc:933] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:16:56.383265 29288 tablet_copy_client.cc:323] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:16:56.385051 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:16:56.385607 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 23f7ed53101a4cc3b4e19149d5a4a1a8 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-23f7ed53101a4cc3b4e19149d5a4a1a8
I20260709 12:16:56.394228 28376 tablet_copy_source_session.cc:215] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:56.404384 29288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:56.478035 29288 tablet_copy_client.cc:806] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:56.479138 29288 tablet_copy_client.cc:670] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:56.484115 29288 tablet_copy_client.cc:538] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:56.492805 29290 ts_tablet_manager.cc:933] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:16:56.534224 29290 tablet_copy_client.cc:323] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:16:56.539846 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 8fa1ea35643e43ea8ab0f71a59be9e29 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:16:56.540606 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 8fa1ea35643e43ea8ab0f71a59be9e29 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-8fa1ea35643e43ea8ab0f71a59be9e29
I20260709 12:16:56.541296 29288 tablet_bootstrap.cc:492] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:56.549153 28451 tablet_copy_source_session.cc:215] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:56.554303 29290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fa1ea35643e43ea8ab0f71a59be9e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:56.606662 29290 tablet_copy_client.cc:806] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:56.607388 29290 tablet_copy_client.cc:670] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:56.612670 29290 tablet_copy_client.cc:538] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:56.647094 29288 tablet_bootstrap.cc:492] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:56.648006 29290 tablet_bootstrap.cc:492] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:56.648294 29288 tablet_bootstrap.cc:492] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:56.657310 29288 ts_tablet_manager.cc:1403] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.116s	user 0.008s	sys 0.019s
I20260709 12:16:56.672348 29288 raft_consensus.cc:359] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:56.673342 29288 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:56.674170 29288 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:56.706357 29288 ts_tablet_manager.cc:1434] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.000s
I20260709 12:16:56.708559 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-23f7ed53101a4cc3b4e19149d5a4a1a8 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:16:56.709049 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-23f7ed53101a4cc3b4e19149d5a4a1a8 on tablet 23f7ed53101a4cc3b4e19149d5a4a1a8 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:56.746985 29290 tablet_bootstrap.cc:492] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:56.748014 29290 tablet_bootstrap.cc:492] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:56.748788 29290 ts_tablet_manager.cc:1403] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.101s	user 0.020s	sys 0.016s
I20260709 12:16:56.753877 29288 ts_tablet_manager.cc:933] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:16:56.762522 29290 raft_consensus.cc:359] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:56.763635 29290 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:56.764426 29296 ts_tablet_manager.cc:933] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:16:56.764497 29290 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:56.767292 29290 ts_tablet_manager.cc:1434] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:16:56.770150 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-8fa1ea35643e43ea8ab0f71a59be9e29 received from {username='slave'} at 127.0.0.1:53280
I20260709 12:16:56.770603 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-8fa1ea35643e43ea8ab0f71a59be9e29 on tablet 8fa1ea35643e43ea8ab0f71a59be9e29 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:56.792768 29288 tablet_copy_client.cc:323] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:16:56.796337 28302 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet e77b6d26dd064626ab3b950e5a6c272f from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:16:56.797036 28302 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet e77b6d26dd064626ab3b950e5a6c272f from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-e77b6d26dd064626ab3b950e5a6c272f
I20260709 12:16:56.806665 29296 tablet_copy_client.cc:323] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:16:56.805712 28302 tablet_copy_source_session.cc:215] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:56.809157 28301 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet 448f610d25494d2c875c5042ec162f71 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:16:56.809811 28301 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet 448f610d25494d2c875c5042ec162f71 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-448f610d25494d2c875c5042ec162f71
I20260709 12:16:56.811960 29288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77b6d26dd064626ab3b950e5a6c272f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:56.822441 28301 tablet_copy_source_session.cc:215] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:56.826153 29296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448f610d25494d2c875c5042ec162f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:56.840266 29288 tablet_copy_client.cc:806] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:56.841010 29288 tablet_copy_client.cc:670] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:56.852717 29288 tablet_copy_client.cc:538] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:56.855794 28987 raft_consensus.cc:1217] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:56.873276 29296 tablet_copy_client.cc:806] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:56.874050 29296 tablet_copy_client.cc:670] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:56.906574 29288 tablet_bootstrap.cc:492] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:56.909507 29296 tablet_copy_client.cc:538] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:56.939829 29296 tablet_bootstrap.cc:492] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:16:56.952361 29288 tablet_bootstrap.cc:492] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:56.953584 29288 tablet_bootstrap.cc:492] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:56.954587 29288 ts_tablet_manager.cc:1403] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.048s	user 0.026s	sys 0.004s
I20260709 12:16:56.962651 29288 raft_consensus.cc:359] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:56.963667 29288 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:56.964479 29288 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:56.967388 29288 ts_tablet_manager.cc:1434] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:16:56.969856 28301 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-e77b6d26dd064626ab3b950e5a6c272f received from {username='slave'} at 127.0.0.1:39592
I20260709 12:16:56.970384 28301 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-e77b6d26dd064626ab3b950e5a6c272f on tablet e77b6d26dd064626ab3b950e5a6c272f with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:57.032330 28987 raft_consensus.cc:1217] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:57.044265 29296 tablet_bootstrap.cc:492] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:57.046442 29296 tablet_bootstrap.cc:492] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:16:57.047402 29296 ts_tablet_manager.cc:1403] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.108s	user 0.013s	sys 0.020s
I20260709 12:16:57.073105 29296 raft_consensus.cc:359] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:57.074460 29296 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:16:57.075558 29296 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:16:57.089514 29296 ts_tablet_manager.cc:1434] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 12:16:57.094821 28301 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-448f610d25494d2c875c5042ec162f71 received from {username='slave'} at 127.0.0.1:39592
I20260709 12:16:57.095330 28301 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-448f610d25494d2c875c5042ec162f71 on tablet 448f610d25494d2c875c5042ec162f71 with peer f526369c0d0743b192be6e205540a059
I20260709 12:16:57.356768 28987 raft_consensus.cc:1217] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:57.372381 29277 raft_consensus.cc:1064] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:57.375162 29277 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:57.381723 28987 raft_consensus.cc:1217] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:57.433378 28987 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.435304 28436 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.437808 29263 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.442940 28291 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.438743 29232 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:16:57.455421 29303 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:57.484855 29263 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.527204 29301 raft_consensus.cc:1064] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:57.525993 28987 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.536710 29301 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:57.546818 28190 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.558231 28436 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.563673 28283 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.608536 28987 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.619400 28367 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.625806 29250 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.629253 28291 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:57.631162 29250 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.638509 29067 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:57.688562 29067 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.696040 28358 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.736833 28291 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.738802 28987 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.842209 28367 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:57.863111 28988 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.865540 29263 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:57.875017 28189 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.886786 29277 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.891852 28283 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.899269 28988 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.900799 29306 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:57.921885 28283 raft_consensus.cc:2955] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.926128 28440 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:57.929383 29276 raft_consensus.cc:1064] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:57.932094 29276 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:57.932801 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:57.936457 28367 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:57.945894 29067 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:57.964978 29245 raft_consensus.cc:1064] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:16:57.965626 28186 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:57.968079 29245 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:57.979388 29301 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:57.990631 28367 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.017771 28988 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:58.023794 29280 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:58.029814 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 23f7ed53101a4cc3b4e19149d5a4a1a8 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33090
I20260709 12:16:58.042018 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8fa1ea35643e43ea8ab0f71a59be9e29 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:58.045687 28190 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.055187 28987 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.055851 28367 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.038946 28988 raft_consensus.cc:2955] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.058068 29317 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.060160 29317 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.074116 29323 tablet_replica.cc:333] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:16:58.075400 29323 raft_consensus.cc:2243] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:58.076261 29323 raft_consensus.cc:2272] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:58.076803 28440 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.080178 29318 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:58.095757 29323 ts_tablet_manager.cc:1916] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:58.100935 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fa1ea35643e43ea8ab0f71a59be9e29 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59772
I20260709 12:16:58.146540 28991 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.148594 28436 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.150465 28366 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.150918 29276 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.153946 29320 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:58.164841 29327 tablet_replica.cc:333] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:16:58.166235 29276 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:16:58.183604 29324 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:16:58.248291 29327 raft_consensus.cc:2243] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:58.263028 29276 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.286710 29327 raft_consensus.cc:2272] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:58.298840 29327 ts_tablet_manager.cc:1916] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:58.297801 29334 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:16:58.323175 29331 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:16:58.349185 28436 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:16:58.384218 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 88092.1us
I20260709 12:16:58.384085 28434 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.403126 29323 ts_tablet_manager.cc:1929] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:58.403683 29323 log.cc:1199] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/23f7ed53101a4cc3b4e19149d5a4a1a8
I20260709 12:16:58.403683 28364 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.411518 28988 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.416539 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 23f7ed53101a4cc3b4e19149d5a4a1a8 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
W20260709 12:16:58.429719 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45049.1us
I20260709 12:16:58.444206 29327 ts_tablet_manager.cc:1929] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:58.445128 29327 log.cc:1199] T 8fa1ea35643e43ea8ab0f71a59be9e29 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/8fa1ea35643e43ea8ab0f71a59be9e29
I20260709 12:16:58.448479 28367 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.450192 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 8fa1ea35643e43ea8ab0f71a59be9e29 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:58.466069 28189 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.548694 28190 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.614938 28988 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.641573 28290 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:16:58.656910 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 111740us
W20260709 12:16:58.659121 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 102022us
I20260709 12:16:58.663910 28987 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:58.665534 29319 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:58.674348 29334 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.680562 28987 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.702335 28436 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:58.715340 28290 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:16:58.722330 28988 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:58.726480 29322 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:58.734287 29338 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:58.744874 28186 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:58.748829 29317 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.755776 28987 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.761291 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e77b6d26dd064626ab3b950e5a6c272f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:58.763063 28434 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:16:58.770526 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 48528.4us
I20260709 12:16:58.768718 28438 raft_consensus.cc:2955] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.775280 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet e77b6d26dd064626ab3b950e5a6c272f with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55934
I20260709 12:16:58.783828 29276 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:16:58.801208 29341 tablet_replica.cc:333] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:16:58.803313 29341 raft_consensus.cc:2243] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:58.804307 29341 raft_consensus.cc:2272] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:58.815522 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 448f610d25494d2c875c5042ec162f71 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:58.817343 28190 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.824257 29341 ts_tablet_manager.cc:1916] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:58.837878 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet 448f610d25494d2c875c5042ec162f71 with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55934
I20260709 12:16:58.817515 28436 raft_consensus.cc:2955] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:16:58.877146 29341 ts_tablet_manager.cc:1929] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:58.877693 29341 log.cc:1199] T e77b6d26dd064626ab3b950e5a6c272f P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/e77b6d26dd064626ab3b950e5a6c272f
I20260709 12:16:58.879724 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet e77b6d26dd064626ab3b950e5a6c272f (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:16:58.880244 29341 tablet_replica.cc:333] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:16:58.881569 29341 raft_consensus.cc:2243] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:58.882495 29341 raft_consensus.cc:2272] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:58.892673 29341 ts_tablet_manager.cc:1916] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:58.910846 29341 ts_tablet_manager.cc:1929] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:58.911386 29341 log.cc:1199] T 448f610d25494d2c875c5042ec162f71 P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/448f610d25494d2c875c5042ec162f71
I20260709 12:16:58.913873 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet 448f610d25494d2c875c5042ec162f71 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
W20260709 12:16:59.298069 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 100735us
I20260709 12:16:59.993319 28436 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:00.001411 28362 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.002421 28291 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:00.003983 28404 consensus_peers.cc:597] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66010729a96e49ebb751b59d4c7f5f3a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:00.004158 29301 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:00.005115 29335 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:00.019457 29335 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.025458 28291 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.040083 28364 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.047175 28186 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:00.058269 28291 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:00.088878 28436 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.089378 28364 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.091354 29345 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:00.093672 29348 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:00.099797 28255 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cdc4169a890e44e3ad14a98135e7071e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:00.173210 29348 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:17:00.250070 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 74922.9us
I20260709 12:17:00.253386 28364 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.265651 28436 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:17:00.293131 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55552.5us
I20260709 12:17:00.331135 28190 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:00.350764 28438 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:17:00.377725 29357 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.010s
I20260709 12:17:00.394006 28291 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.395421 28364 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.399125 29280 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:00.404394 28404 consensus_peers.cc:597] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 61d225f747aa45ff89df6c0ba308faad. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:00.406020 29280 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:00.421952 29335 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.428640 28364 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.431528 28288 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.449095 28367 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:00.457494 28186 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:17:00.470575 28330 consensus_peers.cc:597] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 819ca6350ca84ff49e7254638d8755e7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:00.481359 28288 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.482872 28438 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.485386 29342 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:00.488087 29263 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:00.505340 29263 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.514920 28288 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.529852 28438 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.549750 28186 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:00.603574 29369 ts_tablet_manager.cc:933] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:00.610697 29369 tablet_copy_client.cc:323] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:17:00.613204 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 66010729a96e49ebb751b59d4c7f5f3a from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:17:00.613979 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 66010729a96e49ebb751b59d4c7f5f3a from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-66010729a96e49ebb751b59d4c7f5f3a
I20260709 12:17:00.627170 28451 tablet_copy_source_session.cc:215] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:00.631826 29369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66010729a96e49ebb751b59d4c7f5f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:00.664534 29369 tablet_copy_client.cc:806] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:00.672657 29369 tablet_copy_client.cc:670] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:00.675719 29371 ts_tablet_manager.cc:933] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:17:00.679458 29369 tablet_copy_client.cc:538] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:00.710484 29369 tablet_bootstrap.cc:492] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:17:00.712343 29371 tablet_copy_client.cc:323] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:17:00.714744 28301 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet cdc4169a890e44e3ad14a98135e7071e from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:17:00.715401 28301 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet cdc4169a890e44e3ad14a98135e7071e from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-cdc4169a890e44e3ad14a98135e7071e
I20260709 12:17:00.745532 29369 tablet_bootstrap.cc:492] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:00.746699 29369 tablet_bootstrap.cc:492] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:17:00.746742 28301 tablet_copy_source_session.cc:215] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:00.747601 29369 ts_tablet_manager.cc:1403] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.037s	user 0.025s	sys 0.005s
I20260709 12:17:00.751231 29369 raft_consensus.cc:359] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:00.752151 29369 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:17:00.752867 29369 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:00.751371 29371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc4169a890e44e3ad14a98135e7071e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:00.813372 29369 ts_tablet_manager.cc:1434] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.065s	user 0.030s	sys 0.029s
I20260709 12:17:00.815646 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-66010729a96e49ebb751b59d4c7f5f3a received from {username='slave'} at 127.0.0.1:53280
I20260709 12:17:00.816501 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-66010729a96e49ebb751b59d4c7f5f3a on tablet 66010729a96e49ebb751b59d4c7f5f3a with peer f526369c0d0743b192be6e205540a059
I20260709 12:17:00.822893 29371 tablet_copy_client.cc:806] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:00.823568 29371 tablet_copy_client.cc:670] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:00.828359 29371 tablet_copy_client.cc:538] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:00.838677 29371 tablet_bootstrap.cc:492] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:17:00.902442 29371 tablet_bootstrap.cc:492] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:00.903501 29371 tablet_bootstrap.cc:492] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:17:00.904286 29371 ts_tablet_manager.cc:1403] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.012s
I20260709 12:17:00.907578 29371 raft_consensus.cc:359] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:00.908471 29371 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:17:00.909134 29371 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:00.919111 29371 ts_tablet_manager.cc:1434] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:17:00.924896 28301 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-cdc4169a890e44e3ad14a98135e7071e received from {username='slave'} at 127.0.0.1:39592
I20260709 12:17:00.925428 28301 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-cdc4169a890e44e3ad14a98135e7071e on tablet cdc4169a890e44e3ad14a98135e7071e with peer f526369c0d0743b192be6e205540a059
I20260709 12:17:00.997428 29371 ts_tablet_manager.cc:933] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:01.010412 29371 tablet_copy_client.cc:323] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:17:01.014820 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 819ca6350ca84ff49e7254638d8755e7 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:17:01.015486 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 819ca6350ca84ff49e7254638d8755e7 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-819ca6350ca84ff49e7254638d8755e7
I20260709 12:17:01.026573 28376 tablet_copy_source_session.cc:215] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:01.039995 29369 ts_tablet_manager.cc:933] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:01.046336 29371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819ca6350ca84ff49e7254638d8755e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:01.046739 29369 tablet_copy_client.cc:323] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:17:01.051517 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 61d225f747aa45ff89df6c0ba308faad from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:17:01.052129 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 61d225f747aa45ff89df6c0ba308faad from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-61d225f747aa45ff89df6c0ba308faad
I20260709 12:17:01.060827 28451 tablet_copy_source_session.cc:215] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:01.068773 29371 tablet_copy_client.cc:806] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:01.075062 29371 tablet_copy_client.cc:670] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:01.083199 29371 tablet_copy_client.cc:538] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:01.091460 29369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61d225f747aa45ff89df6c0ba308faad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:01.112332 29371 tablet_bootstrap.cc:492] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:17:01.127002 29369 tablet_copy_client.cc:806] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:01.129297 29369 tablet_copy_client.cc:670] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:01.133554 29369 tablet_copy_client.cc:538] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:01.161377 28988 raft_consensus.cc:1217] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:01.202010 29369 tablet_bootstrap.cc:492] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:17:01.234089 29371 tablet_bootstrap.cc:492] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:01.237783 29371 tablet_bootstrap.cc:492] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:17:01.241559 29371 ts_tablet_manager.cc:1403] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.129s	user 0.031s	sys 0.024s
I20260709 12:17:01.245658 29371 raft_consensus.cc:359] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:01.251796 29371 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:17:01.253441 28988 raft_consensus.cc:1217] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:01.253096 29371 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:01.257231 29371 ts_tablet_manager.cc:1434] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
W20260709 12:17:01.263311 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47657.6us
I20260709 12:17:01.274739 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-819ca6350ca84ff49e7254638d8755e7 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:17:01.275202 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-819ca6350ca84ff49e7254638d8755e7 on tablet 819ca6350ca84ff49e7254638d8755e7 with peer f526369c0d0743b192be6e205540a059
I20260709 12:17:01.284178 29369 tablet_bootstrap.cc:492] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:01.285039 29369 tablet_bootstrap.cc:492] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:17:01.285706 29369 ts_tablet_manager.cc:1403] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.084s	user 0.028s	sys 0.013s
I20260709 12:17:01.291484 29369 raft_consensus.cc:359] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:01.292376 29369 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:17:01.293179 29369 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:01.295509 29369 ts_tablet_manager.cc:1434] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:17:01.301365 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-61d225f747aa45ff89df6c0ba308faad received from {username='slave'} at 127.0.0.1:53280
I20260709 12:17:01.301957 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-61d225f747aa45ff89df6c0ba308faad on tablet 61d225f747aa45ff89df6c0ba308faad with peer f526369c0d0743b192be6e205540a059
I20260709 12:17:01.398705 28988 raft_consensus.cc:1217] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:01.620318 28988 raft_consensus.cc:1217] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:01.673538 29335 raft_consensus.cc:1064] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:01.676661 29335 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:01.677708 29381 raft_consensus.cc:1064] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:01.683709 29381 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:01.693392 28362 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:01.695442 28987 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:01.699076 29280 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:01.700519 29362 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:01.718462 28288 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:01.725327 28440 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:01.726419 28988 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:01.727564 29383 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:01.729117 28364 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:01.730195 29381 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:01.731504 29346 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:01.732937 29360 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:01.740783 29362 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.747356 28987 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.763065 29346 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.772130 28364 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.773842 28988 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.773869 28440 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.771070 28291 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.794797 28364 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.823112 28190 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.826069 28187 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:01.858467 28291 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:01.861299 29263 raft_consensus.cc:1064] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:01.863812 29263 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:01.863845 28991 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:01.866691 28364 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:01.868196 29346 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:01.871794 29346 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:01.875248 29381 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.887557 28364 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.897214 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cdc4169a890e44e3ad14a98135e7071e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:01.938712 28438 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:01.938552 28991 raft_consensus.cc:2955] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:01.964133 28362 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:01.965772 29362 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:01.966375 28991 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:01.973179 29362 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:01.981163 28189 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:01.996008 29335 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.000824 28987 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.003350 28362 raft_consensus.cc:2955] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.017355 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet cdc4169a890e44e3ad14a98135e7071e with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33090
I20260709 12:17:02.034528 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 66010729a96e49ebb751b59d4c7f5f3a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:02.042142 29399 tablet_replica.cc:333] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:17:02.043643 29399 raft_consensus.cc:2243] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:02.067382 28991 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.071977 28438 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.073345 28291 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.056458 29399 raft_consensus.cc:2272] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:02.074451 29340 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:02.083029 29379 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:02.072793 28189 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:02.084682 29399 ts_tablet_manager.cc:1916] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:02.086460 29379 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:17:02.138757 29398 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:17:02.178306 29399 ts_tablet_manager.cc:1929] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:02.179008 29399 log.cc:1199] T cdc4169a890e44e3ad14a98135e7071e P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/cdc4169a890e44e3ad14a98135e7071e
I20260709 12:17:02.189208 29340 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.190719 28288 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.199928 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 66010729a96e49ebb751b59d4c7f5f3a with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59772
I20260709 12:17:02.224249 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet cdc4169a890e44e3ad14a98135e7071e (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
W20260709 12:17:02.248013 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 63224.2us
I20260709 12:17:02.250782 28438 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.259465 29407 tablet_replica.cc:333] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:17:02.261155 29407 raft_consensus.cc:2243] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:02.262213 29407 raft_consensus.cc:2272] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:02.256215 28987 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.284017 29407 ts_tablet_manager.cc:1916] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:02.287684 29362 raft_consensus.cc:1064] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:02.290521 29362 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:02.296387 28187 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.322211 28987 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.329677 28367 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.329102 29362 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:02.330525 28288 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.331776 29406 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:02.341459 28367 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:02.342932 29362 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:02.359460 28988 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:02.361418 29263 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:02.369129 28434 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:02.375342 29263 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:02.384796 28988 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.383802 29340 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.420146 29407 ts_tablet_manager.cc:1929] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:02.421586 29407 log.cc:1199] T 66010729a96e49ebb751b59d4c7f5f3a P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/66010729a96e49ebb751b59d4c7f5f3a
I20260709 12:17:02.427047 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 66010729a96e49ebb751b59d4c7f5f3a (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:17:02.427752 28438 raft_consensus.cc:2955] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.435007 28187 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 4, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.448944 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 819ca6350ca84ff49e7254638d8755e7 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59772
I20260709 12:17:02.450162 29407 tablet_replica.cc:333] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:17:02.462277 29406 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.464500 28988 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.471673 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 819ca6350ca84ff49e7254638d8755e7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:02.486246 28288 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.496681 28187 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.503528 28364 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.531718 29407 raft_consensus.cc:2243] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:02.532684 29407 raft_consensus.cc:2272] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:02.540165 29407 ts_tablet_manager.cc:1916] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:02.562347 29407 ts_tablet_manager.cc:1929] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:02.562883 29407 log.cc:1199] T 819ca6350ca84ff49e7254638d8755e7 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/819ca6350ca84ff49e7254638d8755e7
I20260709 12:17:02.566145 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 819ca6350ca84ff49e7254638d8755e7 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:17:02.574608 28438 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:02.589300 28988 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:02.589720 28288 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:02.591082 29406 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:02.593873 29335 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:02.601028 29406 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.607441 28988 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.615758 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 61d225f747aa45ff89df6c0ba308faad with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:02.620523 28288 raft_consensus.cc:2955] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:02.632157 28186 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 3 to 4, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:02.648136 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet 61d225f747aa45ff89df6c0ba308faad with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55934
I20260709 12:17:02.669330 29412 tablet_replica.cc:333] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:17:02.670550 29412 raft_consensus.cc:2243] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:02.671520 29412 raft_consensus.cc:2272] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:02.675050 29412 ts_tablet_manager.cc:1916] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:02.695044 29412 ts_tablet_manager.cc:1929] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:02.695588 29412 log.cc:1199] T 61d225f747aa45ff89df6c0ba308faad P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/61d225f747aa45ff89df6c0ba308faad
I20260709 12:17:02.698196 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet 61d225f747aa45ff89df6c0ba308faad (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:17:03.752614 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:03.918479 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:03.919078 28364 raft_consensus.cc:606] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:03.921563 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:03.922044 28364 raft_consensus.cc:606] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:03.924312 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a388b90c9b140649cada36c135d7080"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:03.924785 28364 raft_consensus.cc:606] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:03.938015 28434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:03.938709 28434 raft_consensus.cc:606] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:03.953675 28290 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:03.954917 28287 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:17:03.955588 28287 raft_consensus.cc:606] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:03.958873 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:03.959501 28364 raft_consensus.cc:606] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:03.960791 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:03.962181 28434 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:03.962626 28255 consensus_peers.cc:597] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e2ac8850682043e5bf8caf7fc64fae05. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:03.963231 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:03.964327 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:03.964111 29346 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:03.964511 28364 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:03.965941 29417 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:03.972675 29346 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:03.974785 28434 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:03.990267 28364 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:03.991214 28366 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:03.995872 28187 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.010639 28290 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:04.011162 28330 consensus_peers.cc:597] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 114fa146124747358c997ac1008e1b93. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.012475 29430 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:04.010707 28434 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:04.020344 29430 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:04.025170 29419 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.036187 28290 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.044188 28434 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.058701 28290 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:04.062741 28190 catalog_manager.cc:5697] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.068035 28434 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:04.068574 28255 consensus_peers.cc:597] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c8442db4dc44c619fd67179aa212063. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.069458 29417 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:04.070181 28364 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:04.071792 29417 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:04.077018 29417 raft_consensus.cc:993] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:04.077513 29417 raft_consensus.cc:1081] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:04.081947 28988 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46960
I20260709 12:17:04.082760 28988 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:04.083175 28988 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.085079 29429 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.087602 28364 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.090634 28434 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.091635 28988 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.095819 28988 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:04.096465 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:04.097184 28366 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.097570 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:04.098169 28290 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.098449 28290 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:04.099171 28290 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.100391 28290 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.106595 28366 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:04.107970 28290 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:04.109467 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:04.112392 29448 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:04.107692 28187 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.117767 29448 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:04.118876 29448 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.121179 28434 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:04.161958 29423 raft_consensus.cc:993] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:04.162536 29423 raft_consensus.cc:1081] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:04.166780 28988 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46950
I20260709 12:17:04.178150 28988 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:04.178592 28988 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.167207 29419 raft_consensus.cc:993] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:04.182188 29419 raft_consensus.cc:1081] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:04.189323 28987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a388b90c9b140649cada36c135d7080"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46950
I20260709 12:17:04.190105 28987 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:04.190466 28987 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.226001 29413 raft_consensus.cc:993] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:17:04.226573 29413 raft_consensus.cc:1081] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:17:04.230525 28362 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:17:04.231236 28362 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:04.231781 28362 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.262184 28362 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:04.271508 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:04.272193 28438 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.272478 28438 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:04.273262 28438 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:04.274974 28438 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.276748 28987 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.301882 28362 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:04.312268 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800274e00 after lost signal to thread 27119
W20260709 12:17:04.313529 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08002fb1e0 after lost signal to thread 28224
W20260709 12:17:04.314945 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08002dddc0 after lost signal to thread 28316
W20260709 12:17:04.316138 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08002dd560 after lost signal to thread 28391
W20260709 12:17:04.317682 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800334960 after lost signal to thread 28465
W20260709 12:17:04.319024 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800326860 after lost signal to thread 29015
I20260709 12:17:04.320453 28987 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:04.320542 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:04.321573 28288 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.329835 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:04.330730 28367 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.331136 28367 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:04.332046 28367 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.333091 28367 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.336979 28438 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:04.339509 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:04.340814 29379 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:04.342568 29379 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:04.343930 28288 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:04.375185 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:04.269047 28988 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.376549 29449 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:04.378679 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:04.378886 29449 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:04.379390 28291 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.379535 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:04.380113 28366 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.380505 28988 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:04.380179 29449 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.365444 28367 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:04.343639 29379 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:04.387369 28366 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:04.387868 28291 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:04.388520 28366 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.389703 28366 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.398710 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:04.399889 29452 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:04.400556 29424 raft_consensus.cc:993] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:04.401042 29424 raft_consensus.cc:1081] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:04.406504 28987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46950
I20260709 12:17:04.407148 28987 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:04.407601 28987 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.408623 28186 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:04.423974 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:04.424677 28290 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.424749 28288 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.425338 28284 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:04.433157 28290 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:04.439695 29452 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:04.441025 29452 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.444756 28186 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 1 to 2, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:04.449230 28366 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:04.456995 28987 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.470083 29414 raft_consensus.cc:993] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:17:04.470721 29414 raft_consensus.cc:1081] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:17:04.476303 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:04.476303 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:04.477138 28438 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.477218 28366 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.481112 28366 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:04.482137 28366 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.483543 28366 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.484603 28438 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:04.485116 28987 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:04.490577 28956 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:04.492004 29455 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:04.492575 29455 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:04.493521 29455 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:04.512190 28366 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:04.516177 29457 ts_tablet_manager.cc:933] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:04.513564 28291 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59778
I20260709 12:17:04.520388 28291 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:04.520859 28291 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.519917 28186 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 1 to 2, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:04.519994 28404 consensus_peers.cc:597] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ee4696ba90a447fa07c4d09407217f6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.530422 28366 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:04.527227 28186 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 1 to 2, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:04.543320 29426 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:04.542093 28291 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:04.554587 28287 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:04.556265 29457 tablet_copy_client.cc:323] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:17:04.557531 28291 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:04.558558 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 114fa146124747358c997ac1008e1b93 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:17:04.557443 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:04.559966 28367 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:04.560307 28367 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:04.559296 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 114fa146124747358c997ac1008e1b93 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-114fa146124747358c997ac1008e1b93
I20260709 12:17:04.562393 28367 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:04.563763 28367 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.567360 29404 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:04.557444 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:04.570060 28434 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:04.585469 28367 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:04.590117 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:04.590468 28376 tablet_copy_source_session.cc:215] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:04.591171 29346 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:04.591650 29346 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:04.592522 29346 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:04.594204 29457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 114fa146124747358c997ac1008e1b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:04.599382 28434 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:04.590507 29458 ts_tablet_manager.cc:933] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:17:04.616173 28190 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 1 to 2, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:04.625221 29461 ts_tablet_manager.cc:933] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:17:04.626173 28366 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.628050 29415 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.649582 28367 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:04.651659 29455 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:04.661679 28190 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 1 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:04.706755 28287 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:04.715677 28186 catalog_manager.cc:5697] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:04.728322 29455 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:04.738258 29461 tablet_copy_client.cc:323] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:17:04.759127 28301 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet 6c8442db4dc44c619fd67179aa212063 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:17:04.759871 28301 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet 6c8442db4dc44c619fd67179aa212063 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-6c8442db4dc44c619fd67179aa212063
I20260709 12:17:04.835819 28362 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:04.842492 28301 tablet_copy_source_session.cc:215] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:04.853444 28434 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:04.826181 29458 tablet_copy_client.cc:323] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:17:04.860234 29457 tablet_copy_client.cc:806] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:04.861006 29457 tablet_copy_client.cc:670] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:04.860837 29455 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:04.863819 29461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c8442db4dc44c619fd67179aa212063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:04.871977 28301 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet e2ac8850682043e5bf8caf7fc64fae05 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:17:04.875234 28301 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet e2ac8850682043e5bf8caf7fc64fae05 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-e2ac8850682043e5bf8caf7fc64fae05
I20260709 12:17:04.872205 29379 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:17:04.907984 28301 tablet_copy_source_session.cc:215] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:04.916854 29458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2ac8850682043e5bf8caf7fc64fae05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:04.922668 28367 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:04.929337 28287 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:04.958737 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 89169.9us
I20260709 12:17:04.961233 28291 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:04.972122 29457 tablet_copy_client.cc:538] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:04.948521 29455 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:17:04.996327 29424 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:17:04.990691 29448 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 12:17:05.027230 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 63019.6us
I20260709 12:17:05.104396 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:05.149446 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 158090us
I20260709 12:17:05.159607 28440 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:05.167972 28433 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:05.173853 29461 tablet_copy_client.cc:806] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.174610 29461 tablet_copy_client.cc:670] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:05.185699 29486 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:17:05.183604 29483 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:17:05.210135 28366 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:05.214169 29457 tablet_bootstrap.cc:492] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:17:05.223341 28366 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:05.228250 28291 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:05.256733 28288 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:17:05.296370 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 62721us
W20260709 12:17:05.343361 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 93076.1us
I20260709 12:17:05.306806 29481 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.126s
I20260709 12:17:05.395982 29346 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:05.400535 29495 ts_tablet_manager.cc:933] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:05.393791 29455 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 12:17:05.498487 29452 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:17:05.501791 29488 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.207s	user 0.002s	sys 0.000s
I20260709 12:17:05.503892 29461 tablet_copy_client.cc:538] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:17:05.524235 29493 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
W20260709 12:17:05.525624 29487 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:17:05.533491 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46960 (request call id 125) took 1038 ms. Trace:
I20260709 12:17:05.534235 29458 rpcz_store.cc:276] 0709 12:17:04.495570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:04.495947 (+   377us) service_pool.cc:224] Handling call
0709 12:17:04.616170 (+120223us) ts_tablet_manager.cc:934] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
0709 12:17:04.913306 (+297136us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:17:05.088683 (+175377us) tablet_metadata.cc:665] Metadata flushed
0709 12:17:05.088809 (+   126us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:17:05.532696 (+443887us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:17:05.533018 (+   322us) tablet_replica.cc:206] Creating consensus instance
0709 12:17:05.533340 (+   322us) ts_tablet_manager.cc:989] Replica registered.
0709 12:17:05.533408 (+    68us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":45836,"tablet-copy.queue_time_us":93806,"thread_start_us":252746,"threads_started":2}
I20260709 12:17:05.538901 29458 tablet_copy_client.cc:806] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.539960 29458 tablet_copy_client.cc:670] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
W20260709 12:17:05.559506 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 196993us
W20260709 12:17:05.581907 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 231916us
W20260709 12:17:05.589916 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 243847us
W20260709 12:17:05.601473 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46788.1us
I20260709 12:17:05.614629 28434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:05.615589 28434 raft_consensus.cc:606] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:05.650211 29468 raft_consensus.cc:993] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:17:05.663743 28292 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:17:05.664486 28292 raft_consensus.cc:606] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:05.664907 29495 tablet_copy_client.cc:323] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:17:05.660738 29428 raft_consensus.cc:1081] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:17:05.669325 29458 tablet_copy_client.cc:538] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:05.680092 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 3ee4696ba90a447fa07c4d09407217f6 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:17:05.680092 28433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:05.701788 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 3ee4696ba90a447fa07c4d09407217f6 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-3ee4696ba90a447fa07c4d09407217f6
I20260709 12:17:05.702080 28433 raft_consensus.cc:606] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:05.731436 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 72011.9us
I20260709 12:17:05.756162 28362 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:17:05.756960 28362 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:05.757400 28362 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:05.800034 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 58123.6us
I20260709 12:17:05.819159 28362 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:05.826582 28362 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:05.801412 28433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d63db98cb72412194c214fa082eee94"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:05.832492 29482 raft_consensus.cc:993] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:17:05.833191 29417 raft_consensus.cc:1081] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
W20260709 12:17:05.852579 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 232931us
I20260709 12:17:05.864146 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:05.864851 28432 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:05.865144 28432 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:05.848307 28433 raft_consensus.cc:606] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
W20260709 12:17:05.906832 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 105948us
I20260709 12:17:05.958757 29461 tablet_bootstrap.cc:492] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:17:05.959806 28367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:05.960433 28367 raft_consensus.cc:606] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:05.969328 29458 tablet_bootstrap.cc:492] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
W20260709 12:17:05.975684 29506 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:17:05.993285 28434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:05.994067 28434 raft_consensus.cc:606] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:06.003764 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:06.005908 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:06.007790 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:06.022322 28438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:17:06.023011 28438 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:06.023411 28438 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.045548 28438 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:06.050233 28438 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:06.059448 29496 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:06.060153 29496 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:06.086162 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:06.086992 28363 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.089962 29496 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:06.095645 29415 raft_consensus.cc:993] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:17:06.096261 29415 raft_consensus.cc:1081] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:17:06.109169 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:06.110194 28290 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:06.110591 28290 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:06.112723 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 340217us
I20260709 12:17:06.122380 28290 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:06.125610 28290 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.130450 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:06.156006 28290 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:06.146485 28432 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:06.165144 28432 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.168493 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:06.169204 28291 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.201586 28363 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:06.210175 28291 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:06.213565 29457 tablet_bootstrap.cc:492] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.237402 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:06.224753 29511 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.143s	user 0.001s	sys 0.001s
W20260709 12:17:06.227200 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 79999.5us
I20260709 12:17:06.267856 29485 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:06.297859 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:06.293678 29457 tablet_bootstrap.cc:492] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:17:06.318816 29480 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:06.311570 29485 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:06.328737 29457 ts_tablet_manager.cc:1403] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.116s	user 0.051s	sys 0.051s
I20260709 12:17:06.327162 29480 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:06.363557 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 70719.5us
W20260709 12:17:06.307035 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55025.8us
I20260709 12:17:06.392246 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:06.389721 28358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:17:06.403645 28358 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:06.404059 28358 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.397085 28432 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:06.395366 29457 raft_consensus.cc:359] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:06.419545 29457 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:17:06.420342 29457 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:06.402200 29480 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:06.386688 29485 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:06.468943 29531 raft_consensus.cc:993] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:06.469576 29531 raft_consensus.cc:1081] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:06.543421 28358 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:06.558585 29457 ts_tablet_manager.cc:1434] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.210s	user 0.008s	sys 0.000s
I20260709 12:17:06.559751 29458 tablet_bootstrap.cc:700] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Time spent opening tablet: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:17:06.561290 28988 raft_consensus.cc:1217] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:06.584846 29528 raft_consensus.cc:993] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:17:06.585424 29528 raft_consensus.cc:1081] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
W20260709 12:17:06.586403 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 143774us
I20260709 12:17:06.592922 28360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:17:06.654771 28989 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d63db98cb72412194c214fa082eee94"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46934
I20260709 12:17:06.655562 28989 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:06.656075 28989 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:06.779523 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:06.827327 28989 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:06.886595 29461 tablet_bootstrap.cc:492] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.963160 29461 tablet_bootstrap.cc:492] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:17:06.964162 29461 ts_tablet_manager.cc:1403] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.006s	user 0.023s	sys 0.014s
I20260709 12:17:06.968448 29461 raft_consensus.cc:359] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:06.969609 29461 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:17:06.922652 28358 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:06.924239 28363 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.007s	sys 0.004s
W20260709 12:17:06.974959 28363 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.008s	sys 0.005s
I20260709 12:17:06.976027 28365 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:06.927035 28360 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:06.953500 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:06.955514 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:06.982419 28286 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.982323 28438 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:06.983088 28438 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:06.983882 28438 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:06.984808 28438 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.989902 28360 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.996104 28989 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:06.998636 28439 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.021s	sys 0.018s
W20260709 12:17:06.999034 28439 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.021s	sys 0.018s
I20260709 12:17:06.999696 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:07.000344 28434 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:07.000638 28434 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:07.001412 28434 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:07.002415 28434 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:07.003726 28988 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:07.004113 29423 raft_consensus.cc:993] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:07.004606 29423 raft_consensus.cc:1081] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:07.006846 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:07.007463 28358 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:07.009979 28434 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
W20260709 12:17:07.011159 29485 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.026s	sys 0.038s
W20260709 12:17:07.011530 29485 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.026s	sys 0.038s
I20260709 12:17:07.011803 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:07.012470 28436 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:07.013527 29524 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:07.014202 29524 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:07.014317 28991 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46950
I20260709 12:17:07.014945 28991 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:07.015307 28991 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:07.015184 29524 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:07.022722 28330 consensus_peers.cc:597] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ee4696ba90a447fa07c4d09407217f6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:07.027244 28991 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:07.027834 28438 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:06.970558 29461 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:07.009827 28286 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:07.043130 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:07.045089 28991 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:07.048539 28360 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:07.050361 28365 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:07.053493 28358 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:07.060205 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:07.060653 29508 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:07.061282 29508 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:07.056167 29461 ts_tablet_manager.cc:1434] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.092s	user 0.010s	sys 0.000s
I20260709 12:17:07.060904 28364 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:07.062280 29508 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:07.062870 28364 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:07.047600 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:07.064617 28439 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:07.067205 28364 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:07.068773 28364 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:07.077122 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 121396us
I20260709 12:17:07.078817 28364 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:07.079768 28360 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:07.091130 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:07.092732 29522 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:07.093338 29522 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:07.099016 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:07.099186 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:07.099925 28436 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:07.100236 28436 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:07.102250 29538 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:07.101106 28436 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:07.094341 29522 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:07.103152 28436 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:07.103542 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:07.110395 29539 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:07.102824 29538 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:07.116262 29538 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:07.117794 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:07.129061 28439 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
W20260709 12:17:07.136950 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:07.138455 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:07.145113 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-114fa146124747358c997ac1008e1b93 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:17:07.145612 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-114fa146124747358c997ac1008e1b93 on tablet 114fa146124747358c997ac1008e1b93 with peer f526369c0d0743b192be6e205540a059
I20260709 12:17:07.150498 29501 raft_consensus.cc:1064] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:07.154102 29471 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:07.154728 29471 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:07.153281 29458 tablet_bootstrap.cc:492] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.157199 29458 tablet_bootstrap.cc:492] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:17:07.159436 29458 ts_tablet_manager.cc:1403] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.190s	user 0.031s	sys 0.021s
I20260709 12:17:07.159885 29471 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:07.153276 29501 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:07.162859 29458 raft_consensus.cc:359] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:07.163758 29458 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:17:07.164680 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:07.164451 29458 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:07.180415 29458 ts_tablet_manager.cc:1434] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:17:07.182575 28988 raft_consensus.cc:1217] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:17:07.218185 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 112103us
I20260709 12:17:07.224027 28286 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:07.235214 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:07.245195 28286 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:17:07.246650 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:07.247228 28286 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:07.254957 28286 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:07.631911 29538 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:07.632529 29538 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:07.640214 29538 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:07.750157 29538 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.750887 29538 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:07.753424 29538 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:08.371033 29381 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.371922 29381 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:08.374542 29381 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:08.375824 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5120 after lost signal to thread 27119
I20260709 12:17:08.390609 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1133) took 1439 ms. Trace:
I20260709 12:17:08.422581 29550 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:08.425702 29550 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.426415 29381 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.423197 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1141) took 1469 ms. Trace:
I20260709 12:17:08.427102 29381 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.427706 28289 rpcz_store.cc:276] 0709 12:17:06.953641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:08.421979 (+1468338us) service_pool.cc:224] Handling call
0709 12:17:08.422464 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:08.422695 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:08.422726 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:08.422772 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:08.422802 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:08.422856 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:08.422999 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:08.428802 29550 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:08.430382 29381 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:08.430961 28288 rpcz_store.cc:276] 0709 12:17:06.951461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:08.389494 (+1438033us) service_pool.cc:224] Handling call
0709 12:17:08.389981 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:08.390194 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:08.390222 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:08.390270 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:08.390303 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:08.390361 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:08.390527 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:08.436669 29550 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:08.436872 29381 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:08.437335 29550 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:08.437592 29381 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:08.438429 29551 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.443542 29551 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.449753 29381 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:08.452288 29551 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:08.451031 29550 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:08.455832 29550 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.457217 29381 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.457876 29381 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.459450 29551 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.456485 29550 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:08.462674 29551 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.472157 29381 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:08.474116 29381 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.476660 29381 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.473301 29550 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:08.477790 29551 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:08.485193 29381 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:08.521221 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:08.664772 29551 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.666213 29551 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.668052 29550 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.668757 29550 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.671888 29550 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:08.672616 29550 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:08.678042 29550 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.870415 29550 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:08.871263 29550 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:08.871845 29550 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:08.794425 29381 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.052605 29381 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.055068 29381 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.055759 29381 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.056324 29381 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.057137 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1140) took 2103 ms. Trace:
I20260709 12:17:09.057546 28287 rpcz_store.cc:276] 0709 12:17:06.953377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.056204 (+2102827us) service_pool.cc:224] Handling call
0709 12:17:09.056588 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:09.056765 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:09.056789 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:09.056826 (+    37us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:09.056874 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:09.056939 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:09.057076 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:09.058547 29381 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:09.051359 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:09.059214 29381 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.059963 29381 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.051481 29448 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:08.695408 29551 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:08.701601 28436 consensus_meta.cc:287] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.598s	user 0.010s	sys 0.022s
I20260709 12:17:09.059566 29524 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.103084 29448 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.103420 29524 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.062155 29556 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.059545 29553 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.105370 29448 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.105386 29524 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.106575 29556 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.108367 29556 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.109081 28436 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
W20260709 12:17:09.109539 28436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 1137) took 2011 ms (client timeout 2102 ms). Trace:
W20260709 12:17:09.109848 28436 rpcz_store.cc:269] 0709 12:17:07.098640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:07.098811 (+   171us) service_pool.cc:224] Handling call
0709 12:17:09.109450 (+2010639us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:09.003947 29552 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.066176 29554 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.065110 29550 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.068369 29491 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.115221 29554 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.076740 29558 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.078621 29426 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.081866 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1136) took 2003 ms. Trace:
I20260709 12:17:09.084398 29560 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.085091 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1134) took 2133 ms. Trace:
I20260709 12:17:09.129297 28290 rpcz_store.cc:276] 0709 12:17:06.951775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.083925 (+2132150us) service_pool.cc:224] Handling call
0709 12:17:09.084379 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:09.084643 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:09.084675 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:09.084723 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:09.084757 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:09.084814 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:09.084953 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:09.086699 29561 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.088981 29564 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.089157 28451 tablet_copy_source_session.cc:215] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:09.089891 29455 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.098483 29485 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.098829 29415 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.099733 29562 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.097806 29568 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.100551 29566 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.102401 29563 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.105777 29553 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.067193 29555 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.114323 29423 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.110663 29552 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.124708 28432 rpcz_store.cc:276] 0709 12:17:07.078547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:07.078706 (+   159us) service_pool.cc:224] Handling call
0709 12:17:09.062978 (+1984272us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b58001d4698. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:09.063195 (+   217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:09.063453 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:09.063506 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:09.064001 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:17:09.081121 (+ 17120us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:09.081171 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:09.081250 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:09.081401 (+   151us) raft_consensus.cc:1685] finished
0709 12:17:09.081422 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:09.081763 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11608960,"thread_start_us":14303,"threads_started":1,"wal-append.queue_time_us":15318}
I20260709 12:17:09.118901 29491 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:09.121582 29558 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.199266 28451 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:53280 (request call id 25) took 3519 ms. Trace:
I20260709 12:17:09.190891 29455 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.195326 29551 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:09.199316 28301 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-6c8442db4dc44c619fd67179aa212063 received from {username='slave'} at 127.0.0.1:39592
I20260709 12:17:09.213619 29558 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.191771 29415 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.138276 29426 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.196796 29485 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.204929 28451 rpcz_store.cc:276] 0709 12:17:05.679770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:05.679907 (+   137us) service_pool.cc:224] Handling call
0709 12:17:09.088642 (+3408735us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400415354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8913f7 00007f85c3827036 00007f85c3830a3e 00007f85bb0f427b 00007f85c38309ba 00007f85c382bacc 00007f85c3826c7b 00007f85c381b8d1 00007f85c28cf4b4 00007f85c28cf442
0709 12:17:09.199168 (+110526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785941888}
I20260709 12:17:09.186746 29561 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.223284 29556 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.216632 28301 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-6c8442db4dc44c619fd67179aa212063 on tablet 6c8442db4dc44c619fd67179aa212063 with peer f526369c0d0743b192be6e205540a059
I20260709 12:17:09.216809 29550 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.197926 29555 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.224670 29524 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.197379 29423 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.191907 29560 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.198083 29568 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.214912 29551 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.193676 29564 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.200673 29563 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.204423 29562 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.200323 29566 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:09.233326 29556 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.537415 29556 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.249964 29550 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.540488 29550 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.454358 29552 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.262612 29565 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.485538 29570 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.541682 29565 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.541906 29570 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.544167 29565 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.415521 29573 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.544852 29565 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.545286 29573 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.547451 29560 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.547621 29573 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.261906 29426 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.545356 29565 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:09.648530 29426 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.548409 29524 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:09.261088 28257 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.3:33721 timed out after 1.855s (SENT) [suppressed 3 similar messages]
W20260709 12:17:09.650326 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 389277us
I20260709 12:17:09.651912 29524 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.649224 29426 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.263799 29491 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.654294 29426 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.654857 29426 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.655403 29426 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.656629 29491 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.657264 29491 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.657724 29426 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.659672 29491 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.549674 29562 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.262494 29425 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.660516 29425 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.663141 29425 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.663821 29425 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.664552 29425 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.263407 29485 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.263531 29571 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.454850 29575 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.665925 29575 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.667593 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:09.668238 29575 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.262233 29557 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.668900 29557 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.669481 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:09.665925 29571 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.671295 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:09.672228 29571 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.552697 29564 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.672953 29564 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.672978 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:09.673789 28287 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:09.675791 29557 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.554998 29566 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.410828 29415 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.413961 29448 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.677022 29448 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.679920 29448 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.311121 29553 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.563761 29568 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.564720 29455 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.567656 29551 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.415267 29558 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.681695 29558 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.683059 29425 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.684482 29558 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.410557 29561 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.582311 29423 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.537962 29555 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.540534 29563 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.538690 29556 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.686197 29556 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.688989 29556 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.622428 29579 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.698318 29579 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.701146 29579 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:09.629897 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 72313.7us
W20260709 12:17:09.630457 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 331553us
I20260709 12:17:09.544196 29570 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.301190 28302 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-e2ac8850682043e5bf8caf7fc64fae05 received from {username='slave'} at 127.0.0.1:39592
I20260709 12:17:09.706000 28302 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-e2ac8850682043e5bf8caf7fc64fae05 on tablet e2ac8850682043e5bf8caf7fc64fae05 with peer f526369c0d0743b192be6e205540a059
I20260709 12:17:09.436993 29572 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.707942 29572 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.710600 29572 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.673532 29564 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.390669 29569 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.711844 29415 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.711689 29569 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.712400 29415 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.714346 29569 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.714473 29415 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.714996 29415 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.715480 29415 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.716547 29564 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.717891 29570 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.718403 29570 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.720472 29570 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.720942 29570 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.728241 29565 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.731061 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:09.737583 29485 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.738336 29485 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.740645 29485 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.741191 29485 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.741715 29485 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.743300 29557 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.746398 29557 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:09.748687 29557 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.750064 28432 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:09.759488 28432 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:09.745604 29485 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.782126 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.717476 29415 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.721462 29570 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.803583 29570 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:09.839932 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:09.844071 29564 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.844836 29564 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:09.846498 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 217360us
I20260709 12:17:09.847141 29564 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.850560 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.851262 28432 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:09.852993 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.853123 29381 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.853686 29569 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:09.854387 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:09.854336 29569 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.855068 28436 raft_consensus.cc:2376] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2. Re-sending same reply.
I20260709 12:17:09.856554 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.857201 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:09.857964 28405 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.010s	sys 0.006s
I20260709 12:17:09.856531 29569 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:09.858353 28405 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.010s	sys 0.006s
I20260709 12:17:09.858959 29423 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:09.859560 29423 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.860551 29565 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.861023 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.862421 29569 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.862505 29575 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.870368 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.873138 28433 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:09.863812 29415 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.872354 29425 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.873844 29415 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.874061 29425 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:09.866655 29498 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.867959 29426 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:09.860458 29582 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.868511 29570 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.869055 29485 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.868530 29423 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.870078 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.870878 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.871742 28440 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:09.872912 29575 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.863015 29554 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.877110 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.878857 29582 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.885286 28439 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:09.879448 29426 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:09.877903 29565 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.881600 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.884714 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.877538 29569 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.920053 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:09.883800 29485 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.960705 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.968573 28432 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:09.967883 28440 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:09.935810 29495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee4696ba90a447fa07c4d09407217f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.885251 29491 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.961267 29415 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.922606 29426 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:09.965379 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:09.880693 29570 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.938444 28189 catalog_manager.cc:5697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 2, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 2 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:09.967276 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:09.967373 29569 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.887884 29575 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.970615 29423 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.971232 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:09.990705 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4e80 after lost signal to thread 28224
I20260709 12:17:09.934245 29577 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:09.883720 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.989526 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.988905 29565 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.876046 29564 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:09.994079 28360 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
W20260709 12:17:09.994195 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 93298.9us
I20260709 12:17:09.972302 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:09.998888 29530 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.968125 29425 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:09.998974 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.995422 29415 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.003202 28435 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:09.999826 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.002094 29569 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:10.004326 28432 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:10.001629 28358 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:09.995226 29480 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:09.934775 29582 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:09.999094 29577 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:09.993810 29508 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:09.998637 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.004107 29425 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:10.012727 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.012841 28362 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:10.014111 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:09.886071 29498 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:10.021885 28313 debug-util.cc:398] Leaking SignalData structure 0x7b080014cdc0 after lost signal to thread 28316
W20260709 12:17:10.024295 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08001f8540 after lost signal to thread 28317
I20260709 12:17:10.002826 29565 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:10.018502 29474 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.024837 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1238) took 3205 ms. Trace:
I20260709 12:17:10.004264 29564 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.026952 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.028100 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:10.035511 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:09.996846 29491 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.030409 29556 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:10.031841 29485 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:10.003155 29423 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.042095 29507 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:10.006340 29569 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.032708 29565 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.033018 29555 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:10.003721 29415 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:10.033344 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.023481 29425 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:10.033489 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.054585 28287 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:10.034287 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.055375 28288 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:10.056107 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.036715 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.034956 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.016139 29480 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.039069 28988 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.039294 28286 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.063275 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.043125 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.064124 28291 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:10.046233 28358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:10.065038 28358 raft_consensus.cc:606] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:10.030192 29474 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:10.046680 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.047762 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.057250 28289 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:10.074855 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.076152 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.079003 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.080931 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.062436 29507 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.086598 29577 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:10.088658 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.042021 29556 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:10.089342 28288 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:10.090543 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.090704 28360 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:10.052598 29471 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:10.096697 29471 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.097038 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.029165 29508 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:10.102699 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.104282 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.099118 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.100333 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.011304 29530 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.106227 28365 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:10.114295 28284 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:10.115604 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.116869 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.097139 29471 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:10.054028 29555 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:10.167773 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:10.104730 29507 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.237389 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.168941 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.124563 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.409596 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.041795 28359 rpcz_store.cc:276] 0709 12:17:06.817638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:06.817844 (+   206us) service_pool.cc:224] Handling call
0709 12:17:10.019458 (+3201614us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b5800113798. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:10.022102 (+  2644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:10.022312 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:10.022340 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:10.022391 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:10.022423 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:10.022487 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:10.022637 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3046272}
W20260709 12:17:10.326503 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 107267us
I20260709 12:17:10.240074 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.094251 28285 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:10.118707 29590 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:10.417608 28290 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:10.418576 28288 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:10.367846 29423 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.253041 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.236147 29518 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:10.425129 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.443256 28358 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:10.257616 29498 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:10.418237 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.276613 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.447606 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.450892 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.451763 28358 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:10.446764 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.426941 29507 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:10.428136 29564 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.162881 29588 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.458546 28284 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:10.444665 29415 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:10.446070 29565 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.461966 29556 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.436343 29590 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.479084 29569 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.465325 28985 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:10.471058 29491 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:10.469241 29508 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:10.465947 29518 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.464792 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.486272 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.808130 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.809118 28186 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 111) took 3629 ms. Trace:
I20260709 12:17:10.793184 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.804001 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.755424 28187 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 211) took 4496 ms. Trace:
I20260709 12:17:10.811713 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.791530 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.813115 28187 rpcz_store.cc:276] 0709 12:17:06.259228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:06.259430 (+   202us) service_pool.cc:224] Handling call
0709 12:17:10.755065 (+4495635us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3}
I20260709 12:17:10.589751 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.797503 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.649356 29471 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.592658 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.783953 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.811355 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.806598 28190 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 220) took 4563 ms. Trace:
I20260709 12:17:10.822031 28190 rpcz_store.cc:276] 0709 12:17:06.242627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:06.242789 (+   162us) service_pool.cc:224] Handling call
0709 12:17:10.806347 (+4563558us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2}
I20260709 12:17:10.817569 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.818220 29423 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:10.826893 29423 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.828708 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.828904 28285 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:10.809629 28186 rpcz_store.cc:276] 0709 12:17:07.179320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:07.179509 (+   189us) service_pool.cc:224] Handling call
0709 12:17:10.808832 (+3629323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26914,"reported_tablets":1}
I20260709 12:17:10.818480 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.806177 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:10.794941 28313 debug-util.cc:398] Leaking SignalData structure 0x7b080005fc40 after lost signal to thread 28391
I20260709 12:17:10.835631 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:10.835635 28313 debug-util.cc:398] Leaking SignalData structure 0x7b080005fba0 after lost signal to thread 28392
I20260709 12:17:10.832094 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.803012 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.784755 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.814571 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.570148 29590 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.577010 29555 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:10.819355 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.478380 29480 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.847589 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.848798 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.849045 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.849972 29471 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:10.850172 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.851130 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.851935 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.847595 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.852813 28989 raft_consensus.cc:1217] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:10.851979 28991 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:10.850219 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.854421 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.703333 29495 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46934 (request call id 211) took 5395 ms. Trace:
I20260709 12:17:10.858511 28987 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:10.858130 29495 rpcz_store.cc:276] 0709 12:17:05.307686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:05.307848 (+   162us) service_pool.cc:224] Handling call
0709 12:17:05.401016 (+ 93168us) ts_tablet_manager.cc:934] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
0709 12:17:09.876846 (+4475830us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:17:10.573126 (+696280us) tablet_metadata.cc:665] Metadata flushed
0709 12:17:10.573260 (+   134us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:17:10.702327 (+129067us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:17:10.702703 (+   376us) tablet_replica.cc:206] Creating consensus instance
0709 12:17:10.703147 (+   444us) ts_tablet_manager.cc:989] Replica registered.
0709 12:17:10.703225 (+    78us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":22962,"tablet-copy.queue_time_us":91533,"thread_start_us":332603,"threads_started":2}
I20260709 12:17:10.859794 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.860694 29495 tablet_copy_client.cc:806] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:10.861924 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.861338 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.829643 29423 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.864419 28360 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:10.861670 29495 tablet_copy_client.cc:670] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:10.870043 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.473209 29564 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:10.874191 29530 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:10.874790 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.875098 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.871286 28364 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:10.478294 29565 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.880153 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.879271 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.879805 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.882726 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.883615 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.882741 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.884640 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.885279 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.887032 28987 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:10.884678 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.889133 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.878376 28433 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:10.892259 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.472277 29588 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:10.896406 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.898238 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:10.901628 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.902268 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.902244 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.903421 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.903087 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.904462 28289 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:10.904879 28440 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:10.897684 28291 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:10.907315 28285 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:10.907886 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.896612 28287 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
W20260709 12:17:10.910516 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002f67c0 after lost signal to thread 28465
I20260709 12:17:10.912214 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.913133 28991 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:10.890105 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.886102 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:10.912554 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8e40 after lost signal to thread 29015
W20260709 12:17:10.916532 28313 debug-util.cc:398] Leaking SignalData structure 0x7b0800345820 after lost signal to thread 29016
I20260709 12:17:10.917135 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.919461 29518 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:10.472733 29507 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.923861 28292 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:10.590816 29425 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:10.925640 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.929883 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.930193 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.930872 28440 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:10.932097 29564 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:10.932901 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.934258 28286 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:10.937106 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.586480 29570 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:10.939105 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.942595 29474 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:10.943199 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:10.947049 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.947759 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.950673 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.953823 28362 raft_consensus.cc:2410] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:10.954519 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.912932 29588 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:10.959033 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.960073 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.961890 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.961225 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.959971 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.967290 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.968803 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:10.970968 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002f3660 after lost signal to thread 29523
W20260709 12:17:10.971807 28330 connection.cc:256] RPC call timeout handler was delayed by -1.6793176359999507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 12:17:10.972221 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.972597 28286 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:10.974064 28438 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:10.934197 29565 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:10.976025 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.975864 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.979815 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.979648 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:10.981016 28330 connection.cc:256] RPC call timeout handler was delayed by -1.0683542299998408s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
I20260709 12:17:10.981894 29555 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:10.982400 29555 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:10.987586 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.988719 28359 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:10.982822 29555 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:10.989768 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.991279 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:10.991653 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:10.995097 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:10.995965 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:10.996373 28292 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:10.996017 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:10.999758 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.000013 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.000484 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.000798 28991 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:11.003283 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.001075 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.006626 29507 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.008478 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.009161 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.010174 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.012712 29565 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:11.013689 29564 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.013955 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.014350 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.015781 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.016314 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.016623 28292 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:11.021116 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.024963 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.028267 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.029163 28285 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:11.033366 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.033488 29474 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:11.034070 28954 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:11.037617 29529 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.039062 29529 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.039186 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:11.040172 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.040735 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.042285 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.041513 28363 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:11.043967 29425 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:11.044998 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.047224 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.050420 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.051301 28286 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:11.053002 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.055687 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.051224 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.039600 29529 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.037621 28330 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.061903 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.071637 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:11.073088 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.078927 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:11.079849 28286 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.090955 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.104367 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.089248 29529 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.109359 29529 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.113375 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.117790 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.118167 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.119786 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.119822 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.122884 28189 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 233) took 1380 ms. Trace:
I20260709 12:17:11.124476 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:11.089799 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 52353us
W20260709 12:17:11.092110 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 53094.2us
I20260709 12:17:11.101293 29425 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.128003 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.128621 29529 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.129042 29529 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.128050 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.134382 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:11.125984 28189 rpcz_store.cc:276] 0709 12:17:09.742744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:09.742948 (+   204us) service_pool.cc:224] Handling call
0709 12:17:11.122690 (+1379742us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:10.823106 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:10.823926 (+   820us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:10.823984 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:17:10.831772 (+  7788us) tablet.cc:662] Decoding operations
0709 12:17:10.839403 (+  7631us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:10.839548 (+   145us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:10.839593 (+    45us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:10.840059 (+   466us) tablet.cc:701] Row locks acquired
0709 12:17:10.840124 (+    65us) write_op.cc:260] PREPARE: finished
0709 12:17:10.840385 (+   261us) write_op.cc:270] Start()
0709 12:17:10.840578 (+   193us) write_op.cc:276] Timestamp: P: 1783599430840294 usec, L: 0
0709 12:17:10.840630 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 12:17:10.841875 (+  1245us) log.cc:844] Serialized 538 byte log entry
0709 12:17:10.989334 (+147459us) op_driver.cc:464] REPLICATION: finished
0709 12:17:11.086151 (+ 96817us) write_op.cc:301] APPLY: starting
0709 12:17:11.086319 (+   168us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:11.086464 (+   145us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:11.086591 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:11.086947 (+   356us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:11.087224 (+   277us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:11.087271 (+    47us) write_op.cc:312] APPLY: finished
0709 12:17:11.119947 (+ 32676us) log.cc:844] Serialized 23 byte log entry
0709 12:17:11.120186 (+   239us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:11.120290 (+   104us) write_op.cc:454] Released schema lock
0709 12:17:11.120361 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:11.120939 (+   578us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:11.120971 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":760721,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":96456,"apply.run_cpu_time_us":2736,"apply.run_wall_time_us":34737,"num_ops":1,"prepare.queue_time_us":62632,"prepare.run_cpu_time_us":26730,"prepare.run_wall_time_us":57120,"raft.queue_time_us":83787,"raft.run_cpu_time_us":8916,"raft.run_wall_time_us":39099,"replication_time_us":148428,"spinlock_wait_cycles":595968,"thread_start_us":139321,"threads_started":4,"wal-append.queue_time_us":59852}]]}
I20260709 12:17:11.136052 29518 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:11.138441 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.138871 29518 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.139346 29518 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:11.139755 29518 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.139511 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:11.141610 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.142746 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.143590 28332 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:11.132737 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 60396.1us
I20260709 12:17:11.119802 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.149288 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.150681 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.151569 28363 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:11.152894 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.146185 29529 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.160456 28439 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:11.169273 28286 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
W20260709 12:17:11.169979 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 50361.8us
I20260709 12:17:11.170295 29588 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.171435 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.185045 29474 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:11.152951 29565 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.184131 29588 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.183566 28330 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.162719 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.198204 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.147684 29425 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:11.192382 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.211165 29555 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.212901 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.234800 29604 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.245719 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.245952 29604 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.247704 29604 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:11.247989 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64555.5us
I20260709 12:17:11.248777 28989 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.231679 29565 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:11.249861 29598 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.250135 29565 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.250339 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:11.250754 29507 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:11.252802 29491 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.253261 29491 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:11.250974 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 53025.5us
W20260709 12:17:11.254419 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 101466us
I20260709 12:17:11.233212 29588 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:11.259565 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 68105.5us
I20260709 12:17:11.253628 29491 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.262758 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.266170 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.263826 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.268079 29425 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.269402 28432 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:11.270767 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:11.272166 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.273041 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:11.273898 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.274551 29564 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.274978 29564 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.275413 29564 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.276067 29471 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.276479 29471 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.276849 29471 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.278550 29423 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.279002 29423 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.279412 29423 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.258203 29598 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.275568 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.281489 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.286346 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.287043 28432 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:11.288619 29530 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.289075 29530 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.289450 29530 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.259879 29570 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.261237 29507 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.285125 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:11.288283 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.298506 28438 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:11.283011 29598 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:11.318012 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 127141us
I20260709 12:17:11.320425 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:11.288354 29529 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.331207 29570 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.332283 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.332449 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:11.333709 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:11.333016 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48001.7us
I20260709 12:17:11.298924 29507 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.343184 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.364113 29480 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.364715 29480 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:11.365108 29480 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.368561 29606 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.369066 29606 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.369479 29606 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.352661 29507 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.374508 28440 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
W20260709 12:17:11.376699 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 69286.2us
W20260709 12:17:11.377935 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 105088us
I20260709 12:17:11.373929 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.380868 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.370002 29529 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.382723 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.381496 28432 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:11.382799 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:11.376516 29555 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.402472 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.399466 29604 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.405085 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:11.407821 29604 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.408080 29594 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.408247 29530 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.408653 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.409266 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:11.410226 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 77929.7us
I20260709 12:17:11.398768 29570 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.411428 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.412925 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:11.418385 29570 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.419829 29530 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.420142 29565 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:11.430557 29565 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.423079 29480 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.432271 29518 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.433336 29480 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.433952 29608 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.436290 29608 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.436781 29608 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:11.425563 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 48204us
I20260709 12:17:11.438913 29474 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.439405 29474 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.439814 29474 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.401940 29471 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:11.450131 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48277.4us
I20260709 12:17:11.452930 29594 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:11.453603 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44468.1us
W20260709 12:17:11.456158 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 193223us
I20260709 12:17:11.456892 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:11.458065 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:11.455719 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:11.450285 29565 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:11.461964 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.462127 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.455567 29594 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:11.471331 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.471848 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:11.472517 29590 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.463229 29529 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.473048 29529 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.473333 29590 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.473513 29529 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.473790 29590 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:11.474231 29590 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.474632 29480 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.475492 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:11.478258 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.479313 29508 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:11.479948 29508 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.471213 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.478386 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:11.467520 29594 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.482131 28285 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.478104 29565 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.482298 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.483513 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.483865 29518 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.486953 29508 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.487936 28432 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:11.491636 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.493095 29570 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.493530 29570 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.493942 29570 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.476593 29480 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.498584 29480 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.504683 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:11.505498 28291 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.507768 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:11.499718 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:11.497586 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.521826 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.522266 29565 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:11.523792 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.524003 29607 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.524369 29518 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.526916 29518 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.527345 29518 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.524825 29508 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:11.508133 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.526355 29607 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.530777 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.534425 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.536432 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.537557 29613 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.538036 29613 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.538421 29613 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.539489 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.540576 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.541378 28436 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:11.541604 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.542382 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.535202 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:11.531936 29607 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.548579 29615 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:17:11.557768 29596 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.558317 29596 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:11.558784 29596 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.559167 29612 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.562088 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.568545 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.576488 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.562124 29471 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.590421 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.591253 29518 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.592931 29612 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:11.597978 29518 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:11.598676 28257 leader_election.cc:336] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Timed out: RequestConsensusVote RPC to 127.26.123.3:33721 timed out after 1.855s (SENT)
I20260709 12:17:11.599221 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:11.603690 28285 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:11.611552 28291 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:11.613628 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:11.597862 29471 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.598645 29518 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:11.621693 29602 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:11.622522 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 139173us
I20260709 12:17:11.603731 29564 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.618993 29555 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.629916 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.618877 29471 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.631585 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.635304 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:11.627753 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.640671 29564 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.636554 29555 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:11.642221 29614 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:11.635428 29471 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.646998 29613 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.604614 29594 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.649462 29480 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.666941 29613 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.659248 29529 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.668890 29529 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.659651 29555 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.662727 29602 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.664819 29425 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:11.679600 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.669704 29480 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:11.691012 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 48798.5us
I20260709 12:17:11.691638 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:11.650548 29612 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.670202 29425 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:11.633718 29518 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.661546 29607 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.671795 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:11.653236 29614 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.696656 29471 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.697014 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.730654 29474 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.734063 29588 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.734577 29474 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.735359 28435 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:11.736383 29588 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.737067 29518 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:11.741168 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.681986 29598 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.742375 29590 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.743800 29598 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.744511 29590 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.744809 29612 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:11.698016 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.700115 29606 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.701552 29594 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.636749 29507 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.711539 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:11.714232 29425 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.678577 29491 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.645615 29608 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.716605 29596 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.784215 29596 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:11.718366 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.785950 29588 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.786180 28287 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:11.786455 29588 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:11.719175 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.719372 29607 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.720027 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.787384 28438 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:11.787554 29588 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.788355 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:11.719373 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:11.670066 29602 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:11.790503 29602 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.723816 29423 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.742733 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:11.792415 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 61710.3us
I20260709 12:17:11.769692 28991 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:11.769665 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 71739.5us
I20260709 12:17:11.769565 29529 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:17:11.773118 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:11.774451 28363 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:11.779094 28439 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.732123 29614 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.745973 29612 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:11.731179 29570 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.780503 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.780663 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.781203 29425 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.804656 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.805384 28432 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.781495 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.781962 28358 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:11.786670 29491 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.787020 28359 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.755092 28285 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.792483 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:17:11.794122 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 171744us
I20260709 12:17:11.798204 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:11.803432 28440 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.808627 28439 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.809147 28436 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:11.814320 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.815891 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:11.816771 29491 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:11.817303 29491 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:11.818392 29529 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:11.818887 29529 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:11.818295 29491 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.819362 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:11.819746 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.820353 28438 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:11.821794 29608 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:11.823144 28330 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:11.819135 29619 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.814374 28434 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:11.831961 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:11.833770 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:11.840507 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:11.840759 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:11.841861 29518 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:11.842136 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.829107 29598 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.843415 29474 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.844043 29474 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:11.844362 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:11.845261 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:11.826198 29590 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:11.849225 29594 raft_consensus.cc:993] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:11.849526 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:11.850098 29507 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:11.828588 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 55631.5us
I20260709 12:17:11.851714 29423 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.852063 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:11.853120 29471 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.795521 29596 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:17:11.859282 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:11.865865 29508 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:11.865804 29423 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:11.867624 29507 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:11.868508 29423 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.868911 29423 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:11.869359 29507 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:11.870078 29507 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:11.869890 29423 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.871712 29423 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.873287 29590 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:11.874449 29590 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.874825 29590 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:11.875021 29628 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.875913 29590 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.877223 29590 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.887467 29629 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:17:11.826828 29529 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.826260 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.858093 29594 raft_consensus.cc:1081] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:11.858318 28285 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
W20260709 12:17:11.906924 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 47724.4us
W20260709 12:17:11.917086 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 82910us
I20260709 12:17:11.922071 29614 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.923518 28292 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:11.924669 29614 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:11.926021 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:11.925715 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:11.927914 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.925709 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.929337 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:11.931815 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 99772us
I20260709 12:17:11.934410 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:11.937063 28432 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:11.939457 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:11.939527 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:11.940433 28283 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.942834 29588 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:11.943996 28435 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:11.945082 29588 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:11.900928 29508 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:11.950357 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:11.961225 28283 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:11.896870 29615 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:17:11.963735 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:12.014688 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51033.6us
I20260709 12:17:11.893953 29617 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:17:11.945834 29555 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:17:12.020493 28436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:12.030889 29423 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:12.031167 29631 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:12.033994 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 176302us
I20260709 12:17:12.033504 28436 raft_consensus.cc:606] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:12.056509 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.032714 29604 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.055552 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:12.064845 28987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46950
I20260709 12:17:12.065587 28987 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.066080 28987 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.068878 28438 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:12.048084 29631 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:12.057958 29606 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:12.072927 29631 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.055147 29423 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:12.082800 28438 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:12.086721 28441 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.056622 29602 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.083313 29570 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:12.088130 28436 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.089427 29423 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.078111 28433 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.082253 29604 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:12.091198 29633 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.093505 28255 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:12.095537 29604 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.097668 29602 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:12.091655 29631 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:12.098950 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.099565 29615 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:12.107106 29615 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:12.108114 29615 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.111223 29602 raft_consensus.cc:2764] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:17:12.109498 29615 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:12.113463 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:12.121438 29607 raft_consensus.cc:1064] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:12.121901 28367 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:12.125972 29633 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.128931 29594 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:12.129585 29594 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.133595 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:12.126528 29633 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:12.098874 29529 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.126762 29471 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.143694 29529 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:12.144886 29633 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:12.147607 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 108597us
I20260709 12:17:12.139704 28987 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.154633 28991 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:12.157929 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.158548 28987 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:12.158960 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.159171 28433 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:12.159719 28436 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
W20260709 12:17:12.161010 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 62219.9us
I20260709 12:17:12.177331 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:12.177655 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:12.182773 28367 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.184827 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:12.183384 28367 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:12.193540 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.195016 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.195930 29565 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:12.196561 29570 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.196957 29480 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.197448 29617 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:17:12.196748 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.190403 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:12.202188 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.198187 28367 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.221859 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.233144 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:12.235236 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.236258 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:12.233808 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.233144 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.233470 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:12.191881 29619 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:12.238095 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:12.313112 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.313830 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.318341 28440 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:12.222841 29588 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.322757 29588 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:12.323583 29588 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.330758 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:12.332433 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 142117us
I20260709 12:17:12.238970 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:12.334429 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:12.336236 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:12.344027 28441 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.346238 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.348887 29590 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.350643 29619 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:12.352622 29633 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.353166 29607 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.355150 29619 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.355228 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.369468 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:12.335517 29644 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.380968 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:12.382609 28437 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:12.374290 29624 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.385352 28285 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.390806 29630 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:12.391366 29630 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:12.392344 29630 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.396490 28367 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.375211 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:12.357681 29474 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.376981 29643 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:12.344311 29594 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:12.384142 29618 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.360325 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.363724 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.363358 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:12.363044 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:12.418416 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 49037.1us
I20260709 12:17:12.448863 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:12.449900 28363 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:12.456168 28437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:12.459441 28358 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.460616 28437 raft_consensus.cc:606] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:12.364106 29596 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.364585 29636 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.402070 29630 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:12.359393 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:12.467221 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 125965us
I20260709 12:17:12.364077 29639 raft_consensus.cc:2764] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 12:17:12.483587 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.475318 28366 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.358590 29633 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:12.478834 28441 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
W20260709 12:17:12.480566 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 117305us
I20260709 12:17:12.486366 28439 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.464641 28987 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:12.457576 29507 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.429191 29598 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:12.457885 29618 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:12.489146 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 170477us
I20260709 12:17:12.476184 29596 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:12.491755 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 294982us
I20260709 12:17:12.479768 29474 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.480939 29636 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:12.459077 29565 consensus_queue.cc:1243] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer f526369c0d0743b192be6e205540a059 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:12.491869 29631 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:17:12.489264 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 114182us
W20260709 12:17:12.517874 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.518303 29627 raft_consensus.cc:2764] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:17:12.519088 29626 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.512873 29598 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.522107 28984 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.522423 29647 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:12.511562 28361 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.523258 29598 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:12.523059 29647 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.514401 29619 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:17:12.525307 29647 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:12.503933 29507 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:12.504267 29636 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.496429 29606 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:17:12.527523 29631 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:17:12.556097 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:12.553079 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 53617.8us
W20260709 12:17:12.585376 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:12.586315 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.621655 29654 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.118s
W20260709 12:17:12.622802 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.622525 29631 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.623319 28436 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:12.624493 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 66291.1us
W20260709 12:17:12.624886 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 110856us
I20260709 12:17:12.631234 28440 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:12.628827 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.722571 28436 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:12.551841 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.725934 28292 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
W20260709 12:17:12.725963 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 97062.7us
I20260709 12:17:12.727485 28292 raft_consensus.cc:606] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.728519 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:12.731344 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 106161us
I20260709 12:17:12.732125 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.731870 28437 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:12.737928 29491 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.744839 28257 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.748096 28367 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:12.730304 29650 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.180s
I20260709 12:17:12.753506 28360 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.753940 28360 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:12.735248 28283 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.554606 29618 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.562677 29596 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.748833 29629 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.761001 29646 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.729429 29588 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.763895 29588 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:12.748962 29615 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.636s	user 0.014s	sys 0.004s
I20260709 12:17:12.764797 29530 raft_consensus.cc:993] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
W20260709 12:17:12.764914 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 222745us
I20260709 12:17:12.765799 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.735569 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:12.769289 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.778478 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:12.779482 28365 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.778478 28437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:12.780445 28437 raft_consensus.cc:606] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:12.780611 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.782124 29608 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.782541 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.782688 29491 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.784428 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.784718 29570 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:12.785395 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.786191 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:12.787391 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.790401 29626 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.791077 29623 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.790688 28360 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.791301 29626 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.791508 29647 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.792061 28360 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.792357 29623 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.792819 29626 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.790606 29622 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.793203 29626 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:12.793625 29622 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:12.794080 29622 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.794227 29626 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.798580 28365 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:12.735884 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:12.807682 28437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:12.808365 28437 raft_consensus.cc:606] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:12.801687 29623 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.792122 29647 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:12.803201 29615 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.790411 29508 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.822943 29508 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:12.806130 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:12.813756 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:12.824246 28991 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:12.824028 29508 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.815476 29644 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.825240 29644 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:12.817126 29425 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:12.818010 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52343.4us
I20260709 12:17:12.771008 29588 raft_consensus.cc:1081] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:17:12.812304 29626 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.820627 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:12.831794 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.823105 29647 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.825331 29508 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.815979 28359 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:12.828699 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:17:12.820817 29659 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.002s
I20260709 12:17:12.826828 29425 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:12.839561 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:12.836513 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.822314 29623 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:12.843299 29614 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.843743 29643 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.844735 29425 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.846184 29646 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:17:12.857256 28360 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.848618 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.849138 29588 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.849511 29618 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.836169 29633 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.848117 29647 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.858182 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.860792 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.861079 29661 raft_consensus.cc:993] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:17:12.860322 29623 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.863349 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.861721 29661 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.869168 29639 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.851732 29643 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:12.874539 28986 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
W20260709 12:17:12.863912 29423 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:17:12.881487 29660 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.867297 29633 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:12.864490 29644 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.881053 28437 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.882154 28434 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:12.865505 29614 raft_consensus.cc:1081] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:17:12.885888 28433 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:12.887126 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.884940 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:12.888042 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:12.889235 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.889357 29643 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.890692 29643 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.894347 29623 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.874114 29425 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:12.893326 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.889119 29588 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.882493 28986 pending_rounds.cc:85] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 4
I20260709 12:17:12.895653 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:12.898229 29661 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.882643 29630 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.899443 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.900758 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.906215 29633 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.909494 28986 pending_rounds.cc:107] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:12.914561 28986 raft_consensus.cc:2889] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:12.906991 29596 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.915515 29423 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.915938 29423 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:12.916302 29423 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:12.916647 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.908463 29644 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.910820 28332 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:12.930428 29530 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:12.930970 29588 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.931993 29530 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.932367 29646 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.922262 29507 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.933569 29507 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:12.934074 29507 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.934681 29630 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.934719 29615 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:12.935456 29615 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.936255 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:12.939369 29630 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.940270 29615 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:12.940534 29630 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:12.931792 29622 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:12.934270 29530 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:12.941581 29630 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.943591 29622 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:12.946152 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 45460.1us
I20260709 12:17:12.946262 29604 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:12.947089 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:12.948141 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.948504 29530 raft_consensus.cc:2764] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:12.949509 29530 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:12.916103 29636 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:12.952229 29636 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:12.952368 29423 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:12.952858 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59970.8us
I20260709 12:17:12.927407 28257 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:12.961489 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:12.963228 29647 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:12.966653 29647 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:12.972380 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45135.1us
I20260709 12:17:12.973204 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.974639 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:12.979681 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.978561 29629 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:12.985404 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:12.986250 28359 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:12.986887 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:12.987708 28985 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:17:12.988891 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:12.990267 28360 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:12.989836 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:12.993755 29651 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.994242 29651 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:12.994235 28364 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:12.994706 29651 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.981810 29631 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:17:12.981705 29491 raft_consensus.cc:2764] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:12.994321 29615 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:12.997013 29615 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:12.998210 28990 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:13.000298 29646 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:13.005559 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:13.006744 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:12.988447 29661 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.020424 29604 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:13.024008 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:13.009948 29596 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:17:12.997396 29615 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:13.032727 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:13.034278 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.017132 28186 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:13.001564 29646 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:17:13.043915 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.038615 29660 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.045045 29491 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.045424 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:13.046131 28433 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.047201 29495 tablet_copy_client.cc:538] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:13.049943 28186 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:13.052302 28186 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:13.066229 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:13.071049 28257 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.043617 29423 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.056572 28984 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:17:13.091377 28984 pending_rounds.cc:85] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 4
I20260709 12:17:13.091732 28984 pending_rounds.cc:107] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:13.091982 28984 raft_consensus.cc:2889] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:13.069770 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:13.093151 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.039911 29644 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:17:13.094321 29660 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:13.095491 29491 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:13.101166 29631 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:13.113646 29666 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:17:13.107395 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51653.3us
I20260709 12:17:13.121255 28440 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.103540 29471 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:13.121974 29471 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:13.123119 29614 raft_consensus.cc:1064] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
W20260709 12:17:13.123554 29614 raft_consensus.cc:1068] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Unable to promote non-voter f526369c0d0743b192be6e205540a059: Service unavailable: leader transfer in progress
I20260709 12:17:13.124270 29471 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:13.104888 29639 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.125114 29602 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.125499 29602 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:13.126358 29602 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.105304 29491 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:13.121640 28440 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:13.127954 29491 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.128793 28440 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:13.131126 28440 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.122113 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:13.101202 29423 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
W20260709 12:17:13.111951 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 45812.9us
I20260709 12:17:13.116744 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.142584 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:17:13.145469 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.115631 29651 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:13.161131 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44473.3us
I20260709 12:17:13.162014 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.164789 28187 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:13.171097 29614 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.170979 28187 catalog_manager.cc:5697] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:13.177898 29596 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.178449 29596 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.180543 29596 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.180631 28433 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:13.181280 29614 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.169695 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:13.160084 29530 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.222159 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:13.228101 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 58492.2us
W20260709 12:17:13.225412 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.224277 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:13.229379 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.228165 29608 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.231930 29647 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:13.236109 29647 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:13.238113 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 76185.8us
I20260709 12:17:13.237035 29647 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.238976 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.242491 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:13.226846 29614 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.238209 28433 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:13.278551 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.271853 29507 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.283630 29602 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.283799 28364 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.284924 29507 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.276979 29661 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.287391 29570 raft_consensus.cc:993] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:13.287951 29570 raft_consensus.cc:1081] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:13.296029 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:13.255801 29588 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:13.313005 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.325865 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:13.327921 28439 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.303179 29425 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:13.328958 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50517.5us
I20260709 12:17:13.329783 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.332655 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:13.337477 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:13.332618 28440 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:13.305441 29639 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.329209 29425 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.347452 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:13.352469 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:13.280740 29615 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:13.374248 28360 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
W20260709 12:17:13.379207 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:13.379797 29674 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:17:13.380173 29652 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:13.305326 29622 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:17:13.381968 28363 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:13.391335 28363 pending_rounds.cc:85] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 1
I20260709 12:17:13.391582 28363 pending_rounds.cc:107] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:13.391855 28363 raft_consensus.cc:2889] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:13.382280 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.382951 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:13.384745 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.355948 29632 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.398527 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:13.399384 28358 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:13.401239 29622 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.401665 29622 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:13.402192 29615 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.403012 28363 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.405115 28363 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.405841 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:13.405998 28985 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46934
I20260709 12:17:13.406584 28985 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:13.406965 28985 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.407482 28365 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:13.407958 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.387655 29651 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.415741 29425 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.417160 29608 raft_consensus.cc:993] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:13.388543 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:13.424259 28364 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:13.390146 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.425407 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.421259 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:13.385787 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:13.421924 28363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:17:13.426694 28363 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:13.427125 28363 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.391606 29643 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:13.447903 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:13.448662 28440 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:13.450531 28364 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.452718 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:13.460011 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:13.422484 29612 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.466763 28362 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.433380 29644 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.417218 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:13.436507 29652 raft_consensus.cc:1081] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:17:13.478965 28436 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:13.470994 28332 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:13.476258 28367 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.445351 29643 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.486304 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:13.487008 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:13.446162 29651 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:13.445279 29570 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.488895 28283 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:13.490633 28291 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.500536 28287 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:13.491801 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 268604us
W20260709 12:17:13.513767 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.478255 29649 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.522317 28441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:17:13.526825 28441 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:13.527269 28441 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.484681 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:13.535286 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.491411 29643 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.535589 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:13.535805 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:13.532392 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.539978 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:13.541901 28286 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:13.498405 29425 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:13.542539 29425 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:13.542945 29425 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.546025 28441 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:17:13.546469 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.546543 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.547291 28286 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.547513 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.478021 29588 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
W20260709 12:17:13.559427 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.567890 29495 tablet_bootstrap.cc:492] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:17:13.478756 29612 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:13.550633 29508 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.570209 29508 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.571568 29612 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:13.572433 29508 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.573768 29530 consensus_queue.cc:1059] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:17:13.575238 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.575927 28289 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.576047 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.551715 28290 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.577497 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.537997 29649 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.578203 28290 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.579939 29649 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.580142 29530 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.580518 29530 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:13.581401 29530 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.582952 28284 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.583441 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.584026 28288 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.585783 28290 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.586715 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.587335 28290 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.523223 29570 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:13.536254 29507 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.583942 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.597649 28284 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.575361 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.525846 29630 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:13.554986 29491 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:13.605420 28439 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:13.605510 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:13.606114 28435 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:13.607061 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 137108us
I20260709 12:17:13.608026 28366 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.609273 28438 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.549808 28441 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:13.593288 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.556236 29615 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:13.599784 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 52376.8us
W20260709 12:17:13.612150 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 157001us
I20260709 12:17:13.602072 28289 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.603978 28288 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.616780 28288 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.617812 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.615674 28363 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.679284 28288 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.598783 29508 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:13.682415 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:13.688639 28283 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.681577 28286 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.691114 28363 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.697492 28433 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.697615 29652 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:13.700438 28290 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
W20260709 12:17:13.611613 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 147219us
I20260709 12:17:13.600101 29507 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.690582 29508 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:13.611699 29630 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:13.692085 29530 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.698643 29652 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.698814 29649 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.795374 28361 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.693298 29588 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.094s
W20260709 12:17:13.819006 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.830681 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.831396 28283 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.831470 29507 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.832374 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.832813 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.832973 28283 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:13.834365 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.836616 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.835258 29615 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:13.837531 29615 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:13.839287 29588 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:13.839795 29615 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:13.834488 29614 consensus_meta.cc:287] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.580s	user 0.005s	sys 0.004s
I20260709 12:17:13.796871 29508 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.723997 28985 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:13.797559 29652 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:13.860062 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.857069 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.827385 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:13.857995 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.825125 29570 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:13.845727 29614 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:13.865782 28286 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:13.864477 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 162807us
W20260709 12:17:13.869122 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.870309 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:13.870786 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.839807 29588 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:13.880436 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.843495 29691 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:17:13.881779 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.884938 29630 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:13.828406 29646 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:13.886363 29652 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.887370 29652 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.885789 29491 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:13.891117 28288 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:13.896131 28291 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:13.899466 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:13.901419 29670 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.567s	user 0.000s	sys 0.005s
I20260709 12:17:13.905103 29588 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:13.909884 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.874025 28286 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.918968 28284 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:13.919219 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.920010 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:13.920344 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:13.917771 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47501.6us
I20260709 12:17:13.910358 29695 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:13.928622 28288 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:13.929808 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.931109 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.932286 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.933190 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.937374 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.938216 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.942989 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:13.944204 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:13.948114 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.948906 28284 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:13.953431 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:13.945855 29588 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.955036 29570 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:13.965858 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.966579 28286 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:13.969750 28288 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:13.950989 29688 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:13.971025 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:13.970973 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.973269 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.978251 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:13.951133 29491 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:17:13.979569 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.982540 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:13.985308 29632 raft_consensus.cc:1064] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:13.941839 29695 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:17:13.989126 28287 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:13.990324 28441 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:13.990525 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:13.995054 28288 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 2: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:13.997702 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:13.999162 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:13.999408 28441 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:13.999847 28288 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
W20260709 12:17:13.991851 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.985581 29680 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:14.003232 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:14.003449 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:13.964820 29508 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.009339 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:14.009764 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.010033 28437 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:14.010839 28332 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:14.014108 29594 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:14.014621 29594 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:14.015511 29594 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.018005 29696 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:14.018406 29696 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.020009 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.021082 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:14.021817 28363 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:14.019212 29696 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.023591 29696 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:13.996810 29631 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:14.027089 29491 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:17:13.996294 29632 raft_consensus.cc:1068] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Unable to promote non-voter f526369c0d0743b192be6e205540a059: Service unavailable: leader transfer in progress
I20260709 12:17:14.005087 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:13.978892 29697 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:14.027495 29491 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.011766 28288 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
W20260709 12:17:14.031893 29683 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:17:14.035316 29491 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.004870 28286 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:14.028666 28287 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.037751 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.041349 28363 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:14.032102 29680 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.033126 29697 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.045555 28286 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 12:17:14.043792 28287 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.045599 28284 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:14.046797 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:14.036218 28433 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.054603 28441 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:14.055898 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 102575us
I20260709 12:17:14.056396 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.057113 28288 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:14.057878 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:14.021162 29425 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:14.059695 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.063614 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:14.063918 28363 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:14.060143 28284 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.067508 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:14.072183 28332 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:14.073441 28991 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.074270 29507 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:14.078359 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:14.078534 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:14.079252 28365 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:14.059487 29491 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:14.086180 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.081110 29596 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:14.103767 29507 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:14.105346 29643 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.002s
I20260709 12:17:14.105748 28986 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46934
I20260709 12:17:14.106135 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.054580 29697 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:14.063625 29623 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:14.110227 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:14.119297 28284 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:14.123574 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:14.077915 29688 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.127306 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.131304 28284 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:14.108671 29596 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:14.127331 28986 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:14.134011 28989 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.135072 28988 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 3. Ops: []
I20260709 12:17:14.108954 29570 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.112274 29507 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:14.140437 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 77178.7us
I20260709 12:17:14.165318 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:14.168864 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:14.168994 29570 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.170131 28441 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:14.170492 28441 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.171499 28434 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:14.171963 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.177397 28292 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.182773 28367 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.186897 28361 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.186543 28441 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.187614 28441 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.141347 28986 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.122256 29643 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.192728 28365 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:14.187656 29702 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:17:14.179879 28362 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:14.197580 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.140199 29596 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.189109 29507 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:14.169255 29622 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:14.204380 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.140357 29623 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.221329 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.232555 28441 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:14.234591 28363 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.234622 29704 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:17:14.241330 29704 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:14.243252 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:14.244257 28985 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:14.244680 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.245317 28284 raft_consensus.cc:2376] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2. Re-sending same reply.
I20260709 12:17:14.246007 29639 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:14.246254 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.247478 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.247613 29705 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:14.248065 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.241703 29704 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.249140 29643 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:14.259224 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.260280 28290 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:14.238481 29688 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.232751 29530 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:17:14.179179 29680 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.255239 28284 raft_consensus.cc:2376] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2. Re-sending same reply.
W20260709 12:17:14.249398 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.231884 29633 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 12:17:14.262238 28404 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.031s (SENT) [suppressed 81 similar messages]
I20260709 12:17:14.252934 29596 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.238929 29622 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:14.267251 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.250245 29425 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:14.275532 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.269271 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.254364 29614 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
W20260709 12:17:14.282281 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 204000us
I20260709 12:17:14.283064 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.285614 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.288393 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.290963 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.292315 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.293365 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.295054 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.294106 28283 raft_consensus.cc:2376] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2. Re-sending same reply.
I20260709 12:17:14.298305 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.299335 28283 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.300348 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:14.301131 28441 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.306846 28434 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:14.311586 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:14.327728 29715 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:17:14.328449 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:14.329596 29710 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:17:14.330133 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.331717 28284 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.332628 28288 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.332886 28284 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.333570 28986 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.334118 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:14.335651 28290 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:14.336901 28290 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.340395 28287 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:14.342622 28437 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.342981 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:14.343118 28441 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:14.344826 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:14.347800 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:14.350167 28291 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:14.349303 29594 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:14.352104 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 289155us
I20260709 12:17:14.290139 29643 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:17:14.356081 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:14.356246 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 187410us
I20260709 12:17:14.358180 29623 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:14.360692 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:14.346325 29660 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:14.364123 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.359892 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:14.366375 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:14.369182 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.377123 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:14.377350 28291 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.371857 29615 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:14.375413 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:14.370882 28988 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:14.354735 29633 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:14.381884 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:14.377810 28362 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.294345 29704 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.385257 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:14.390532 29686 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:14.391847 28437 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:14.393709 28332 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:14.394001 28283 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:14.396522 29699 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:14.396893 28989 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.397389 29647 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:14.397974 29647 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:14.398736 29698 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:17:14.398892 29647 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.400682 29632 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:14.401103 29632 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:14.367935 29594 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:14.404381 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:14.404239 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:14.352082 29626 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:17:14.399040 29508 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:14.389896 29652 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:14.413923 28292 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
W20260709 12:17:14.414626 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.393100 29654 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:14.417800 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:14.401291 28433 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:14.420475 29652 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.421777 28332 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:14.421635 29652 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.422698 29649 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:14.423132 29649 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:14.403155 28986 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:14.427594 28291 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:14.429175 28362 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
W20260709 12:17:14.442543 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:14.443214 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.431989 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:14.418740 29704 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.423867 29632 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.402575 29594 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:14.466625 29711 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.108s	user 0.003s	sys 0.000s
I20260709 12:17:14.469776 29646 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:14.475642 28436 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.524725 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1193) took 1061 ms. Trace:
W20260709 12:17:14.482785 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.483889 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.502872 29660 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:14.406208 29631 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:17:14.422720 29425 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:14.511183 28437 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:14.497097 28189 catalog_manager.cc:5697] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:14.516232 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.522655 29693 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.522878 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:14.457552 29695 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:14.525105 28436 rpcz_store.cc:276] 0709 12:17:13.463357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.479755 (+ 16398us) service_pool.cc:224] Handling call
0709 12:17:14.475185 (+995430us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b58001d4698. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:14.475429 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:14.524424 (+ 48995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:14.524652 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12698880}
W20260709 12:17:14.543103 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 165921us
I20260709 12:17:14.542631 28437 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.542682 28432 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:14.523722 29596 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:14.524247 29646 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:14.566454 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:14.551491 28437 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.568818 28437 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.569144 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:14.572427 28432 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:14.571262 29705 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:17:14.575368 29696 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:14.575863 29696 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:14.576244 29696 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.520718 29649 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.543643 29695 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:14.546706 29423 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.583616 28437 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
W20260709 12:17:14.593101 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.584004 29632 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.560722 29507 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.622166 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1199) took 1071 ms. Trace:
I20260709 12:17:14.559033 29693 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.631508 29507 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:14.631834 29693 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:14.628105 28435 rpcz_store.cc:276] 0709 12:17:13.545433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.545558 (+   125us) service_pool.cc:224] Handling call
0709 12:17:13.605932 (+ 60374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:13.613237 (+  7305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:14.616410 (+1003173us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b54002e4ad0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:14.616519 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61799808}
I20260709 12:17:14.613678 29471 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:14.655961 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 62999.2us
I20260709 12:17:14.660634 28988 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.663806 28988 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.599520 29423 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.653540 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:14.672039 29629 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:14.672652 29629 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:14.673584 29629 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.655350 29695 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.619976 29646 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:14.693301 29646 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.703352 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:14.691640 29723 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
W20260709 12:17:14.725248 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 183958us
I20260709 12:17:14.705396 29423 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:14.728555 29697 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.596s	user 0.009s	sys 0.003s
I20260709 12:17:14.727039 29508 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:14.739037 29508 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:14.739492 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1311) took 1165 ms. Trace:
I20260709 12:17:14.739885 28292 rpcz_store.cc:276] 0709 12:17:13.574392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.606946 (+ 32554us) service_pool.cc:224] Handling call
0709 12:17:14.739229 (+1132283us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540018f454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:14.739388 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284645760}
I20260709 12:17:14.741012 28285 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:14.741423 29508 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:14.744829 28438 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:14.737918 29697 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:14.782477 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.794662 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:14.797549 29508 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:14.771580 29680 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:14.800709 29425 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:14.825404 29425 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:14.827883 29703 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.355s	user 0.005s	sys 0.000s
I20260709 12:17:14.831241 29696 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.835687 29696 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:14.836323 29649 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.839426 29508 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:14.840723 29647 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:14.841104 29647 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:14.842074 29647 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.843377 29647 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:14.875461 28362 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.875895 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:14.827282 29680 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:14.879819 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 448210us
I20260709 12:17:14.874009 29423 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.904350 28362 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.907900 28363 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:14.909358 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:14.912386 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.916831 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:14.917098 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:14.918120 28365 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:14.918447 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:14.919811 28330 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:14.916838 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:14.929908 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.934623 28366 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:14.940470 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:14.924893 29507 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:14.924996 29660 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:14.949270 28330 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:14.936172 29691 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:14.933597 29714 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:14.956030 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:14.956256 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.956182 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:14.956686 28437 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:14.956905 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:14.958360 28332 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:14.955358 28330 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:14.964457 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:14.965279 28434 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:14.968314 29694 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:14.970404 29694 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:14.970880 29694 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:14.971721 29694 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:14.921255 29423 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:14.980660 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 58843.7us
I20260709 12:17:14.950697 29507 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:14.952031 29596 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:14.960464 29697 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:15.013669 28437 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
W20260709 12:17:15.025285 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 71398.6us
I20260709 12:17:14.975858 29691 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:15.021389 29741 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.021929 29714 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:14.954556 29696 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:17:15.043032 28363 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:15.012969 29507 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:15.039799 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.036304 29596 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:15.046665 28363 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:15.034214 29721 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.402s	user 0.003s	sys 0.000s
I20260709 12:17:15.012648 29530 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:17:15.036685 29628 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:15.041639 29697 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.061317 29639 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:15.036505 29686 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.040977 29735 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:14.960992 29660 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.054237 29714 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:15.085991 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 127701us
I20260709 12:17:15.093573 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:15.094345 28363 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:15.096331 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 141055us
I20260709 12:17:15.097183 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:15.099929 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:15.100602 29749 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:15.101177 29749 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.103421 29749 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:15.103870 29739 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:17:15.104701 29749 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:15.105221 28363 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:15.076376 29680 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:15.076586 29608 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
W20260709 12:17:15.107514 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.074664 29628 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:15.091491 29735 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.123165 28330 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:15.123803 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:15.075242 29697 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:15.124550 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.094492 29706 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:15.088047 29733 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:15.126354 28290 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 2: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:15.122063 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:15.127972 28285 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:15.128521 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:15.128463 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:15.129292 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:15.146915 28364 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 2: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:15.109542 29643 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:15.134498 29632 consensus_meta.cc:287] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.513s	user 0.012s	sys 0.000s
W20260709 12:17:15.156414 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.166457 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:15.154055 29706 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:15.172155 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43712.4us
I20260709 12:17:15.173247 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:15.095619 29596 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.087743 29639 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.145949 29680 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:15.105278 29749 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.150640 29628 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.188143 29752 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:15.156124 29741 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:15.196599 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:15.197482 28364 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:15.174201 29687 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.199575 29695 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.202664 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:15.179098 29732 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:15.206058 28285 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:15.205646 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.076141 29691 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:15.190148 29704 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:15.190557 29632 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:15.195292 29608 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:15.202754 29530 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
W20260709 12:17:15.214969 29744 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:17:15.215785 29751 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:15.216104 29743 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:17:15.189563 29706 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:15.205701 29495 tablet_bootstrap.cc:492] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:17:15.212363 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:15.193568 29425 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:15.216413 29740 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:15.190925 29680 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.208873 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:15.228018 29660 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:15.229140 29745 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:17:15.204512 29491 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:17:15.174897 29714 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.241397 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:15.246915 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:15.222761 29732 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:15.219431 29704 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.253243 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.218530 29752 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:15.259570 29738 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:17:15.223994 29632 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.284312 29632 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.225139 29608 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:15.286556 28441 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:15.289518 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.251935 29740 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.289618 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:15.294422 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:15.296509 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 108093us
I20260709 12:17:15.277390 29495 tablet_bootstrap.cc:492] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:17:15.278258 29687 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:15.249456 29594 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.249519 29725 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:15.279109 29680 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:15.279599 29471 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:15.280180 29697 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.272948 29751 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:15.297164 28437 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:15.252954 29423 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:15.298143 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.259004 29660 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:15.315059 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.323534 29760 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:15.266239 28285 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:15.286873 29608 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:15.298017 29740 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:15.292877 29632 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:15.315024 29706 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.339962 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:15.340857 28364 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:15.324784 29471 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:15.342716 29471 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:15.344087 29425 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.301903 29757 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:15.348978 28954 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:15.327499 29735 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:15.351962 29632 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.357868 29632 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.358412 29632 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:15.347571 29760 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.358878 29632 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.359380 28364 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:15.349684 29495 ts_tablet_manager.cc:1403] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.782s	user 0.028s	sys 0.015s
W20260709 12:17:15.357903 28257 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.008s	sys 0.013s
W20260709 12:17:15.365540 28257 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.008s	sys 0.014s
W20260709 12:17:15.365871 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 970314us
I20260709 12:17:15.366618 28257 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:15.369150 28363 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:15.319522 29741 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:17:15.298249 29732 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:15.355540 29639 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.360999 29615 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:15.351266 29751 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:15.369109 29425 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.335763 29660 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.333835 29725 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.370744 29757 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:15.353300 29706 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:15.431627 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.384043 29615 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:15.385713 29704 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:15.426785 29425 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.446280 29757 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:15.477123 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.482829 29766 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:15.397063 29761 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:17:15.441054 29615 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.527412 29760 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.528189 29760 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.528723 29760 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.528985 29629 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:15.529588 29629 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:15.531522 29725 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.531826 29629 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.499529 29705 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:15.519347 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 152839us
I20260709 12:17:15.543572 29725 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.544503 29495 raft_consensus.cc:359] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 2 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:17:15.557107 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.558615 29660 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.559520 29632 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:15.559866 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.560840 29760 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.537111 29425 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:15.563267 29649 consensus_meta.cc:287] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.726s	user 0.013s	sys 0.000s
I20260709 12:17:15.559338 29706 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:15.570158 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:15.575333 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:15.592227 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45605.6us
W20260709 12:17:15.635560 29771 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:17:15.637285 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:15.643428 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.577806 29495 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:17:15.555907 29705 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.663633 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.644774 29741 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:15.572573 29507 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.678570 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.679502 28285 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:15.680235 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.680552 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:15.681136 28287 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:15.681263 28285 raft_consensus.cc:2376] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3. Re-sending same reply.
I20260709 12:17:15.645139 29632 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:17:15.687709 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:15.687616 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:15.686609 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:15.690018 28286 raft_consensus.cc:2376] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2. Re-sending same reply.
I20260709 12:17:15.679051 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:15.690891 28290 raft_consensus.cc:2376] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2. Re-sending same reply.
I20260709 12:17:15.690321 28285 raft_consensus.cc:2376] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2. Re-sending same reply.
I20260709 12:17:15.692806 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:15.693547 28289 raft_consensus.cc:2376] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3. Re-sending same reply.
I20260709 12:17:15.695039 28365 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:15.534200 29751 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.711580 28438 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:15.713773 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:15.714656 28285 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:15.740437 29615 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:15.750022 28330 leader_election.cc:336] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.912s (SENT)
I20260709 12:17:15.750696 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:15.768112 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 404446us
I20260709 12:17:15.758141 28290 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:15.712654 29649 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:15.788623 29714 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:15.814494 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.677806 29741 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:15.695842 29740 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.811434 29695 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:15.685353 29495 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:15.829564 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.830610 28289 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:15.833148 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.834781 28290 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:15.837643 28290 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:15.839859 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.842149 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.845152 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.842665 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.843446 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:15.823334 29649 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:15.828212 29695 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:15.848611 29615 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.925326 28283 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:15.968650 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:15.973803 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 130477us
I20260709 12:17:15.975971 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:15.944825 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.943931 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.977654 28434 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:15.941251 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:15.978207 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.909997 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:15.976207 28330 leader_election.cc:336] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.028s (SENT)
I20260709 12:17:15.917685 28285 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:15.979794 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:15.945323 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.969635 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.971599 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:15.953577 29784 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.133s	user 0.002s	sys 0.000s
I20260709 12:17:15.989231 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.990334 28440 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:15.849321 29706 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:15.842357 29723 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:15.990376 29588 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:15.928004 28292 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:16.028558 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.980211 29752 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.035864 28440 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:15.986891 29508 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:16.037766 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:15.953962 29775 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 12:17:15.990489 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:15.994504 29507 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:15.994145 29741 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:16.072713 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.040916 29588 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.034363 29706 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.088193 28285 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.037454 29530 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.037858 29660 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:16.033181 29705 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:16.050552 29725 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.107826 28285 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
W20260709 12:17:16.111176 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.067102 29740 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.034765 29723 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.092814 29777 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:17:16.118640 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 137397us
I20260709 12:17:16.090801 28433 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
W20260709 12:17:16.127794 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 186658us
I20260709 12:17:16.126468 28437 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:16.109966 29705 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.123028 29699 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:16.123695 29777 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:16.143083 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.123170 29695 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:16.127467 29779 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.336s	user 0.000s	sys 0.001s
I20260709 12:17:16.124691 29660 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:16.107396 29530 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.112020 29725 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.130007 29615 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:16.164845 28330 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.957s (SENT)
I20260709 12:17:16.167366 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:16.133482 29787 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:16.155454 29781 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:17:16.131331 29507 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:17:16.150391 29789 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:16.154464 29508 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.151506 29699 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.136339 29752 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:17:16.177343 29714 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:16.144209 29740 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
W20260709 12:17:16.193300 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.197943 28954 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:16.170761 29788 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:16.202943 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 74436.7us
W20260709 12:17:16.186360 29769 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 12:17:16.145753 29705 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:16.170423 29629 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:17:16.215044 29725 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.177543 29615 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:16.220314 29780 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:17:16.193967 29508 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.223510 29508 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.228996 29629 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:16.229825 29629 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:16.206785 29777 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.230909 29530 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:16.191272 29789 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.206590 29741 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:16.217931 29770 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:17:16.231742 29588 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:16.179867 29660 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:16.210881 29790 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.336790 29790 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.340173 29790 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.182085 29787 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.210853 29680 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:16.243175 29785 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:17:16.216724 29697 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:16.342540 29697 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.343525 29787 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:16.236518 29778 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:17:16.243672 29652 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.344663 29652 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:16.345830 29660 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.345707 29652 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.346894 29652 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.208701 29788 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:16.250415 29783 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:17:16.250936 29495 ts_tablet_manager.cc:1434] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.876s	user 0.032s	sys 0.025s
W20260709 12:17:16.348999 29495 ts_tablet_manager.cc:1467] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Tablet startup took 983ms
W20260709 12:17:16.252188 29786 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:17:16.250471 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.244244 29706 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:16.303177 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.354874 28358 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:16.303833 29530 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.245926 29741 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:16.309270 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 111437us
I20260709 12:17:16.315843 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.318511 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:16.324738 28330 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.077s (SENT)
I20260709 12:17:16.359078 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:16.326684 28360 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:16.327728 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:16.330247 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 125101us
I20260709 12:17:16.250684 29629 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.350183 28364 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:16.358448 28366 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:16.363013 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.363869 28366 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:16.362394 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.341236 29680 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.369223 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.370384 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.371387 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.372540 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:16.373952 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49258.4us
I20260709 12:17:16.368384 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.364593 28360 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:16.367626 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.376668 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.376828 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.377369 28358 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:16.368233 29789 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.379179 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.380293 28366 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:16.377511 28360 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:16.383912 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.382110 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.385763 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:16.388671 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.349689 29788 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.354703 29723 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:16.384430 28362 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:16.398655 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1269) took 1049 ms. Trace:
I20260709 12:17:16.393105 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.399195 28439 rpcz_store.cc:276] 0709 12:17:15.349028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.349171 (+   143us) service_pool.cc:224] Handling call
0709 12:17:15.349543 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:15.351129 (+  1586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.351158 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:15.354731 (+  3573us) log.cc:844] Serialized 26 byte log entry
0709 12:17:15.771897 (+417166us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:15.798191 (+ 26294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.798264 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:16.398371 (+600107us) raft_consensus.cc:1685] finished
0709 12:17:16.398395 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:16.398571 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1138,"raft.queue_time_us":7418,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":254281,"threads_started":1,"wal-append.queue_time_us":479047}
W20260709 12:17:16.349406 29495 ts_tablet_manager.cc:1469] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Trace:
0709 12:17:05.307686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:05.307848 (+   162us) service_pool.cc:224] Handling call
0709 12:17:05.401016 (+ 93168us) ts_tablet_manager.cc:934] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
0709 12:17:09.876846 (+4475830us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:17:10.573126 (+696280us) tablet_metadata.cc:665] Metadata flushed
0709 12:17:10.573260 (+   134us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:17:10.702327 (+129067us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:17:10.702703 (+   376us) tablet_replica.cc:206] Creating consensus instance
0709 12:17:10.703147 (+   444us) ts_tablet_manager.cc:989] Replica registered.
0709 12:17:10.703225 (+    78us) inbound_call.cc:177] Queueing success response
0709 12:17:13.567425 (+2864200us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:17:15.374455 (+1807030us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:17:15.541722 (+167267us) tablet_replica.cc:261] Starting instrumentation
0709 12:17:15.542440 (+   718us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":22962,"mutex_wait_us":23933,"raft.queue_time_us":108482,"spinlock_wait_cycles":16362752,"tablet-copy.queue_time_us":91533,"tablet-download-pool.queue_time_us":4330,"tablet-download-pool.run_cpu_time_us":1995,"tablet-download-pool.run_wall_time_us":2172520,"thread_start_us":485169,"threads_started":3}
I20260709 12:17:16.379910 29725 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.368983 29530 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.383109 29705 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:16.416961 29471 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.417922 29660 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:16.421207 29734 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.421564 29660 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.425789 29660 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.429476 29643 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:16.402765 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44366.2us
I20260709 12:17:16.441533 29680 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:16.445083 29782 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.404s	user 0.002s	sys 0.000s
I20260709 12:17:16.448954 29804 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:16.449769 29804 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.456453 29680 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:16.413793 29725 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.462683 29754 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:16.463366 29680 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.434674 29699 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:16.466006 29774 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.468758 29530 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:16.474272 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.475324 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.488480 28437 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:16.444130 29734 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:16.491447 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.492363 28438 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 2: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:16.495797 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-3ee4696ba90a447fa07c4d09407217f6 received from {username='slave'} at 127.0.0.1:53280
I20260709 12:17:16.496040 29804 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.496425 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-3ee4696ba90a447fa07c4d09407217f6 on tablet 3ee4696ba90a447fa07c4d09407217f6 with peer f526369c0d0743b192be6e205540a059
I20260709 12:17:16.516639 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:16.486792 29787 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:16.520045 28359 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1342) took 1217 ms (client timeout 1592 ms). Trace:
W20260709 12:17:16.520622 28359 rpcz_store.cc:269] 0709 12:17:15.302603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.317425 (+ 14822us) service_pool.cc:224] Handling call
0709 12:17:16.497292 (+1179867us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b5800113798. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:16.519901 (+ 22609us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6051072}
I20260709 12:17:16.493572 29774 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:16.547977 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:16.560230 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 246808us
I20260709 12:17:16.585863 28988 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:17:16.534786 29741 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:16.589608 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:16.589990 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.590317 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.590864 28286 raft_consensus.cc:2368] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4.
I20260709 12:17:16.590902 28988 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:16.592233 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.592555 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:16.590395 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.595142 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:16.597393 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.598109 28287 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.598392 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.599073 28290 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:16.600558 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.600414 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.600787 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.601315 28284 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:16.601421 28291 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.498404 29734 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.607152 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:16.608815 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:16.518186 29754 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.613546 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.612735 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:16.615691 28289 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.617408 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.618257 28292 raft_consensus.cc:2368] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3.
I20260709 12:17:16.618592 28283 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.533151 29787 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.620041 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.621595 28291 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
W20260709 12:17:16.633467 28330 leader_election.cc:336] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.097s (SENT)
I20260709 12:17:16.641024 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:16.641686 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.642409 28284 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:16.603353 29705 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.594563 29530 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:16.649845 28990 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:16.651638 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:16.662438 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45890.4us
I20260709 12:17:16.641999 28991 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:16.662523 28988 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:16.652367 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:16.665457 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:16.671693 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 268438us
I20260709 12:17:16.672628 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.668717 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:16.674748 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:16.676783 28289 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:16.677106 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.677892 28283 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:16.678125 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.679445 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:16.679956 28287 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
W20260709 12:17:16.681936 28330 leader_election.cc:336] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.179s (SENT)
I20260709 12:17:16.623692 29706 raft_consensus.cc:2764] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:17:16.685384 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:16.685935 28292 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:16.690585 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:16.690976 28287 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.688088 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.686693 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:16.693076 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:16.693154 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:16.693692 28439 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.602216 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:16.642052 29810 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.697952 28436 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.698310 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:16.705179 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.705291 28287 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:16.699844 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:16.669451 29805 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:16.706822 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:16.709511 28285 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:16.712517 28290 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
W20260709 12:17:16.744505 29802 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:17:16.745878 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:16.765980 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:16.684614 29491 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.685588 29812 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.809975 28439 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:16.685523 29596 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.675958 29651 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.689229 29798 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:16.684324 28985 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:16.694047 29794 raft_consensus.cc:2764] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:17:16.803530 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.822395 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.803808 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.802006 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.824036 28290 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:16.804878 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.694510 29688 raft_consensus.cc:2764] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:17:16.703477 29695 raft_consensus.cc:2764] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:17:16.811604 28285 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:16.812609 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 100639us
I20260709 12:17:16.716019 29810 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.815209 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:16.671205 29705 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.815769 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.819986 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.746616 29788 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.874907 28291 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.875778 28439 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:16.876282 29810 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:16.873970 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 298157us
I20260709 12:17:16.876753 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.811383 29725 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:16.676692 29530 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:16.802616 29741 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:16.868759 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 61392.9us
I20260709 12:17:16.877817 28432 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:16.878432 28283 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
W20260709 12:17:16.874567 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59460.7us
I20260709 12:17:16.829363 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:16.660898 29774 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.889240 28289 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:16.890550 29815 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.891954 29796 raft_consensus.cc:2764] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:17:16.895370 28439 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:16.869036 29797 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.888491 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:16.909374 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:16.910871 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:16.877908 28433 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:16.818353 29651 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.827756 29680 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:16.920436 28333 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.039s	sys 0.105s
I20260709 12:17:16.913983 28438 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:16.913939 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:16.921979 28333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.040s	sys 0.105s
I20260709 12:17:16.922757 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:16.925567 28287 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:16.879837 29688 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:16.917670 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:16.927202 28436 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.928563 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.929243 28436 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:16.893584 29810 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:16.930061 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 118080us
I20260709 12:17:16.930756 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:16.915948 29797 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:16.925443 29790 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:16.919883 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:16.809634 29805 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.930039 29680 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.927608 29774 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.917994 29794 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:16.927801 29660 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:16.926163 29725 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:16.938341 28436 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:16.939922 29797 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.896152 29741 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.918751 29723 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:16.920831 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:16.964020 29723 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.814867 29812 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:16.969055 29795 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:16.951534 29804 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:16.969487 29730 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.939592 29688 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.953498 29660 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.949070 28433 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:16.965413 29806 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:16.969291 29822 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:17:16.962944 29787 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.977461 29687 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.968197 29651 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:16.993769 29723 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:16.994647 29818 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:17:16.988250 29660 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:16.981879 29795 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:17.030041 29805 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:17.030325 29652 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.005s
I20260709 12:17:17.032702 29687 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:16.997005 29754 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:16.996793 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 103855us
I20260709 12:17:17.058192 29799 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.066133 29508 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.982544 29804 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:16.955811 29794 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.042863 29795 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.077071 29799 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:17.041869 29705 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.072337 29508 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:17.080696 29799 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.073767 29812 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.096431 29491 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.098079 29508 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.099560 29508 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.102296 28435 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.093286 29787 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.103176 29787 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.102212 29810 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:17:17.106609 29746 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.958s	user 0.010s	sys 0.021s
I20260709 12:17:17.106916 29810 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.119745 29749 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:17.024580 29806 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.123577 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.124401 28432 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.127246 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:17.137269 29823 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:17:17.149174 29806 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.157397 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1218) took 1185 ms. Trace:
I20260709 12:17:17.157904 28441 rpcz_store.cc:276] 0709 12:17:15.971913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.973438 (+  1525us) service_pool.cc:224] Handling call
0709 12:17:15.973786 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:16.135220 (+161434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:16.135253 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:16.135770 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:17:16.325991 (+190221us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:16.613421 (+287430us) log.cc:844] Serialized 14 byte log entry
0709 12:17:16.613792 (+   371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.154997 (+541205us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54002e2cd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:17.155051 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:17.155107 (+    56us) raft_consensus.cc:1685] finished
0709 12:17:17.157135 (+  2028us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:17.157322 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29647,"prepare.queue_time_us":107343,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":36357,"raft.run_cpu_time_us":454,"raft.run_wall_time_us":36332,"spinlock_wait_cycles":370289024,"thread_start_us":393000,"threads_started":2,"wal-append.queue_time_us":153634}
I20260709 12:17:15.905479 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:17.166059 28187 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 215) took 4041 ms. Trace:
I20260709 12:17:17.167340 29714 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:17.167845 29714 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:17.168747 29714 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.171648 29714 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.166548 28187 rpcz_store.cc:276] 0709 12:17:13.124679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.124861 (+   182us) service_pool.cc:224] Handling call
0709 12:17:17.165944 (+4041083us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:13.738826 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:13.876059 (+137233us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:13.881812 (+  5753us) write_op.cc:435] Acquired schema lock
0709 12:17:13.881830 (+    18us) tablet.cc:662] Decoding operations
0709 12:17:13.882296 (+   466us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:13.882343 (+    47us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:13.882370 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:17:13.882695 (+   325us) tablet.cc:701] Row locks acquired
0709 12:17:13.882714 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:17:13.882853 (+   139us) write_op.cc:270] Start()
0709 12:17:13.882982 (+   129us) write_op.cc:276] Timestamp: P: 1783599433882814 usec, L: 0
0709 12:17:13.883003 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:17:13.883672 (+   669us) log.cc:844] Serialized 918 byte log entry
0709 12:17:14.134455 (+250783us) op_driver.cc:464] REPLICATION: finished
0709 12:17:14.330094 (+195639us) write_op.cc:301] APPLY: starting
0709 12:17:14.345652 (+ 15558us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:14.355249 (+  9597us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:14.369861 (+ 14612us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:14.370329 (+   468us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:14.370501 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:17:14.370543 (+    42us) write_op.cc:312] APPLY: finished
0709 12:17:14.370773 (+   230us) log.cc:844] Serialized 29 byte log entry
0709 12:17:14.370944 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:14.371035 (+    91us) write_op.cc:454] Released schema lock
0709 12:17:14.371079 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:14.371458 (+   379us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:14.371485 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":130025,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":155574,"apply.run_cpu_time_us":2241,"apply.run_wall_time_us":41342,"num_ops":2,"prepare.queue_time_us":517076,"prepare.run_cpu_time_us":4551,"prepare.run_wall_time_us":181401,"replication_time_us":251309,"thread_start_us":135580,"threads_started":1}]]}
I20260709 12:17:17.127822 29794 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:17.186466 29804 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.147831 29688 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.168548 29797 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.193017 29741 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.199574 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.202575 29491 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.205351 29491 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:17.210549 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.216034 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:17.218693 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:17.194995 29787 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.222532 29812 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.226234 29805 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.231794 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:17.234791 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:17.207515 29491 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.222225 29760 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:17.225266 29792 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:17.227854 29614 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.241811 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:17.245615 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:17.247095 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:17.236150 29812 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.234134 29643 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:17.240273 29632 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.256448 29643 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.250352 29423 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.251648 29804 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.252079 29741 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.264153 29814 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
W20260709 12:17:17.264449 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 335867us
I20260709 12:17:17.244609 29806 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:17:17.242887 29760 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.268620 28286 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.252106 29792 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.279294 29819 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.267757 29741 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.283803 29741 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.259086 29799 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.291134 29799 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.292872 29643 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.295356 29760 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.273252 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:17.259475 29660 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.302735 29792 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.278779 29423 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:17.312613 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:17.313659 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.316306 28362 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:17.316877 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.321787 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.322130 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.323051 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.315086 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.323024 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.325253 28432 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.326550 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:17.328666 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 398188us
I20260709 12:17:17.326912 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:17.336172 28437 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.327791 29749 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:17.347957 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:17.336670 29832 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.337330 29614 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.332127 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:17.324424 29491 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.317256 29827 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.316395 29797 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.358345 29749 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.333105 29736 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:17:17.332849 29688 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:17:17.346626 29792 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:17.340034 29660 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.364421 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1341) took 2062 ms. Trace:
I20260709 12:17:17.370824 29792 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.371337 28365 rpcz_store.cc:276] 0709 12:17:15.302273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.306152 (+  3879us) service_pool.cc:224] Handling call
0709 12:17:15.306570 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:15.695488 (+388918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:17.364126 (+1668638us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b5400126954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:17.364325 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892901632}
I20260709 12:17:17.364512 29754 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.309155 29423 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.365093 29508 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:17.336458 29814 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.367149 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.374482 28437 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:17.374733 29423 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.364009 29804 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.373548 29508 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:17.377007 29508 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.343245 29810 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.357607 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:17.356745 29705 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.405282 28435 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:17.396266 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:17.412559 28954 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.100s	sys 0.112s
W20260709 12:17:17.412943 28954 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.100s	sys 0.112s
W20260709 12:17:17.413231 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 540799us
I20260709 12:17:17.394520 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:17.414217 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.415668 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.416067 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:17.417973 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:17.418682 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.418038 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:17.421286 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.351547 29832 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.427250 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:17.377615 29734 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.361204 29795 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.395669 29733 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.446552 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:17.444865 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:17.448840 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:17.447511 28439 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:17.442219 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:17.383042 29806 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
W20260709 12:17:17.434084 29776 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.678s	user 0.005s	sys 0.000s
I20260709 12:17:17.429932 29814 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.455022 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:17.410001 29754 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:17.410111 29705 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:17.455611 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:17.426024 29491 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:17:17.459311 29588 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:17.484622 28362 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:17.484963 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.462188 28985 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.463189 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:17.463482 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1343) took 2160 ms. Trace:
I20260709 12:17:17.457489 29614 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.455829 29471 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.425627 29423 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:17:17.449352 29661 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.409613 29804 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:17:17.450102 29733 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.467366 29777 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.468039 29615 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.470396 28360 raft_consensus.cc:2410] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:17.455806 29594 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.470904 29810 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.456725 29814 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.443668 29643 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:17:17.443907 29831 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.476286 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:17.555611 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 79417.2us
I20260709 12:17:17.556334 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:17.537098 29661 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.558746 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.560422 28362 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:17.483426 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.486364 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:17.461585 29821 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.507s	user 0.002s	sys 0.000s
I20260709 12:17:17.459249 29730 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.453903 29796 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.464126 29697 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.475592 29705 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.481513 29805 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:17.510598 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47489.8us
I20260709 12:17:17.494639 29741 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.510084 29471 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.493880 29614 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.536188 29588 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.557626 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:17.539621 29733 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.562390 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.482760 29806 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:17.511543 29792 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.549888 29814 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.553508 29737 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:17:17.511287 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.566023 29730 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:17.564442 29661 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.571044 29800 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.573623 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:17.583474 28991 raft_consensus.cc:1217] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 2 LEARNER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:17:17.602140 29614 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:17.625475 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 51929.2us
I20260709 12:17:17.626332 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:17.627547 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:17.541122 29594 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:17.553411 29651 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.578935 29471 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.560256 29831 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.612097 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:17.631119 28990 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.606118 29795 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.574594 29775 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.577000 29649 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.602283 29714 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:17.608180 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50643.8us
I20260709 12:17:17.618520 29730 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.657814 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:17.612370 29660 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.586943 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:17.662720 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:17.621662 29800 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.622011 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:17.631492 29794 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.667718 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:17.628202 29792 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:17.638190 29491 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:17.630246 29822 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.667142 28988 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 2: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:17.630748 29594 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.657349 29775 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.612747 29806 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.669968 29804 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:17.665197 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:17.674589 29800 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.660151 29651 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.493602 28361 rpcz_store.cc:276] 0709 12:17:15.302915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:15.315407 (+ 12492us) service_pool.cc:224] Handling call
0709 12:17:15.340027 (+ 24620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:15.363602 (+ 23575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:15.368285 (+  4683us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:15.435685 (+ 67400us) log.cc:844] Serialized 26 byte log entry
0709 12:17:15.568175 (+132490us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:15.568249 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:15.568343 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:17.363224 (+1794881us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b24003d6dd8. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6209c4 00007f85bb61eacb 00007f85bb7d9dc9 00007f85bb7d9a55 00007f85bb7d909d 00007f85be40b10e 00007f85be42288c 00007f85be421bb6 00007f85be4217d8 00007f85be42b221
0709 12:17:17.463104 (+ 99880us) raft_consensus.cc:1685] finished
0709 12:17:17.463128 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:17.463375 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12437,"raft.queue_time_us":22737,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":7106,"spinlock_wait_cycles":3421440,"thread_start_us":121815,"threads_started":1,"wal-append.queue_time_us":214964}
I20260709 12:17:17.695974 29491 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:17.740844 29833 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:17.620676 29754 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:17.666445 29714 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.694016 28985 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.741571 29804 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.723191 29775 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.728217 29832 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.743306 29822 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.747828 29805 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:17.696369 29792 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.743831 29836 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:17.757992 29491 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.790257 28291 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:17.790947 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 153863us
I20260709 12:17:17.752854 29733 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:17.795953 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:17.775223 29795 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.815759 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.817909 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.818672 28440 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.819803 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.828733 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.812546 29741 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.835881 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:17.837270 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 169630us
I20260709 12:17:17.785362 29836 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.814662 29795 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:17.836977 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.846695 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:17.847390 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.848060 28283 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.830067 29822 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.850554 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:17.851310 28291 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:17.851693 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:17.852269 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:17.849079 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.852299 28283 raft_consensus.cc:2376] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3. Re-sending same reply.
I20260709 12:17:17.853129 28288 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.852890 28291 raft_consensus.cc:2376] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3. Re-sending same reply.
I20260709 12:17:17.854068 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.854260 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.855319 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.860731 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:17.860081 28289 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:17.862972 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.864312 29831 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.857909 29847 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.849017 29660 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:17.865087 29847 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:17.868240 28358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:17.868983 28358 raft_consensus.cc:606] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:17.863669 28289 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.843114 29819 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.869349 28358 raft_consensus.cc:614] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:17.872013 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:17.875267 28291 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.872627 28284 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.887768 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.888548 28284 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.890498 28288 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:17.892843 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:17.898622 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:17.856511 29741 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:17.900269 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:17.901073 28289 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.902871 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:17.904788 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:17.901852 29844 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.906438 29844 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:17.906854 29844 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.908185 29800 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.908504 29740 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:17.859961 29822 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.913501 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:17.914810 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:17.885504 29819 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.878959 29847 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.909972 29845 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.928474 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:17.934046 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 276333us
I20260709 12:17:17.934953 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:17.935448 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.935738 29588 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.936208 29588 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.936632 29588 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.938489 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:17.940155 29649 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:17.940564 29471 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.940755 29649 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:17.942391 29841 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:17:17.943598 29822 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.944185 29507 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.944739 29507 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:17.945245 29588 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.879693 29660 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.915972 29751 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.950431 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.922377 29819 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.960970 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.966336 28365 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:17.920770 29816 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:17.967537 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.968442 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.969594 28440 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:17.969561 28441 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
W20260709 12:17:17.957809 29827 consensus_meta.cc:287] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.563s	user 0.012s	sys 0.000s
I20260709 12:17:17.971534 28362 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:17.913874 29741 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.973409 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.970851 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.982465 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:17.984715 28363 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:17.984974 28289 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:17.961110 29751 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:17.963436 29792 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.991379 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:17.988812 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.948268 29822 consensus_queue.cc:1059] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:17:17.968612 29790 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.992452 29530 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:17.986399 29827 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:17.997110 29594 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:17.972258 29836 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.970494 29806 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.001596 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.998834 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:18.013505 28284 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
W20260709 12:17:18.012233 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 98824.4us
I20260709 12:17:18.003899 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:17.986617 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:18.015409 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:17.977806 29816 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.976531 29588 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:18.017251 28434 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:18.018162 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:18.014940 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:17.998832 29649 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:17.933346 29795 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:17.996727 29661 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:18.018812 28404 connection.cc:256] RPC call timeout handler was delayed by -1.2950609140000324s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 12:17:17.999610 29751 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.031677 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:17.991016 29614 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.023598 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:18.032228 28434 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:18.032568 28990 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:18.025133 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:18.015736 29847 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:18.037724 29847 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.038090 29471 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:18.038476 29842 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:18.039058 29842 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:18.036204 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:18.029310 29734 consensus_meta.cc:287] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.000s
I20260709 12:17:18.040846 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:18.038177 29847 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.041761 28439 raft_consensus.cc:2368] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3.
I20260709 12:17:18.044158 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:18.001055 29660 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:17.945703 29507 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.031036 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:18.023087 28359 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:18.036188 29594 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:18.037025 29804 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.021831 29827 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:18.040041 29842 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.026403 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:18.046602 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:18.053074 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:18.061996 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:18.064435 28285 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:18.064049 29697 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.081681 28284 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:18.059080 29822 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:17:18.068362 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:18.072976 29819 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.082605 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:18.059120 29800 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.083031 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:18.079231 28361 raft_consensus.cc:2410] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:18.019299 29790 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.060637 29736 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.084287 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:18.062139 29794 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.097410 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:18.098722 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:18.099169 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:18.099714 28290 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.103878 28440 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:18.104491 28440 raft_consensus.cc:606] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:18.104871 28440 raft_consensus.cc:614] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:18.089612 29734 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:18.095786 29680 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:18.100379 29789 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:18.094194 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 67885.9us
I20260709 12:17:18.096952 29857 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:18.112066 29733 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:18.114425 29705 consensus_meta.cc:287] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.537s	user 0.012s	sys 0.000s
I20260709 12:17:18.116323 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:18.117033 29790 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:18.083741 29827 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.107761 29734 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:18.124161 29816 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.125276 29661 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.125505 29614 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.128952 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:18.089182 29697 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.108700 29740 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.164546 29614 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:18.110522 29680 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:18.127107 29789 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.165951 29789 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.125913 29741 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.164299 29740 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.104684 29730 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:18.127034 29848 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:18.163520 28290 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:18.157675 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:18.170873 29697 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.165541 29680 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.179426 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:18.210493 28364 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.180869 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:18.182243 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:18.211282 28366 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:18.102792 29794 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.152546 29775 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.191978 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:18.216564 28289 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:18.217144 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:18.220094 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:18.196085 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:18.157428 29734 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.212299 28257 leader_election.cc:400] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3.
I20260709 12:17:18.152170 29800 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:17:18.232735 29800 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.233148 29800 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:18.155041 29705 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.214867 29827 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:18.215893 29819 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.222100 28257 leader_election.cc:403] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:18.223946 29775 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:18.224431 28287 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:18.224779 29740 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.226040 29741 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.227574 29491 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.154651 29857 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.194592 29848 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.181514 29847 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.362797 28364 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:18.251776 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 90) took 1142 ms. Trace:
W20260709 12:17:18.350620 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 196613us
I20260709 12:17:18.796934 28435 rpcz_store.cc:276] 0709 12:17:17.109766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.109938 (+   172us) service_pool.cc:224] Handling call
0709 12:17:17.393182 (+283244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.411094 (+ 17912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.251562 (+840468us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400348850. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:18.251705 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650609920}
W20260709 12:17:18.246932 29777 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.000s
I20260709 12:17:18.735643 29730 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.749339 29775 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3.
I20260709 12:17:18.801972 28285 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:17:18.749259 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 551020us
I20260709 12:17:18.803630 28285 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:17:18.437268 29814 consensus_meta.cc:287] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.802s	user 0.009s	sys 0.002s
I20260709 12:17:18.193643 29844 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:18.751089 29845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.016s	sys 0.073s
W20260709 12:17:18.812944 29845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.017s	sys 0.075s
I20260709 12:17:18.785352 28441 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:18.785975 29842 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:18.814414 29842 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.816555 29842 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.817822 29737 raft_consensus.cc:476] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 12:17:18.787523 29491 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:18.788313 28283 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:18.825031 29491 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:18.825690 29491 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.828585 29848 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:18.829799 29800 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.831312 29800 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.831514 29491 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:18.790948 29741 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:18.788980 29751 consensus_meta.cc:287] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.738s	user 0.012s	sys 0.001s
I20260709 12:17:18.364817 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:18.840519 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 475796us
I20260709 12:17:18.841784 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:17:18.842833 28286 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1235) took 2251 ms (client timeout 2179 ms). Trace:
I20260709 12:17:18.243584 29796 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.480727 29804 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.844122 28367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:18.246191 29697 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.509s
I20260709 12:17:18.215022 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:18.233383 29507 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:18.809970 28284 raft_consensus.cc:2368] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3.
I20260709 12:17:18.803282 29822 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:18.847015 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:18.845093 29804 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.859025 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:18.847651 29855 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.846454 28364 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:18.843396 28286 rpcz_store.cc:269] 0709 12:17:16.591879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.591988 (+   109us) service_pool.cc:224] Handling call
0709 12:17:18.842629 (+2250641us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b54003dc2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:18.842735 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553857792}
I20260709 12:17:18.849664 29827 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:18.844619 29796 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.845851 29705 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:18.843238 29751 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.863436 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:18.838732 29814 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.866899 28286 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:18.867883 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:18.868191 28290 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:18.902556 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:18.850052 29848 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:18.841796 29741 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:18.857986 29588 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:18.864773 29827 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:18.915010 28367 raft_consensus.cc:606] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:18.916576 28362 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:18.918974 28367 raft_consensus.cc:614] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:18.868043 29471 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:18.878324 29868 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:18.864956 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 648843us
I20260709 12:17:18.909498 29754 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:18.909262 29723 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:18.863860 29822 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.930773 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:18.935456 28991 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.912205 29491 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:18.922053 29827 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.863564 29507 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.943125 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1305) took 1131 ms. Trace:
I20260709 12:17:18.944665 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:18.923705 29868 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:18.950158 28438 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.928143 29754 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:18.946117 28437 rpcz_store.cc:276] 0709 12:17:17.811595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.101358 (+289763us) service_pool.cc:224] Handling call
0709 12:17:18.101601 (+   243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:18.101818 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.101834 (+    16us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:18.102273 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 12:17:18.207473 (+105200us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:18.207507 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.207571 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:18.942858 (+735287us) raft_consensus.cc:1685] finished
0709 12:17:18.942877 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.943053 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":78352,"threads_started":1,"wal-append.queue_time_us":725924}
I20260709 12:17:18.942981 29741 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.941691 29491 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:18.972826 29697 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:18.975973 29870 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:17:18.976172 29876 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:18.976752 29876 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.979199 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1270) took 1125 ms. Trace:
I20260709 12:17:18.979728 28291 rpcz_store.cc:276] 0709 12:17:17.853512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.251270 (+397758us) service_pool.cc:224] Handling call
0709 12:17:18.784685 (+533415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:18.785210 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.785266 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:18.786407 (+  1141us) log.cc:844] Serialized 26 byte log entry
0709 12:17:18.882151 (+ 95744us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:18.882194 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.882273 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:18.978887 (+ 96614us) raft_consensus.cc:1685] finished
0709 12:17:18.978912 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.979106 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":95268,"threads_started":1,"wal-append.queue_time_us":159998}
I20260709 12:17:18.981782 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:18.965811 29789 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.799s	user 0.006s	sys 0.009s
I20260709 12:17:18.983682 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:18.983846 29789 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.984658 29873 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:18.985325 29873 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.985948 29789 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.987954 29873 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.989693 29871 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:18.990317 29871 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:18.992628 29871 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:18.993214 29822 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:18.994621 29876 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:18.961872 29736 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.861s	user 0.005s	sys 0.011s
W20260709 12:17:18.962535 29734 consensus_meta.cc:287] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.726s	user 0.012s	sys 0.000s
I20260709 12:17:18.962302 29754 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:18.969134 29680 consensus_meta.cc:287] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.716s	user 0.000s	sys 0.012s
I20260709 12:17:19.010447 29749 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:19.011096 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 88) took 1231 ms. Trace:
I20260709 12:17:19.011510 28292 rpcz_store.cc:276] 0709 12:17:17.779637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.779833 (+   196us) service_pool.cc:224] Handling call
0709 12:17:17.780210 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.859621 (+ 79411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:17.859657 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:17.859704 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:17.859733 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.010798 (+1151065us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b5400059bd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:19.010848 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.010994 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182082304}
I20260709 12:17:18.966061 29847 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:19.014577 29680 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:19.016974 29680 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:19.003994 29734 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:19.011504 29804 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:18.979650 29844 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:19.019280 29751 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:19.021584 29749 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:18.956245 29868 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.004364 29736 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.001665 29491 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.025406 29491 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.025936 28437 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:19.026803 29651 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:19.030077 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:19.033557 29844 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:19.034174 29844 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:19.035892 28439 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:19.036197 29734 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:19.035923 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:19.036962 28434 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.038755 29816 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.039275 29816 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:18.988232 29507 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:19.042512 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:19.043493 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1372) took 1054 ms. Trace:
I20260709 12:17:19.043918 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.043865 28288 rpcz_store.cc:276] 0709 12:17:17.989545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.818732 (+829187us) service_pool.cc:224] Handling call
0709 12:17:18.819088 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:18.819422 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.819452 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:18.820006 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 12:17:18.924853 (+104847us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:18.924897 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.924979 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:19.043285 (+118306us) raft_consensus.cc:1685] finished
0709 12:17:19.043304 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.043427 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":382,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":112,"thread_start_us":95528,"threads_started":1,"wal-append.queue_time_us":148814}
I20260709 12:17:19.046177 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:19.047600 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:19.049122 29871 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.049940 29804 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:19.050354 29740 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:19.050637 29751 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:19.051446 29845 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.051791 29845 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:19.035065 29844 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.052667 29845 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.046167 29507 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:19.054649 29507 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:19.055341 29792 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.041529 29816 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:19.055900 29792 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:19.056381 29804 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:19.056835 29804 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.057631 29845 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.061594 29812 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:19.050562 28190 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 1 to 2, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:19.068475 29804 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:19.022243 29749 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.074186 29792 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:19.075943 29749 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:19.078678 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:19.079566 28365 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:19.062213 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:19.080649 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:19.080837 28432 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.081341 28365 raft_consensus.cc:2368] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3.
I20260709 12:17:19.082883 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1243) took 1239 ms. Trace:
I20260709 12:17:19.069408 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:19.086819 28360 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:19.083391 28441 rpcz_store.cc:276] 0709 12:17:17.843286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.193243 (+349957us) service_pool.cc:224] Handling call
0709 12:17:18.193624 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.813976 (+620352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.082588 (+268612us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54003cc154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:19.082792 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1360347904}
I20260709 12:17:19.088416 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.047528 28186 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 234) took 6319 ms. Trace:
I20260709 12:17:19.070210 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:19.062284 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:19.070884 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:19.071163 29812 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.071628 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:19.090360 28358 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:19.079087 29804 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:19.093674 28991 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
W20260709 12:17:19.095587 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45887.8us
I20260709 12:17:19.100683 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:19.000914 28438 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:19.125540 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.128225 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:19.136493 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 228398us
I20260709 12:17:19.142504 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:18.995414 29741 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.146842 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:19.050050 29871 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 12:17:19.095896 29816 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:19.107396 28432 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:19.113992 29507 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:19.201241 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:19.202248 29792 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.202723 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:19.202806 29792 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.203469 28358 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:19.141631 29749 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.226893 29749 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.127036 29423 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:19.227528 29754 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.200011 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:19.206223 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47815.5us
I20260709 12:17:19.224450 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1376) took 1218 ms. Trace:
I20260709 12:17:19.197500 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:19.208523 28361 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:19.209223 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 94) took 1327 ms. Trace:
I20260709 12:17:19.251776 29796 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.251395 28440 rpcz_store.cc:276] 0709 12:17:17.881777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.650637 (+768860us) service_pool.cc:224] Handling call
0709 12:17:18.793271 (+142634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.793476 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:18.937070 (+143594us) log.cc:844] Serialized 14 byte log entry
0709 12:17:19.208809 (+271739us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:19.208870 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:19.208915 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.208993 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.209146 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3525,"prepare.queue_time_us":185475,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":3639,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":104,"thread_start_us":366321,"threads_started":2}
I20260709 12:17:19.217809 29868 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:19.254693 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.227975 29423 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:19.238183 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:19.135500 28434 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:19.267035 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:19.243584 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:19.239866 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.245919 29651 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:19.205111 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.247939 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:19.248540 29507 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.230069 28283 rpcz_store.cc:276] 0709 12:17:18.006425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.823265 (+816840us) service_pool.cc:224] Handling call
0709 12:17:18.823774 (+   509us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:18.833837 (+ 10063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.833871 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:18.835105 (+  1234us) log.cc:844] Serialized 582 byte log entry
0709 12:17:19.148639 (+313534us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:19.148679 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.148749 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:19.224170 (+ 75421us) raft_consensus.cc:1685] finished
0709 12:17:19.224196 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.224365 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9321,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":313148,"threads_started":1,"wal-append.queue_time_us":386681}
I20260709 12:17:19.269510 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:19.258009 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:19.253666 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:19.224591 29741 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:19.262892 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1380) took 1241 ms. Trace:
I20260709 12:17:19.263887 28255 leader_election.cc:400] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3.
I20260709 12:17:19.126439 29800 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 12:17:19.291981 28291 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:19.271459 28438 raft_consensus.cc:2410] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:19.206400 29816 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:19.274554 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:19.264391 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:19.157449 29734 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:19.301436 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:19.302522 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1373) took 1311 ms. Trace:
I20260709 12:17:19.253665 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:19.279053 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:19.341890 28991 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:19.280539 28434 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:19.281071 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1379) took 1261 ms. Trace:
I20260709 12:17:19.292138 28283 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:19.345371 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:19.271274 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.347430 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:19.348073 28432 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:19.294113 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:19.349766 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:19.196274 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 213286us
I20260709 12:17:19.096118 29812 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:19.295594 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:19.516726 28988 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:19.277976 28286 rpcz_store.cc:276] 0709 12:17:18.021696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.902839 (+881143us) service_pool.cc:224] Handling call
0709 12:17:18.904485 (+  1646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.904719 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:18.995818 (+ 91099us) log.cc:844] Serialized 14 byte log entry
0709 12:17:19.262264 (+266446us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:19.262566 (+   302us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:19.262614 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.262685 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.262824 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":329015,"raft.queue_time_us":516,"thread_start_us":344535,"threads_started":2}
I20260709 12:17:19.303783 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:19.089426 28186 rpcz_store.cc:276] 0709 12:17:12.734238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:12.736435 (+  2197us) service_pool.cc:224] Handling call
0709 12:17:19.047443 (+6311008us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:13.235170 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:13.235753 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:13.235789 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:17:13.235807 (+    18us) tablet.cc:662] Decoding operations
0709 12:17:13.236382 (+   575us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:13.236438 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:13.236466 (+    28us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:17:13.236805 (+   339us) tablet.cc:701] Row locks acquired
0709 12:17:13.236823 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:17:13.236967 (+   144us) write_op.cc:270] Start()
0709 12:17:13.237083 (+   116us) write_op.cc:276] Timestamp: P: 1783599433236928 usec, L: 0
0709 12:17:13.237106 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:17:13.281199 (+ 44093us) log.cc:844] Serialized 1365 byte log entry
0709 12:17:14.031717 (+750518us) op_driver.cc:464] REPLICATION: finished
0709 12:17:14.361201 (+329484us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b180030d4e0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:14.361325 (+   124us) write_op.cc:301] APPLY: starting
0709 12:17:14.361422 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:14.361562 (+   140us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:14.361582 (+    20us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:14.362262 (+   680us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:14.362481 (+   219us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:17:14.362531 (+    50us) write_op.cc:312] APPLY: finished
0709 12:17:14.362773 (+   242us) log.cc:844] Serialized 35 byte log entry
0709 12:17:14.362952 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:14.363053 (+   101us) write_op.cc:454] Released schema lock
0709 12:17:14.363095 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:14.363519 (+   424us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:14.363550 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2322244,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":86808,"apply.run_cpu_time_us":2163,"apply.run_wall_time_us":2162,"num_ops":3,"prepare.queue_time_us":129592,"prepare.run_cpu_time_us":17356,"prepare.run_wall_time_us":390495,"raft.queue_time_us":208595,"raft.run_cpu_time_us":46119,"raft.run_wall_time_us":292141,"replication_time_us":765069,"spinlock_wait_cycles":14541696,"thread_start_us":526148,"threads_started":5,"wal-append.queue_time_us":568338,"wal-append.run_cpu_time_us":42765,"wal-append.run_wall_time_us":2622929}]]}
I20260709 12:17:19.302983 28284 rpcz_store.cc:276] 0709 12:17:17.991043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.082860 (+ 91817us) service_pool.cc:224] Handling call
0709 12:17:19.302251 (+1219391us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54003ab954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:19.302446 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1633940864}
I20260709 12:17:19.306630 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:19.388561 28440 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:19.392302 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.343209 28289 rpcz_store.cc:276] 0709 12:17:18.019639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.867155 (+847516us) service_pool.cc:224] Handling call
0709 12:17:18.867538 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:18.879954 (+ 12416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.879989 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:18.900039 (+ 20050us) log.cc:844] Serialized 26 byte log entry
0709 12:17:18.968237 (+ 68198us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:19.274732 (+306495us) log.cc:844] Serialized 14 byte log entry
0709 12:17:19.275157 (+   425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.276954 (+  1797us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:19.280834 (+  3880us) raft_consensus.cc:1685] finished
0709 12:17:19.280847 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.280992 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":879,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":16144,"thread_start_us":315431,"threads_started":2,"wal-append.queue_time_us":243939}
I20260709 12:17:19.346118 28283 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:19.275815 29423 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.257017 29740 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:19.265221 29822 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:19.278337 29507 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:19.264964 29705 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:19.308355 29588 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:19.299856 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:19.558300 28360 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:19.562511 28432 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:19.563545 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:19.147876 29801 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.324s
I20260709 12:17:19.592706 29751 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.297070 29723 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:19.592995 28359 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56626 (request call id 112) took 2110 ms (client timeout 1866 ms). Trace:
I20260709 12:17:19.480855 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:19.602869 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 328391us
I20260709 12:17:19.608973 29651 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:19.610107 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:19.595515 29884 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 12:17:19.479069 29471 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:19.682776 28364 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
W20260709 12:17:19.739204 28285 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.036s	sys 0.115s
I20260709 12:17:19.683709 28990 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:19.686852 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 96) took 1795 ms. Trace:
I20260709 12:17:19.687924 28440 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:19.689010 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.691900 28288 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
W20260709 12:17:19.692189 28362 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1373) took 1681 ms (client timeout 1995 ms). Trace:
I20260709 12:17:19.698890 28434 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:19.739694 28285 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.036s	sys 0.116s
I20260709 12:17:19.699267 28358 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:19.660782 29766 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:19.728231 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.565063 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:19.593880 29888 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:19.463773 29827 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.752202 28434 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:19.754086 28432 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
W20260709 12:17:19.752737 28362 rpcz_store.cc:269] 0709 12:17:18.011179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.185724 (+174545us) service_pool.cc:224] Handling call
0709 12:17:19.691944 (+1506220us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b5400127354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:19.692112 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606621440}
I20260709 12:17:19.662590 29810 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:19.606606 29796 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:19.742763 28435 rpcz_store.cc:276] 0709 12:17:17.891547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.808072 (+916525us) service_pool.cc:224] Handling call
0709 12:17:18.808419 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.829275 (+ 20856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.829311 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.829352 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:18.829383 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.686529 (+857146us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b5400184554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:19.686595 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.686773 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47500544}
I20260709 12:17:19.738004 29471 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:19.673368 29751 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:19.690019 29800 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:19.696158 29875 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:19.744091 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.769982 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:19.772372 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.600546 29822 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.778537 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:19.785785 28364 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:19.661856 29705 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.778599 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:19.742235 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1244) took 1898 ms. Trace:
I20260709 12:17:19.740777 29882 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:19.787485 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:19.787725 28439 rpcz_store.cc:276] 0709 12:17:17.843583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.197862 (+354279us) service_pool.cc:224] Handling call
0709 12:17:18.198167 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.036320 (+838153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.741865 (+705545us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b540026c5d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:19.742144 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1926164864}
I20260709 12:17:19.788507 28358 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:19.781306 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.782207 28441 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:19.688526 29740 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.789196 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:19.784791 28434 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.765847 29847 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:19.790971 28435 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:19.789407 28439 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:19.772606 29766 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:19.726377 29723 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.773165 29810 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.767694 29888 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:19.766893 29651 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.767705 29507 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:19.790045 28360 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:20.034408 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.778625 29471 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:20.029455 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.040743 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:20.042330 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 764707us
W20260709 12:17:19.694872 28359 rpcz_store.cc:269] 0709 12:17:17.482323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.484657 (+  2334us) service_pool.cc:224] Handling call
0709 12:17:19.592705 (+2108048us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54001d8b54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:19.592862 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235995392}
I20260709 12:17:20.043701 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:20.000111 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1275) took 2115 ms. Trace:
I20260709 12:17:20.043037 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:19.835033 28438 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:19.801370 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.055891 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:20.055981 28441 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:20.056861 28362 raft_consensus.cc:2410] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:20.056767 28439 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:19.802958 28434 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
W20260709 12:17:19.795300 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 491554us
I20260709 12:17:20.058768 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:19.833994 29491 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.808s	user 0.006s	sys 0.004s
W20260709 12:17:19.864125 29889 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:17:19.863327 29885 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 12:17:19.875658 29886 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:17:20.063810 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:19.874761 29822 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:20.000069 29845 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.940s	user 0.010s	sys 0.001s
I20260709 12:17:20.063954 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:19.866204 29892 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 12:17:20.095830 28366 raft_consensus.cc:2410] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:20.061960 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:20.028460 29740 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:19.846347 28361 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.826704 29792 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.067988 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.125232 28360 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 1.
I20260709 12:17:20.125844 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1239) took 1008 ms. Trace:
I20260709 12:17:19.790153 29796 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:19.843807 29827 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:20.071115 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:20.129427 28364 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:20.071341 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:20.053597 28285 rpcz_store.cc:276] 0709 12:17:17.884815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.810747 (+925932us) service_pool.cc:224] Handling call
0709 12:17:18.811487 (+   740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:18.837921 (+ 26434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.837953 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:18.838401 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 12:17:19.149254 (+310853us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:19.745609 (+596355us) log.cc:844] Serialized 14 byte log entry
0709 12:17:19.999695 (+254086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.999813 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:19.999862 (+    49us) raft_consensus.cc:1685] finished
0709 12:17:19.999876 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.000031 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":611,"prepare.queue_time_us":443894,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":35,"raft.queue_time_us":1319,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6049,"thread_start_us":1091850,"threads_started":3,"wal-append.queue_time_us":354720}
I20260709 12:17:20.076222 29898 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.077433 29766 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.080017 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:20.080564 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1267) took 2228 ms. Trace:
I20260709 12:17:20.082427 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:20.133263 28434 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 2: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:20.082993 28439 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:20.133657 28439 pending_rounds.cc:85] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 1
I20260709 12:17:20.133930 28439 pending_rounds.cc:107] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:20.134179 28439 raft_consensus.cc:2889] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:20.085824 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.086581 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.094455 29734 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:19.990144 29825 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 12:17:20.049598 29754 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:20.114771 29814 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:20.087375 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:20.090370 29705 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:19.876005 29816 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:19.857039 29749 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:19.964740 29757 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:20.151196 29757 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:19.793227 29588 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:20.154028 29588 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.111166 29857 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.156284 29588 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:19.867647 29812 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.090929 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41314 (request call id 7) took 1251 ms. Trace:
I20260709 12:17:20.157186 28367 rpcz_store.cc:276] 0709 12:17:18.840003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.843967 (+  3964us) service_pool.cc:224] Handling call
0709 12:17:20.090673 (+1246706us) spinlock_profiling.cc:243] Waited 68.9 ms on lock 0x7b54000a7154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be881853 00007f85c384f60a 00007f85bcdd57d4 00007f85bcdd5762 00007f85bcdd5691 00007f85bcdd560c 00007f85bcdd48c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:20.090824 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158490880}
I20260709 12:17:20.158793 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.159538 28367 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:20.160058 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1245) took 1009 ms. Trace:
I20260709 12:17:20.160342 28367 rpcz_store.cc:276] 0709 12:17:19.150654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.158427 (+1007773us) service_pool.cc:224] Handling call
0709 12:17:20.159992 (+  1565us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:19.785825 29800 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.107383 29888 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.162992 29800 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.100394 29652 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.129631 29777 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:20.166471 29777 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:20.167289 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1287) took 3038 ms. Trace:
I20260709 12:17:20.063668 29847 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:20.167634 29777 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.169183 29777 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.118341 29848 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:20.173882 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 271) took 1215 ms. Trace:
I20260709 12:17:20.174923 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 95) took 2336 ms. Trace:
I20260709 12:17:20.175417 28287 rpcz_store.cc:276] 0709 12:17:17.838140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.838263 (+   123us) service_pool.cc:224] Handling call
0709 12:17:17.838590 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.718107 (+879517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.174540 (+1456433us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b540035fcd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:20.174806 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886948224}
I20260709 12:17:20.103018 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1248) took 1070 ms. Trace:
I20260709 12:17:20.176610 28432 rpcz_store.cc:276] 0709 12:17:19.032507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.762760 (+730253us) service_pool.cc:224] Handling call
0709 12:17:19.763119 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.789627 (+ 26508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:19.789661 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:19.789710 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:19.789765 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.102746 (+312981us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b54002db9d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:20.102803 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.102950 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60386688}
I20260709 12:17:20.108866 29723 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:20.180287 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.110453 29810 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.115540 29530 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:20.116142 29740 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:20.066490 29507 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.182229 29530 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:20.183185 29530 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.183885 29847 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:20.184453 29530 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.184738 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.185433 28364 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:20.126187 28360 rpcz_store.cc:276] 0709 12:17:19.117261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.056218 (+938957us) service_pool.cc:224] Handling call
0709 12:17:20.125755 (+ 69537us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:20.140069 29737 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:20.186981 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1232) took 1118 ms. Trace:
I20260709 12:17:20.176759 29734 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.187374 28360 rpcz_store.cc:276] 0709 12:17:19.068918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.186082 (+1117164us) service_pool.cc:224] Handling call
0709 12:17:20.186473 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.186656 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:20.186680 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:20.186722 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:20.186749 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.186808 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.186917 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40064}
I20260709 12:17:20.189157 28360 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:19.873066 29423 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.137426 28404 leader_election.cc:400] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3.
I20260709 12:17:20.132099 28290 rpcz_store.cc:276] 0709 12:17:17.852593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.221513 (+368920us) service_pool.cc:224] Handling call
0709 12:17:18.867999 (+646486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.901176 (+ 33177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.080307 (+1179131us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b540009e7d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:20.080491 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486079360}
W20260709 12:17:20.195439 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64380.5us
I20260709 12:17:20.195777 28367 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.189515 28433 rpcz_store.cc:276] 0709 12:17:17.129255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.131499 (+  2244us) service_pool.cc:224] Handling call
0709 12:17:17.150982 (+ 19483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:17.536816 (+385834us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:17.725848 (+189032us) log.cc:844] Serialized 14 byte log entry
0709 12:17:17.846217 (+120369us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:17.856182 (+  9965us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:17.857709 (+  1527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.166924 (+2309215us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54002e40d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:20.167005 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.167191 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21698,"prepare.queue_time_us":204274,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":1995,"raft.queue_time_us":89,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":10410,"spinlock_wait_cycles":769737088,"thread_start_us":223265,"threads_started":2,"wal-append.queue_time_us":203608,"wal-append.run_cpu_time_us":1435,"wal-append.run_wall_time_us":1080845}
I20260709 12:17:19.790408 29875 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:20.196534 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:20.197356 28367 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:17:19.847359 29751 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.212010 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:20.214923 29751 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.143350 28363 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.129165 29845 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:20.216223 29845 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.218480 29845 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:20.131839 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51914.6us
I20260709 12:17:20.218675 29794 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:20.219316 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:20.220532 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:20.221365 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:20.136108 29796 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.225898 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:20.227022 29796 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:20.192611 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1257) took 1117 ms. Trace:
I20260709 12:17:20.227793 28440 rpcz_store.cc:276] 0709 12:17:19.075301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.253365 (+178064us) service_pool.cc:224] Handling call
0709 12:17:20.192372 (+939007us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540026bbd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:20.192532 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271101184}
I20260709 12:17:20.197438 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:20.187019 29737 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:20.229770 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:20.229604 29737 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:20.231058 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:20.231995 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:20.232383 29423 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.065522 29868 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.233630 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:20.235278 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 102) took 1259 ms. Trace:
I20260709 12:17:20.235656 28437 rpcz_store.cc:276] 0709 12:17:18.975424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.983407 (+  7983us) service_pool.cc:224] Handling call
0709 12:17:20.235065 (+1251658us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b540026d4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:20.235189 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95486720}
I20260709 12:17:20.236404 29794 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.236848 29794 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.237192 29794 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.238039 29796 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:20.238787 29796 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.198185 29875 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:20.198576 29749 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.239571 29749 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:20.239967 29749 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.199802 29714 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.242794 28439 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:20.244526 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1231) took 1175 ms. Trace:
I20260709 12:17:20.244932 28359 rpcz_store.cc:276] 0709 12:17:19.068656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.191130 (+1122474us) service_pool.cc:224] Handling call
0709 12:17:20.243951 (+ 52821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.244155 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:20.244182 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:20.244231 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:20.244261 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.244320 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.244448 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:20.235996 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:20.246559 28991 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:20.201608 29471 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.206446 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:20.207086 28365 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:20.208616 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:20.208946 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:20.210127 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:20.269420 28990 raft_consensus.cc:2368] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3.
I20260709 12:17:20.212798 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:20.270555 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1378) took 1035 ms. Trace:
I20260709 12:17:20.221913 29816 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.137135 28362 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:20.271799 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1236) took 1173 ms. Trace:
I20260709 12:17:20.222581 29508 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:20.224046 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 1.42734e+06us
W20260709 12:17:20.274263 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52973.2us
I20260709 12:17:20.274600 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:20.239110 29891 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.455s	user 0.003s	sys 0.000s
I20260709 12:17:20.248586 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:20.275383 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:20.276330 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:20.278930 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:20.282776 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:20.283922 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:20.260993 28437 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:17:20.284546 28367 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.261909 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:20.248098 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1248) took 1003 ms. Trace:
I20260709 12:17:20.285352 28365 rpcz_store.cc:276] 0709 12:17:19.244908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.206427 (+961519us) service_pool.cc:224] Handling call
0709 12:17:20.206749 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.247848 (+ 41099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.248039 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:20.286743 28365 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.270502 28987 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:20.272118 28362 rpcz_store.cc:276] 0709 12:17:19.098216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.057573 (+959357us) service_pool.cc:224] Handling call
0709 12:17:20.271735 (+214162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:20.271011 28360 rpcz_store.cc:276] 0709 12:17:19.234709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.188675 (+953966us) service_pool.cc:224] Handling call
0709 12:17:20.188983 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.270248 (+ 81265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.270482 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:20.273366 28435 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:20.288826 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:20.245656 29898 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:20.309151 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 59258.7us
I20260709 12:17:20.297379 28365 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.302470 28987 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 1 LEARNER]: Advancing to term 2
I20260709 12:17:20.272982 28189 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 221) took 7312 ms. Trace:
I20260709 12:17:20.311283 28330 leader_election.cc:400] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3.
I20260709 12:17:20.308225 28363 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:20.327720 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1246) took 1138 ms. Trace:
I20260709 12:17:20.328593 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:20.319017 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.326301 28189 rpcz_store.cc:276] 0709 12:17:12.960616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:12.961201 (+   585us) service_pool.cc:224] Handling call
0709 12:17:20.272912 (+7311711us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:15.091891 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:15.216710 (+124819us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:15.218685 (+  1975us) write_op.cc:435] Acquired schema lock
0709 12:17:15.218706 (+    21us) tablet.cc:662] Decoding operations
0709 12:17:15.258479 (+ 39773us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:15.259789 (+  1310us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:15.265045 (+  5256us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:15.277969 (+ 12924us) tablet.cc:701] Row locks acquired
0709 12:17:15.277990 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:17:15.278148 (+   158us) write_op.cc:270] Start()
0709 12:17:15.278285 (+   137us) write_op.cc:276] Timestamp: P: 1783599435278100 usec, L: 0
0709 12:17:15.278305 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:17:15.279648 (+  1343us) log.cc:844] Serialized 595 byte log entry
0709 12:17:15.323402 (+ 43754us) op_driver.cc:464] REPLICATION: finished
0709 12:17:15.438002 (+114600us) write_op.cc:301] APPLY: starting
0709 12:17:15.438130 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:15.438249 (+   119us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:15.438375 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:15.438658 (+   283us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:15.438831 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:15.438879 (+    48us) write_op.cc:312] APPLY: finished
0709 12:17:15.439067 (+   188us) log.cc:844] Serialized 23 byte log entry
0709 12:17:15.439255 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:15.439352 (+    97us) write_op.cc:454] Released schema lock
0709 12:17:15.439393 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:15.439806 (+   413us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:15.439835 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2025246,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":107849,"apply.run_cpu_time_us":1832,"apply.run_wall_time_us":4607,"num_ops":1,"prepare.queue_time_us":197886,"prepare.run_cpu_time_us":4492,"prepare.run_wall_time_us":202210,"replication_time_us":44941,"spinlock_wait_cycles":1388288,"thread_start_us":80845,"threads_started":2}]]}
I20260709 12:17:20.268663 28986 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:20.337677 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:20.289330 29825 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.236555 29740 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.288580 29530 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.360688 28360 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.325655 29845 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.309809 29760 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:20.289461 29888 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.294097 29796 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:20.273298 29757 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:20.288982 29814 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.277046 29875 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:20.288666 29730 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:20.276017 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 277663us
I20260709 12:17:20.353083 28367 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:20.342803 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:20.361817 29898 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:20.325019 29723 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:20.389194 29845 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.380069 29907 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.380457 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:20.374269 29651 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:20.381153 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:20.381479 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:20.382972 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:20.384166 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:20.435539 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:20.384698 29873 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:20.385368 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:20.385826 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:20.440050 28285 raft_consensus.cc:2368] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3.
I20260709 12:17:20.440625 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:20.440943 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:20.386020 29777 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.387032 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:20.447379 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1386) took 1210 ms. Trace:
I20260709 12:17:20.387578 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:20.448648 28283 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:20.388247 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:20.449473 28289 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:20.328094 28363 rpcz_store.cc:276] 0709 12:17:19.189000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.087104 (+898104us) service_pool.cc:224] Handling call
0709 12:17:20.327655 (+240551us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:20.391067 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:20.392158 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:20.450657 28284 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:20.423544 29766 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.289235 29868 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:17:20.398034 29471 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:20.399991 29740 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.371248 28987 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 2 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:17:20.363590 29751 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:20.414371 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 71641.3us
I20260709 12:17:20.414410 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.332275 29871 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:17:20.415198 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.288604 29822 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.380671 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:20.402951 29639 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.432770 28360 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:20.429010 28440 raft_consensus.cc:2410] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:20.174454 28985 rpcz_store.cc:276] 0709 12:17:18.930620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.934184 (+  3564us) service_pool.cc:224] Handling call
0709 12:17:18.961252 (+ 27068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:18.972193 (+ 10941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.972230 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:18.972780 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 12:17:19.304105 (+331325us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:19.452228 (+148123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.517713 (+ 65485us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:20.139849 (+622136us) raft_consensus.cc:1685] finished
0709 12:17:20.139881 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.146217 (+  6336us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":309890,"threads_started":1,"wal-append.queue_time_us":786615}
I20260709 12:17:20.437813 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:20.441471 28290 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:20.406944 29814 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:20.445226 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1316) took 1178 ms. Trace:
I20260709 12:17:20.446892 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:20.428417 29530 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:20.339346 29508 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:20.447896 28292 rpcz_store.cc:276] 0709 12:17:19.236695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.236889 (+   194us) service_pool.cc:224] Handling call
0709 12:17:19.237262 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.690302 (+453040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:19.690336 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:19.690388 (+    52us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:19.690418 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.447100 (+756682us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54002413d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:20.447160 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:20.447294 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17719,"raft.queue_time_us":18244,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":13523,"spinlock_wait_cycles":856366976}
I20260709 12:17:20.447978 28288 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.431603 29845 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.426415 29796 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:20.449496 28436 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:20.450984 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.430073 29723 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.404269 29760 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:20.425477 29730 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.428031 29898 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.428815 29875 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:20.452718 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 173833us
W20260709 12:17:20.715945 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.921221 28432 raft_consensus.cc:2410] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:17:20.922530 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:20.922662 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:20.919660 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 528586us
W20260709 12:17:20.924207 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 486661us
W20260709 12:17:20.574909 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 290943us
I20260709 12:17:20.919804 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:20.927647 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:20.920478 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.924918 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.930027 28365 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:20.927562 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:20.932304 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.933995 28367 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:20.936517 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.931809 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:20.939970 28365 raft_consensus.cc:2368] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3.
I20260709 12:17:20.940968 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:20.941890 28435 rpcz_store.cc:276] 0709 12:17:19.267201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.792595 (+525394us) service_pool.cc:224] Handling call
0709 12:17:19.792899 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:20.287734 (+494835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:20.444889 (+157155us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b54003cad54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:20.445146 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1101587328}
W20260709 12:17:20.964560 29652 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.801s	user 0.010s	sys 0.000s
I20260709 12:17:20.965164 29652 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.487115 29914 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.966392 29914 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.943106 28365 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:20.970149 28362 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
W20260709 12:17:20.649273 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:20.519007 28366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1225) took 1451 ms (client timeout 1518 ms). Trace:
W20260709 12:17:20.696115 29910 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
W20260709 12:17:20.696162 29909 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
W20260709 12:17:20.974491 28366 rpcz_store.cc:269] 0709 12:17:19.067156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.070649 (+  3493us) service_pool.cc:224] Handling call
0709 12:17:20.479856 (+1409207us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400127854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:20.518904 (+ 39048us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2311496064}
I20260709 12:17:20.448473 28433 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:20.958914 29639 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.448840 29873 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.695978 29734 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:20.996991 29734 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.979499 29723 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:20.997511 29730 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.959090 29766 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.998257 29873 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.998813 29816 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:17:20.999650 29766 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.979928 29825 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:17:21.000466 29917 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:17:20.438827 29651 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.980386 29842 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:21.002744 29842 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:20.968341 29912 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.003456 29912 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:20.519757 29749 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.005262 29639 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.981153 29760 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:20.983553 28441 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 2: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:21.003202 29842 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:21.007484 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:20.452543 29777 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.009182 29908 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.009613 29908 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.009985 29908 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.440076 29907 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.011492 29912 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.012063 29907 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.452587 29775 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.012908 29775 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:20.823791 29794 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.003s
I20260709 12:17:20.521014 29804 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:20.922066 29530 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:20.928567 29847 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.479664 29810 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:20.961299 29814 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:20.959584 29751 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.006116 29749 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:20.985544 29507 consensus_meta.cc:287] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.803s	user 0.006s	sys 0.007s
W20260709 12:17:20.800304 29916 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:17:21.025563 29530 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.007612 29875 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:21.003157 29915 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.424s	user 0.000s	sys 0.000s
W20260709 12:17:21.028455 29848 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.016s	sys 0.030s
I20260709 12:17:21.024382 29794 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.026023 29804 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:21.056437 29848 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.017s	sys 0.030s
I20260709 12:17:21.083374 29898 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.084933 29914 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.086018 29844 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.086516 29844 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:21.087499 29844 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.088943 29844 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.112869 29845 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.114931 29908 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.133466 29908 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.135960 29845 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.162144 28285 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:21.172099 28286 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:17:21.186156 28291 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.022s	sys 0.029s
I20260709 12:17:21.186493 28287 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:20.960608 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:21.196702 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 236015us
W20260709 12:17:21.186906 28291 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.022s	sys 0.029s
I20260709 12:17:21.198343 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:21.200786 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:21.205206 28332 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:17:21.212455 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:20.823094 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:20.995464 28361 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.853s	user 0.007s	sys 0.003s
I20260709 12:17:21.219890 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:21.224260 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 254272us
I20260709 12:17:21.240358 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:21.247079 28985 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:20.983429 29471 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.003314 29651 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:20.983111 29697 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:17:20.998291 29730 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:21.283345 29933 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:21.283888 29933 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:21.284076 29730 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:21.013875 29734 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:21.118738 29842 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.256639 28434 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:21.256697 29775 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.303727 28439 raft_consensus.cc:606] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:21.258170 29804 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:21.264406 29845 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:21.264855 29895 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:21.265879 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 338199us
W20260709 12:17:21.266496 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 257761us
I20260709 12:17:21.264433 29857 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.304281 28439 raft_consensus.cc:614] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:21.306567 29933 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.308684 29933 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.309595 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:52316 (request call id 10) took 1104 ms. Trace:
I20260709 12:17:21.310184 28439 rpcz_store.cc:276] 0709 12:17:20.205544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.242638 (+ 37094us) service_pool.cc:224] Handling call
0709 12:17:21.309216 (+1066578us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540026cad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be881853 00007f85c384f60a 00007f85bcdd57d4 00007f85bcdd5762 00007f85bcdd5691 00007f85bcdd560c 00007f85bcdd48c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:21.309389 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2438112512}
I20260709 12:17:21.288439 29471 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:21.311493 29471 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.285622 29730 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.314699 29845 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.315162 29845 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.315876 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:21.316725 28432 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.318547 29857 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:21.322170 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:21.322785 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:21.323302 28359 raft_consensus.cc:606] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:21.323938 28359 raft_consensus.cc:614] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:21.324201 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 106) took 1063 ms. Trace:
I20260709 12:17:21.324774 28434 rpcz_store.cc:276] 0709 12:17:20.260306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.261619 (+  1313us) service_pool.cc:224] Handling call
0709 12:17:21.323788 (+1062169us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b540026cad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:21.324061 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2232233216}
I20260709 12:17:21.328523 29652 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:21.310105 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 70121.7us
I20260709 12:17:21.330143 29749 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.330492 29723 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:21.330709 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:21.331038 29749 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.331308 29723 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.318893 28441 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:21.310798 29816 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.332758 29857 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.058117 29796 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.334021 29749 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:21.335649 29723 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:21.271085 28288 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:21.272985 29751 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.274274 29932 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.273865 29508 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.278898 28361 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:21.279769 29777 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:21.289505 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 84524.2us
I20260709 12:17:21.295034 29902 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:21.295888 29922 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.296546 29734 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:21.297343 29736 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:21.295023 29794 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:21.333294 29816 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:21.362555 29816 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:21.263654 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.321008 29931 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:21.345152 29757 raft_consensus.cc:1064] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:21.344664 29932 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.345631 29647 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.345803 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:17:21.346040 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:21.346151 29795 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.346714 29737 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:17:21.348315 28361 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1377) took 2114 ms (client timeout 1531 ms). Trace:
I20260709 12:17:21.348685 28364 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:21.351619 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 423936us
I20260709 12:17:21.334214 29842 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:21.357520 29749 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:21.357882 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 435205us
I20260709 12:17:21.329473 29652 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.373407 29848 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.373986 29736 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.376107 29734 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.407255 29734 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:21.376240 28291 raft_consensus.cc:606] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:21.409273 28291 raft_consensus.cc:614] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:21.371169 29647 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:21.410049 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:21.410211 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:21.384924 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:21.384634 29749 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.385883 29825 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:21.386533 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:21.391757 29777 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.397900 29530 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.403905 29848 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.416996 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:21.417348 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.417912 29898 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:21.418421 29898 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.418841 29898 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.404532 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1233) took 2307 ms. Trace:
I20260709 12:17:21.406980 29804 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:21.378412 29932 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.419106 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.408455 29895 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:21.410898 29647 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:21.425698 29842 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:21.426168 29895 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:21.426563 29895 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:21.426918 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:21.427222 29647 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.415997 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:21.428851 29749 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:21.429140 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:21.430102 29795 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:21.414397 29777 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:21.430653 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:21.431362 29795 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.431610 29777 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.403803 29908 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.420956 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:21.434250 28287 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.377177 29902 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.415853 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:21.438340 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:21.429808 29814 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.424655 29652 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:21.425277 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 102755us
I20260709 12:17:21.416497 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:21.419803 28364 rpcz_store.cc:276] 0709 12:17:19.097379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.186084 (+1088705us) service_pool.cc:224] Handling call
0709 12:17:21.348247 (+1162163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:21.392803 (+ 44556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.404276 (+ 11473us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001d9050. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:21.404449 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2671699200}
I20260709 12:17:21.433290 29777 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.443215 28435 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:21.451473 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.452236 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:21.451630 29804 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:21.452769 28440 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:21.453683 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.454203 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.454520 28284 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:21.454957 28285 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:21.455888 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.453028 29804 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:21.381177 28361 rpcz_store.cc:269] 0709 12:17:19.234384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.253368 (+ 18984us) service_pool.cc:224] Handling call
0709 12:17:21.348008 (+2094640us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54001d9054. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:21.348198 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":755632896}
I20260709 12:17:21.456274 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.457134 28284 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:21.458431 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:21.458575 29848 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:21.459091 28284 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.451277 29903 raft_consensus.cc:2764] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:21.460593 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:21.459481 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:21.463114 29736 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:21.464907 29888 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:21.465396 29736 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.466620 29902 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:21.467195 28291 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:21.467933 29903 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.467348 28285 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.459147 29848 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.483971 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:21.484874 29741 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.485383 29741 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:21.439754 29734 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:21.485713 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47500.1us
I20260709 12:17:21.487596 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:21.488622 29775 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.489161 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.489459 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.490229 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:21.468616 29903 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.488926 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.490916 29794 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:21.491720 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:21.492211 29734 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:21.454603 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:21.494043 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:21.427860 29895 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.474975 29868 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.475863 29888 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.476599 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:21.477193 29749 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:21.477910 28435 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:21.478006 29804 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:21.463083 29908 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:21.480813 28436 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:21.490746 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:21.481981 29652 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:21.496833 29868 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:21.492630 29903 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:21.483525 29814 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.490324 28283 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:21.510321 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.511029 28283 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.508057 29816 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:21.474179 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.511917 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:21.492403 28289 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.512885 28291 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:21.514024 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.514639 28291 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:21.515482 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.516654 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:21.496253 29929 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.517328 29929 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:21.518647 29929 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.520057 29929 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.501618 29888 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:21.522856 29888 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.524509 29888 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.526082 28283 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:21.492569 29794 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.495003 29471 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.506245 29749 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.506745 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:21.511442 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:21.508354 29868 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.494206 28288 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:21.460878 29845 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.486531 29741 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.502045 28287 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.527794 29794 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:21.528416 29895 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.541445 29794 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.528451 29749 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:21.528276 28434 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:21.543974 29749 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.528791 28433 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.543354 29794 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.529834 29868 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.551729 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:21.532379 29652 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.554571 29652 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:21.554939 29933 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:21.555673 29652 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.556116 29908 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.556625 29908 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:21.558013 29908 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.559532 29908 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.573094 28987 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.566636 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:21.562431 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:21.568043 28285 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.568738 29898 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.532711 29471 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:21.545611 29749 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.562633 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:21.470304 28432 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
W20260709 12:17:21.539690 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49630us
I20260709 12:17:21.576489 28288 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.527058 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1401) took 1138 ms. Trace:
I20260709 12:17:21.533864 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:21.539867 29903 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.598501 28284 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.579954 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:21.536502 29845 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.601795 28287 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 12:17:21.602502 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.604908 29898 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:21.606105 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 72342.4us
I20260709 12:17:21.606261 29471 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.594660 29903 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:21.607141 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:21.646183 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:21.647814 29933 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.535437 29741 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.590232 28286 rpcz_store.cc:276] 0709 12:17:20.387785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:20.387905 (+   120us) service_pool.cc:224] Handling call
0709 12:17:20.388196 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:21.171554 (+783358us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:21.268051 (+ 96497us) log.cc:844] Serialized 14 byte log entry
0709 12:17:21.501239 (+233188us) log.cc:844] Serialized 14 byte log entry
0709 12:17:21.511807 (+ 10568us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:21.516285 (+  4478us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:21.518002 (+  1717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:21.524160 (+  6158us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b540006fc54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:21.524592 (+   432us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:21.526416 (+  1824us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21508,"prepare.queue_time_us":100288,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":44431,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":249,"spinlock_wait_cycles":1800089856,"thread_start_us":92144,"threads_started":1,"wal-append.queue_time_us":172109}
I20260709 12:17:21.572463 29530 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:21.639288 28288 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
W20260709 12:17:21.642614 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.646315 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:21.610765 29757 raft_consensus.cc:1068] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Unable to promote non-voter f526369c0d0743b192be6e205540a059: Illegal state: Replica 6634fa3fbd594726a5ec3c1686900284 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:17:21.648206 29941 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:17:21.575246 29814 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.673409 28289 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.650172 29903 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.678153 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:21.688545 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.639655 29471 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:21.694993 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:21.695326 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:21.696043 28982 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:21.654711 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47706.3us
I20260709 12:17:21.674088 29933 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:21.699493 28990 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:21.699328 29933 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:21.707860 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:21.708844 29857 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.711325 28982 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:21.712679 28284 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:21.713179 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.674449 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:21.712702 28987 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.714876 29777 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.715278 29777 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:21.715691 29777 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.714573 28287 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:17:21.718012 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.698227 28404 leader_election.cc:400] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3.
I20260709 12:17:21.719378 28404 leader_election.cc:403] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:17:21.719907 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65023.1us
I20260709 12:17:21.720360 28985 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:21.721060 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:21.722052 29530 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3.
I20260709 12:17:21.723764 29902 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.724284 29902 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:21.714390 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 74542.1us
I20260709 12:17:21.707870 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:21.714287 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.727509 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:21.711128 28332 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:21.746093 28291 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
W20260709 12:17:21.782295 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:21.776860 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 49452.4us
I20260709 12:17:21.774561 28358 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:21.791944 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.783069 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.796087 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:21.803480 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:21.814844 28434 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.729163 29814 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.799178 28288 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.805496 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:21.673065 29816 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:21.817529 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.803236 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:21.821965 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:21.822538 28440 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:21.823112 28287 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.724704 29902 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:21.846112 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.707969 29471 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.851687 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:21.852852 28288 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:21.855101 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 67388.1us
W20260709 12:17:21.855687 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 144661us
W20260709 12:17:21.859964 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.861996 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.857446 28289 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:21.858306 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:21.852087 28330 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.567s (SENT) [suppressed 7 similar messages]
I20260709 12:17:21.863607 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:21.866932 29639 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:21.871089 29639 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.873790 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:21.867327 29898 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.872237 29507 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.853714 29508 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.886025 29652 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:21.890041 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:21.884699 28288 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.892655 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:21.914438 28287 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.921159 28289 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.902580 28985 raft_consensus.cc:2410] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:21.898397 29736 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.933550 28440 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
W20260709 12:17:21.935708 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 115764us
I20260709 12:17:21.899260 29507 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:21.937688 28286 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:21.939651 29714 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:21.941278 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:21.915297 28433 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:21.944804 29639 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:21.922350 29749 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:21.924045 29816 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:21.921245 29757 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:17:21.951251 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:21.964253 28364 raft_consensus.cc:2410] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:21.983906 28434 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:22.001709 28255 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:21.899644 29652 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:21.996300 29816 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:21.975064 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:21.973055 29647 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.995384 29848 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.010445 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:22.007694 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 92911.2us
I20260709 12:17:22.013403 28433 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:22.019737 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.000851 29736 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:22.023552 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.024711 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:22.026984 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:22.029984 29508 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:22.030681 29508 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:22.031885 29845 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:22.032095 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.032656 29845 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:22.033932 29845 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.035123 29848 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.008302 29741 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:22.039683 29741 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.031785 29508 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.030216 29796 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.043108 29796 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:22.043504 29796 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:22.044724 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.011615 29816 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.047698 29777 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.050099 28289 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:22.029949 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:21.960822 29903 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.057823 29737 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.059346 29903 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:22.063252 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:22.025962 29639 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:22.037546 29848 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.025359 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:22.070150 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:22.073380 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 221320us
I20260709 12:17:22.053665 29757 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:22.073644 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:22.065797 29868 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:21.970731 29507 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.071485 29639 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:22.102763 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:22.050318 29749 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.104027 28291 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.105316 29647 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:22.105978 29647 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:22.108357 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.108446 29647 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.081274 29777 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:22.146625 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.111941 29639 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.156963 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:22.109346 29736 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:22.164988 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.101843 29757 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:22.164517 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 101880us
I20260709 12:17:22.132733 29734 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:22.172869 28291 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
W20260709 12:17:22.171949 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.170096 29777 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:22.172087 29730 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:22.191437 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 167975us
W20260709 12:17:22.191819 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 101289us
I20260709 12:17:22.173241 29902 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.195541 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.182070 29734 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:22.198460 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.189148 29777 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:22.203589 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.207540 29933 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:22.207957 29933 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.208338 29933 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:22.220690 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.196767 29652 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.216411 29777 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.204627 29868 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:22.219195 29903 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.244742 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.206003 29471 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.257699 29471 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.266620 28438 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:22.248071 29903 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:17:22.258184 29944 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.224s	user 0.000s	sys 0.005s
I20260709 12:17:22.277089 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.271703 29749 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:22.289136 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.289973 28361 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:22.296072 29933 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.302639 29902 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.303586 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.303507 29898 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:22.305301 29842 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:22.306162 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:22.306746 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.240402 29757 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.280731 29903 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.279950 29908 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.318135 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.325781 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.331465 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.331959 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:22.332146 28441 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:22.332468 28358 raft_consensus.cc:2410] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:22.333405 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.326326 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.334085 28363 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:22.339186 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.348214 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.348421 28364 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:22.299481 29816 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.365056 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:22.367062 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 122511us
I20260709 12:17:22.333714 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.374660 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.378094 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.378906 28434 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:22.380172 29796 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.382058 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:22.383997 29814 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.385993 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.387398 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:22.388860 29959 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:17:22.284037 29734 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.391829 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.392670 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.393407 28436 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:22.394205 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.394805 28436 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:22.395382 29933 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.396351 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.392036 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.343261 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:22.392009 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.399372 28982 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:22.399511 28363 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.399852 28985 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:22.400382 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.398553 28438 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:22.391928 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.381914 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:22.402393 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.406217 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.334928 29639 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.401916 29741 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.405771 28987 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:22.416816 29825 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:22.417697 29825 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.410507 29775 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.419056 29775 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:22.419464 29775 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.419936 29816 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.421020 29825 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.422338 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.423045 28367 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:22.423570 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.423928 28440 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:22.424914 29796 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.382112 29795 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:22.392246 29888 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 12:17:22.427587 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.428784 28433 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:22.431126 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.412649 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.413472 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.414160 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.414527 28441 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:22.433629 28438 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:22.432794 28437 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:22.442363 29816 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:22.446271 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.426177 29730 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.452557 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.453009 28365 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:22.440728 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.454735 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:22.455431 29639 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:22.456316 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43560.6us
I20260709 12:17:22.457311 28954 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.461776 29933 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:22.463455 29933 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:22.464057 29933 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:22.464295 29963 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:22.464691 29963 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:22.464911 28290 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:22.465184 29933 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.465716 29963 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.499051 29963 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.499346 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.467036 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.471024 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.471920 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.472735 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.474747 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:22.477466 28433 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:17:22.482105 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1954302860001462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
I20260709 12:17:22.483234 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:22.484030 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.484814 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.486023 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:22.484889 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.500631 28435 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:22.490401 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:22.430009 29842 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.509878 28365 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:22.510435 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:22.508551 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.514041 28361 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:22.519846 28363 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:22.509440 29507 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:22.526779 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:22.516481 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 45564.4us
I20260709 12:17:22.508770 29898 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.547565 28440 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:22.551528 29652 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.551582 29794 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.547427 29908 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:17:22.552402 29902 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:22.552220 29652 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.555472 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:22.558954 28332 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:22.553568 29929 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:22.521261 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.565851 29796 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:22.579041 29777 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.552165 29471 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:17:22.581377 29777 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.565990 29507 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:22.589058 29507 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:22.589535 29507 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.592000 28359 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.593992 28284 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:22.594472 28362 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:22.582438 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:22.595927 29741 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:22.596055 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.597352 28287 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
I20260709 12:17:22.598011 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:22.598851 29796 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:22.599892 29796 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.583806 28989 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:22.601960 28289 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:22.603166 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:22.603834 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.525077 29857 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.586036 28440 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:22.605680 29749 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:22.567643 29903 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:22.606309 29749 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:22.607254 29749 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.608168 29903 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.609635 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.567150 29929 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:22.610513 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.610915 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.612016 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.612172 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.611244 29929 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.604718 28404 leader_election.cc:400] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3.
I20260709 12:17:22.613449 29775 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.613991 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.613673 29929 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.614850 28360 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:22.615311 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.552268 29794 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.605413 29737 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.610424 28363 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:22.616756 28290 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:22.610476 28366 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:22.603242 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.605237 29857 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.605652 29647 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:17:22.615814 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.585767 29652 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.605351 29922 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:17:22.620580 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:22.621961 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:22.615898 29775 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:22.624373 29652 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:22.630872 29652 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:22.631294 29652 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.632046 29741 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:22.634068 29507 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.635290 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.636740 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.622984 29857 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:22.624054 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.638142 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.624851 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.625583 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.640146 28291 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:22.617813 29794 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:22.626720 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.642218 28290 raft_consensus.cc:2376] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4. Re-sending same reply.
I20260709 12:17:22.626859 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:22.613310 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.628109 28190 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 112) took 9513 ms. Trace:
I20260709 12:17:22.628592 29922 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.630288 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:22.646821 29922 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.645004 28190 rpcz_store.cc:276] 0709 12:17:13.085301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:13.085627 (+   326us) service_pool.cc:224] Handling call
0709 12:17:22.598580 (+9512953us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:21.060739 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:21.104207 (+ 43468us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:21.104261 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:17:21.104280 (+    19us) tablet.cc:662] Decoding operations
0709 12:17:21.104687 (+   407us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:21.104754 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:21.104785 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:21.105002 (+   217us) tablet.cc:701] Row locks acquired
0709 12:17:21.105020 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:17:21.105203 (+   183us) write_op.cc:270] Start()
0709 12:17:21.105349 (+   146us) write_op.cc:276] Timestamp: P: 1783599441105142 usec, L: 0
0709 12:17:21.105372 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:17:21.272990 (+167618us) log.cc:844] Serialized 533 byte log entry
0709 12:17:21.476485 (+203495us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be882e28 00007f85c22bd581 00007f85c22bcd58 00007f85c22c19f1 00007f85c22c19a9 00007f85c22c1939 00007f85c22c1901 00007f85c22c0b2d 0000000000388634 0000000000388479
0709 12:17:21.698345 (+221860us) op_driver.cc:464] REPLICATION: finished
0709 12:17:22.060191 (+361846us) write_op.cc:301] APPLY: starting
0709 12:17:22.060340 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:22.060499 (+   159us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:22.060542 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:22.060927 (+   385us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:22.061245 (+   318us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:22.061316 (+    71us) write_op.cc:312] APPLY: finished
0709 12:17:22.158443 (+ 97127us) log.cc:844] Serialized 23 byte log entry
0709 12:17:22.178230 (+ 19787us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:22.185942 (+  7712us) write_op.cc:454] Released schema lock
0709 12:17:22.186016 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:22.257483 (+ 71467us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:22.257518 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2499129,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":354055,"apply.run_cpu_time_us":3550,"apply.run_wall_time_us":179041,"num_ops":1,"prepare.queue_time_us":776329,"prepare.run_cpu_time_us":49450,"prepare.run_wall_time_us":421018,"raft.queue_time_us":228850,"raft.run_cpu_time_us":10049,"raft.run_wall_time_us":98622,"replication_time_us":592778,"spinlock_wait_cycles":361925504,"thread_start_us":677460,"threads_started":4,"wal-append.queue_time_us":139683}]]}
I20260709 12:17:22.648975 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:22.649832 28289 raft_consensus.cc:2376] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3. Re-sending same reply.
I20260709 12:17:22.650398 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:22.650836 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:22.560305 29757 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:22.652223 29757 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.652580 29757 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.652619 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.654436 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.654884 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:22.655736 29794 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:22.656148 29794 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:22.656541 29794 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:22.654467 29736 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:22.657482 29736 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:22.651428 29922 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:22.650918 29471 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.658281 29736 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.658958 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.659710 28290 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:22.660735 29652 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.663816 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:22.666188 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:22.668197 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:22.670047 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:22.670797 28291 raft_consensus.cc:2376] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3. Re-sending same reply.
I20260709 12:17:22.671555 29471 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:22.672057 29471 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.678368 29857 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:22.679045 29857 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.680962 29922 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.681423 29857 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.681972 29922 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.687188 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.688164 28438 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 2: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:22.683880 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.691999 28440 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:22.692109 29922 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.678195 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:22.694294 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44015.8us
W20260709 12:17:22.702539 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 179453us
W20260709 12:17:22.695155 28255 connection.cc:256] RPC call timeout handler was delayed by -2.0450071180002851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
W20260709 12:17:22.703594 28255 connection.cc:256] RPC call timeout handler was delayed by -1.9987484130001576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
W20260709 12:17:22.704056 28255 connection.cc:256] RPC call timeout handler was delayed by -1.9964071180002065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 12:17:22.704452 28255 connection.cc:256] RPC call timeout handler was delayed by -1.9235071180000887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 12:17:22.702158 29903 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.707598 29903 consensus_queue.cc:1243] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:22.708189 29639 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:22.708627 29639 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:22.708895 28286 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:22.709713 29639 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.710964 29639 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.711571 29932 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.712249 29932 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.713120 28434 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:22.698575 28361 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:22.715338 29929 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.715711 29932 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.715969 29929 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.722186 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:22.723038 28986 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:22.724267 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:22.725232 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.725621 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:22.724578 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:22.726011 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:22.726625 28985 raft_consensus.cc:2410] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:22.722196 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:22.728039 28987 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:22.710292 29652 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.701363 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:22.737013 28255 connection.cc:256] RPC call timeout handler was delayed by -1.7588071180002771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
I20260709 12:17:22.737362 28290 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:17:22.739265 28255 connection.cc:256] RPC call timeout handler was delayed by -1.5752071180002076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 12:17:22.734598 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:22.756649 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 55422.8us
I20260709 12:17:22.813948 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:22.814337 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:22.815726 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:22.823606 29795 raft_consensus.cc:2764] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:22.824853 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.825024 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:22.762913 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:22.789949 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.762164 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.761207 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.808316 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:22.761296 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:22.775753 29736 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.780781 29471 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.811070 28362 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:22.844689 28362 pending_rounds.cc:85] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 1
I20260709 12:17:22.824465 29795 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.762148 29794 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.845780 29795 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.845870 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:22.847162 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 85509.2us
W20260709 12:17:22.847357 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 112835us
I20260709 12:17:22.815964 28366 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:17:22.851608 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 108415us
I20260709 12:17:22.809051 29903 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.834200 28358 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:22.810262 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.856159 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:22.852541 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.742760 28361 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:22.862351 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 99677.4us
I20260709 12:17:22.860875 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:22.862270 28360 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.858147 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:22.845018 28362 pending_rounds.cc:107] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:22.864754 28362 raft_consensus.cc:2889] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:22.865638 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:22.866367 28987 raft_consensus.cc:2410] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:22.869153 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:22.874387 28441 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:22.857663 29795 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.875357 29734 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:22.875833 29734 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:22.876207 29734 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.878029 29963 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:22.880373 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 68217.1us
I20260709 12:17:22.881848 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.863914 29929 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.882817 28363 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:22.884364 29922 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:22.890993 29922 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.891487 29922 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.891026 29736 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.861620 29507 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.857215 29978 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:17:22.895620 29969 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:22.896078 29969 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:22.898108 29794 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:22.864243 28359 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:22.887974 29888 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:22.888998 29757 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.862094 29741 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:22.859514 29903 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:22.905818 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:22.899207 29647 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.912232 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.897145 29969 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.920691 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:22.922201 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:22.863574 29737 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:17:22.902688 29842 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:22.934121 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 52382.5us
I20260709 12:17:22.907696 29741 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.943895 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:22.937042 29777 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:22.900697 29908 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:17:22.919927 29978 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:17:22.939175 29757 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:22.943979 29730 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:22.950937 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:22.951519 28287 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:22.951917 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.950997 29730 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.939772 29898 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.930426 29647 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:22.940064 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:22.956358 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:22.957366 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:22.932809 29929 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:22.958168 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.940773 29857 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.959360 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.960057 28288 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:22.911401 29825 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:17:22.960924 29816 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.901343 29888 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:22.944557 29842 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:22.961930 29842 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.961966 29736 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:22.962661 29794 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:22.962358 29888 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:22.963236 29794 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:22.963709 29888 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:22.964627 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:22.941658 29652 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:22.966315 29652 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.966720 29652 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.967031 28291 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.945916 28360 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:22.911291 29903 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.965534 29736 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:22.968923 28364 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:22.977664 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.978472 28286 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:22.975747 28291 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:22.981521 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.984797 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.945276 29741 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.985787 28284 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:22.987735 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:22.990087 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.990886 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:22.991482 28438 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.996325 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.997951 28434 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:22.999920 29647 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:23.002238 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.003149 28292 raft_consensus.cc:2368] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4.
I20260709 12:17:23.005265 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.006117 28285 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:23.007534 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.998080 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:22.980834 29988 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:22.981040 29796 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:23.025249 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:23.028254 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:23.008402 29963 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:23.024231 29988 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:23.031399 29988 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.032604 29471 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:23.033430 28360 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:23.025434 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:22.986217 28361 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:23.041949 28288 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:23.043064 28288 pending_rounds.cc:85] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 1
I20260709 12:17:23.043445 28288 pending_rounds.cc:107] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:23.043766 28288 raft_consensus.cc:2889] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:23.042906 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:23.043669 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 62407.4us
I20260709 12:17:23.044637 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.031543 29749 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:23.050565 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 138044us
I20260709 12:17:23.023155 29814 raft_consensus.cc:2764] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:17:23.057098 28292 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:22.987044 29507 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:17:23.030736 29969 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.064205 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:23.060909 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 75076.6us
I20260709 12:17:23.049242 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:23.024596 29796 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:23.065779 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:23.067112 28289 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:23.067220 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.068532 29736 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.070044 29796 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.070295 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:23.036690 29647 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.988044 29898 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:17:23.064086 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:23.045061 29978 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:23.077611 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:23.046954 29777 raft_consensus.cc:2764] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:23.047189 29737 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.083556 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.088974 29903 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.078631 29932 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:17:23.091635 28288 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:23.092897 29929 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:23.099231 28287 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:23.092047 29978 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:23.094590 28286 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:23.096968 28289 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:23.101621 29993 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:23.079156 29816 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:23.106262 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:23.107870 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43881.4us
I20260709 12:17:23.108659 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.084126 29737 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.109707 29737 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.109966 29932 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:23.078691 29749 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:23.110149 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:23.111927 29749 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:23.083573 29471 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.108660 29908 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.113862 29993 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:23.112560 29987 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:23.114903 28437 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:23.116741 28292 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:23.118033 28286 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:17:23.122123 28434 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:23.121423 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 51196.1us
I20260709 12:17:23.081534 29647 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.099833 29734 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.123463 29507 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.123919 29507 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.124300 29507 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.112867 29929 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:23.125223 29734 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:23.125448 29929 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.126649 29991 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:23.112703 29997 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:17:23.115959 29795 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.132293 28436 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:23.113364 29932 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:23.125150 29647 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:23.138414 29932 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.143112 29775 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:23.118988 29471 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:23.143600 29775 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:23.143157 29932 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.129007 29842 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.138183 29825 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:23.138576 29987 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:23.146644 29730 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:23.152855 29978 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.124387 29993 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.119894 29796 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.145983 29471 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.203050 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:23.176919 29741 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.177119 29922 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.177872 29888 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:23.179145 29857 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:23.179924 28290 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:23.180557 28190 catalog_manager.cc:5697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:23.123199 29908 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:23.184612 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 53429.2us
I20260709 12:17:23.182930 29987 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:23.202843 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:23.175918 29795 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:23.192792 29816 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:23.183238 29775 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.216158 28434 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:23.205255 29868 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:17:23.217338 29741 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:23.218209 29888 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:23.183153 28438 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:23.221122 28436 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:23.219832 28190 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
W20260709 12:17:23.222985 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 130056us
I20260709 12:17:23.184151 29825 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:23.229875 28367 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:23.233964 28190 catalog_manager.cc:5697] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:23.237922 28433 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:23.238417 29816 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.239583 29842 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:23.239849 29507 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.242468 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:23.244305 28190 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 4, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 4 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:23.248234 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.246122 28190 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.230582 29888 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:23.251448 29888 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.252912 29888 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.254997 28190 catalog_manager.cc:5697] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 3, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:23.259307 29737 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.261466 29737 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:23.261929 29775 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.213378 29730 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
W20260709 12:17:23.286026 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 55851us
I20260709 12:17:23.256841 28190 catalog_manager.cc:5697] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 2 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:17:23.287562 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 167740us
W20260709 12:17:23.294119 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 213068us
I20260709 12:17:23.296656 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:23.297837 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 78658us
I20260709 12:17:23.298821 28190 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:23.301206 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:23.304827 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:23.314406 28190 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 2 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:23.318558 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:23.320196 28435 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:23.320533 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.315953 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.243194 29795 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:17:23.321609 28290 raft_consensus.cc:2376] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3. Re-sending same reply.
I20260709 12:17:23.323841 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.347582 28287 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 2: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:23.265813 29992 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:23.354803 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 56390.2us
I20260709 12:17:23.355885 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:23.359138 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.328471 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:23.302861 29903 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:23.331558 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.365193 28439 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:23.365443 28288 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:23.331811 29825 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:23.333636 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.331481 29652 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.339730 29730 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.400171 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.403810 28289 raft_consensus.cc:2376] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3. Re-sending same reply.
W20260709 12:17:23.346516 30002 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:17:23.324687 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:23.366432 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:23.389075 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.417090 28284 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:23.412209 28190 catalog_manager.cc:5697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.331627 29507 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:23.398252 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.303062 29777 raft_consensus.cc:2764] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:17:23.413398 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 112261us
I20260709 12:17:23.397661 28292 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:23.417495 28284 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:23.413841 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:23.347689 29741 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:23.417114 28987 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:23.397243 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:23.417932 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:23.419175 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 63414.4us
I20260709 12:17:23.420192 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:23.423990 28190 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:23.431392 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:23.433763 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:23.435191 28434 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:23.438252 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:23.429355 28190 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 2 to 3, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:23.421104 29736 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:23.447904 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.443486 28439 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:23.443260 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:23.443305 28434 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:23.441668 28190 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.445829 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.458959 28364 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:23.397894 29988 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.462911 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:23.465152 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.461305 28190 catalog_manager.cc:5697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 3, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 3 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.415665 29730 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.466405 28190 catalog_manager.cc:5697] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 2, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 2 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:23.480119 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.434432 29639 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:23.449434 28284 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.436053 29998 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:23.482043 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:23.445627 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:23.486733 28365 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:23.487140 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:23.319322 29987 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.500034 28190 catalog_manager.cc:5697] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.488503 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:23.488988 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.499087 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:23.503374 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:23.503926 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:23.504105 28358 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:23.461978 29737 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.504707 29898 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:23.505013 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.522056 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:23.505440 28991 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:23.505451 29842 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:23.506398 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:23.506743 29995 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 12:17:23.507025 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.532459 28286 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:23.533556 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.534250 28286 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:23.535194 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.507257 30000 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:17:23.464860 29796 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:23.477387 29993 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:23.464610 29647 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:23.504377 30007 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
W20260709 12:17:23.506966 30005 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:17:23.449010 29902 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:17:23.464458 29736 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:23.518364 30015 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:23.518918 29652 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:23.464211 29775 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:17:23.519647 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:23.489904 28985 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:23.485745 29777 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:17:23.499584 29922 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:23.529461 28363 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:23.535416 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.584599 28286 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.538698 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:23.530387 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 112552us
I20260709 12:17:23.530625 29734 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.546308 29998 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:23.547394 28364 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:23.549206 30010 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:23.577255 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:23.541167 29647 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.581772 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:23.550849 29908 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:23.537361 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.585197 28292 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:23.537302 29737 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:23.622530 28358 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:23.625669 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:23.633538 28292 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.591089 29898 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.626138 29647 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:23.635771 29898 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:23.778121 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:23.813644 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:24.098984 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.028622 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:24.100131 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 578170us
I20260709 12:17:24.101042 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:23.902117 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 361384us
I20260709 12:17:23.769009 28286 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:23.591286 29775 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:24.110913 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:24.116331 28363 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:23.638852 28361 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:17:23.639293 30017 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 12:17:23.893649 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:24.110170 29888 consensus_meta.cc:287] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.857s	user 0.014s	sys 0.000s
I20260709 12:17:24.124387 28363 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:24.119529 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:24.121511 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:23.941206 29993 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:23.638100 29737 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:24.099808 29825 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.125890 28363 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:23.643716 29741 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:23.638669 28359 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:24.122170 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:24.111896 29775 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:24.133059 28290 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:23.605883 29734 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.120321 30007 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:24.139039 28363 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:23.643006 29471 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:24.124023 29842 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.017s	sys 0.002s
W20260709 12:17:24.148680 28292 consensus_meta.cc:287] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.000s
I20260709 12:17:24.149842 28292 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:24.150262 28435 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:23.638971 29736 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:24.143600 29842 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.017s	sys 0.002s
I20260709 12:17:24.151556 28290 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:24.152067 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:24.145800 29988 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:24.158051 30026 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.158494 30026 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:24.158846 30026 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:24.159297 30019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.000s	sys 0.013s
W20260709 12:17:24.159684 30019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.000s	sys 0.013s
W20260709 12:17:24.161119 29992 consensus_meta.cc:287] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.000s
I20260709 12:17:24.146167 30000 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.162050 30000 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:24.162424 30000 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.163260 29471 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:24.154690 30020 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.167363 30020 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.164958 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.169982 29794 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:24.171401 28404 leader_election.cc:400] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4.
I20260709 12:17:24.171761 28404 leader_election.cc:403] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:17:24.172374 29794 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4.
I20260709 12:17:24.173530 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:24.174564 29794 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.174993 29794 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:24.161518 29992 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:24.174428 29898 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:24.158702 30024 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.175436 29794 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.182116 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:24.146270 28258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.012s	sys 0.029s
I20260709 12:17:24.193877 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.193990 28990 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:24.194033 28258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.013s	sys 0.029s
W20260709 12:17:24.195538 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.174914 29775 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.216384 29775 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:24.220050 29647 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.000s
I20260709 12:17:24.220219 29741 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.220563 29647 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.222805 29647 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:24.170068 29993 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:24.229562 28358 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.001s
I20260709 12:17:24.231186 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:24.230893 28358 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:24.233234 28363 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:24.207613 30033 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:24.220233 30009 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.705s	user 0.001s	sys 0.013s
I20260709 12:17:24.210207 30032 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:24.215693 30020 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:24.237294 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:24.237921 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 118926us
I20260709 12:17:24.210904 29471 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:17:24.242349 28358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "014b394597e7430489bd093d8616dcad"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:24.243044 28358 raft_consensus.cc:606] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:24.243592 28358 raft_consensus.cc:614] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:24.233124 29902 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:17:24.235110 30001 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:17:24.249974 28360 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:24.244465 30033 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:24.247421 30032 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:24.262207 30032 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.262961 29471 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.262749 30032 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.263470 29471 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.265648 29471 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:24.267987 30026 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.251825 29992 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.258174 29734 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:24.247305 30020 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:17:24.269021 29992 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.269517 30020 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:24.269903 30020 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:24.270706 30020 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.271366 29992 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:24.271768 30020 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.259095 30040 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:24.268419 28987 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:24.279386 29794 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.280907 28287 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:24.249678 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 128261us
I20260709 12:17:24.283776 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:24.284961 30037 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:24.285290 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:24.285538 30037 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:24.286562 30038 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:24.286651 30037 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.287238 30038 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:24.279820 30032 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:24.290782 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.292371 30038 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.293201 29903 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:24.293692 29903 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:24.294642 29903 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.296254 29903 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.299736 29507 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:24.302726 28290 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:24.290441 30000 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.306079 28290 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:24.307868 28290 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:24.309412 28290 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.299574 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:24.310592 28290 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:24.309703 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:24.313174 28330 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:24.313416 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 69876.8us
I20260709 12:17:24.316352 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:24.317308 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:24.311347 29999 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.318878 29794 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:24.328661 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:24.329725 28362 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:24.333436 29507 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.333992 29749 raft_consensus.cc:1064] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:24.334426 30025 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:24.334856 30025 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:24.335322 30026 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:24.339357 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:24.340257 28363 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:24.335934 30025 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:24.344457 29990 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.344523 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:24.344660 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:24.345920 29868 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.346434 29868 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.348613 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:24.349186 30015 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:24.348680 29868 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:24.349577 28989 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:24.346416 29794 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.351135 29794 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.351552 29794 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.355316 29888 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.356043 29888 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.356482 29969 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.357076 29969 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.345131 29990 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.366511 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:24.374624 28362 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:24.371333 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:24.377285 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1311) took 1031 ms. Trace:
I20260709 12:17:24.378680 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:24.377944 28435 rpcz_store.cc:276] 0709 12:17:23.345726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.363501 (+ 17775us) service_pool.cc:224] Handling call
0709 12:17:23.363769 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.154750 (+790981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.376841 (+222091us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b54002e2cd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:24.377170 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1805371776}
I20260709 12:17:24.323221 30000 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:24.357179 29730 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:24.355223 29652 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:24.391611 28364 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:24.392931 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:24.390674 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:24.395323 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:24.398375 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.399847 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.400365 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:24.400861 28366 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.404043 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:24.405962 30024 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:24.406492 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.406848 30024 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.407416 29988 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:24.409068 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.332933 30042 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.410271 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.411051 29988 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.412652 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:24.359341 29825 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:24.373863 30026 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:24.414410 28991 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:24.414521 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:24.358355 29888 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:24.359436 29969 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:24.349668 30015 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:24.375564 29929 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.410234 29652 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:24.428442 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:24.380839 30032 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
W20260709 12:17:24.430493 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.442348 28435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:24.460175 28435 raft_consensus.cc:606] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:24.381378 29868 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:24.460886 29868 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.463228 29868 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:24.382354 29795 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:24.446319 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:24.447649 28366 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
W20260709 12:17:24.450372 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48530.3us
I20260709 12:17:24.413092 30049 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:24.429179 29471 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.418442 30042 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.417353 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:24.457955 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.449318 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:24.421181 30000 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:17:24.390175 29996 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 12:17:24.478737 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:24.452849 28986 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:24.485119 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:24.481941 29993 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:24.482896 28284 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.451197 29825 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.487069 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:24.488422 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:24.451396 30026 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.495529 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1342) took 1135 ms. Trace:
I20260709 12:17:24.480182 30049 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.477538 28982 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:24.509126 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:24.509203 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:24.459785 30013 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:17:24.477303 28361 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:24.514200 29471 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:24.481343 30042 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.515619 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:24.515864 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.496676 28283 rpcz_store.cc:276] 0709 12:17:23.359676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.359791 (+   115us) service_pool.cc:224] Handling call
0709 12:17:23.414939 (+ 55148us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:23.430090 (+ 15151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:23.430122 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:23.442779 (+ 12657us) log.cc:844] Serialized 26 byte log entry
0709 12:17:23.592584 (+149805us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:23.812301 (+219717us) log.cc:844] Serialized 14 byte log entry
0709 12:17:24.483126 (+670825us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b18003c5260. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:17:24.483321 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.483397 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:24.483442 (+    45us) raft_consensus.cc:1685] finished
0709 12:17:24.495310 (+ 11868us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.495466 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4083,"prepare.queue_time_us":219478,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":4782,"raft.queue_time_us":8566,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":15481,"spinlock_wait_cycles":8667520,"thread_start_us":108198,"threads_started":1,"wal-append.queue_time_us":702442}
I20260709 12:17:24.522495 28432 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:24.459538 29929 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.533778 28284 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:24.534693 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1341) took 1175 ms. Trace:
I20260709 12:17:24.535110 28284 rpcz_store.cc:276] 0709 12:17:23.359394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.361924 (+  2530us) service_pool.cc:224] Handling call
0709 12:17:23.416962 (+ 55038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.482204 (+1065242us) spinlock_profiling.cc:243] Waited 94 ms on lock 0x7b5400510ca0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7fca1d 00007f85be7e31a4 00007f85be865447 00007f85be87dfe7 00007f85be87f7de 00007f85be889b49 00007f85be88a416 00007f85be886d8f 00007f85be88683e 00007f85c384b419
0709 12:17:24.534304 (+ 52100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.534615 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216305920}
I20260709 12:17:24.497560 29990 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:24.538108 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.538241 28360 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:24.465240 29795 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:24.518738 29471 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.523896 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:24.503067 29737 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:17:24.524392 30043 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.525229 29825 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:24.506944 30049 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:24.527114 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 289) took 1042 ms. Trace:
I20260709 12:17:24.527370 29996 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:24.529829 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 1395) took 1205 ms. Trace:
I20260709 12:17:24.494653 30037 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.523751 30029 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:24.541538 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:24.543133 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:24.494412 29814 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
W20260709 12:17:24.546444 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 169445us
I20260709 12:17:24.549345 28990 rpcz_store.cc:276] 0709 12:17:23.484736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.484893 (+   157us) service_pool.cc:224] Handling call
0709 12:17:23.503605 (+ 18712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.194680 (+691075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.526821 (+332141us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54001caf54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:24.527018 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586815872}
I20260709 12:17:24.552173 28438 rpcz_store.cc:276] 0709 12:17:23.324323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.324451 (+   128us) service_pool.cc:224] Handling call
0709 12:17:24.529603 (+1205152us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002e4ad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:24.529764 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441642368}
I20260709 12:17:24.553304 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:24.553553 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:24.599788 28285 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:24.531728 30000 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:24.554498 28440 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:17:24.600585 28440 pending_rounds.cc:85] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 4
I20260709 12:17:24.600862 28440 pending_rounds.cc:107] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:17:24.601117 28440 raft_consensus.cc:2889] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:24.553066 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:24.602167 28290 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:24.598486 29471 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:24.520912 29647 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:17:24.519254 29736 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:17:24.615612 29471 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:24.612542 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:24.615505 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 74072.7us
I20260709 12:17:24.616351 29471 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:24.608357 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:24.619635 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:24.620162 28290 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:24.621547 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:24.554150 30037 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.678602 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:24.678990 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 578013us
W20260709 12:17:24.679425 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 57953.1us
I20260709 12:17:24.551990 29990 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:24.681109 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:24.513799 29652 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.683981 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:24.685429 28285 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:24.520895 29993 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:17:24.689376 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:24.693961 28290 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:24.690665 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1352) took 1274 ms. Trace:
I20260709 12:17:24.685511 29471 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.556265 30043 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:24.694944 28292 rpcz_store.cc:276] 0709 12:17:23.416573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.150638 (+734065us) service_pool.cc:224] Handling call
0709 12:17:24.150981 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.151196 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:24.157003 (+  5807us) log.cc:844] Serialized 14 byte log entry
0709 12:17:24.689778 (+532775us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:24.689912 (+   134us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:24.689964 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.690048 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.690590 (+   542us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":310165,"prepare.queue_time_us":6125,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":435191,"thread_start_us":221809,"threads_started":2,"wal-append.queue_time_us":95189}
I20260709 12:17:24.554917 29775 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.130s
I20260709 12:17:24.715080 28438 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:24.714126 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:24.613622 29796 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:24.716650 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:24.717069 28435 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:24.720530 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.723788 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:24.692605 29992 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:24.685070 29990 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:24.727763 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.497856 30015 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.684755 30037 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.739176 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:24.732362 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:24.713028 29999 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:24.740155 28292 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.743932 28438 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:24.713925 29825 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.744169 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:24.745729 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:24.746546 28285 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.675539 29794 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:24.744787 28288 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:24.728427 29992 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.736217 29990 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:24.801162 28360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:24.806366 28360 raft_consensus.cc:606] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:17:24.806766 28360 raft_consensus.cc:614] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:24.809715 29471 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:24.849664 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 125977us
I20260709 12:17:24.736575 29652 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.854736 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:24.854554 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:24.855897 28433 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:24.814684 29471 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.726922 29796 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.614336 30029 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.804977 29992 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.711768 29795 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.883591 28292 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:24.815869 29868 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:24.890913 28433 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:24.752547 30032 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.121s
I20260709 12:17:24.852771 30000 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:24.875371 30026 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:24.794737 29825 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:24.913524 29471 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:24.895889 29868 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.814414 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:24.912595 29929 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:24.957369 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:24.951018 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.959955 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:24.904553 29777 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.973505 28285 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
W20260709 12:17:24.996403 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:24.961138 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 110430us
I20260709 12:17:24.895890 30030 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.401s
I20260709 12:17:24.936004 30052 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:24.996253 28433 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.943456 30035 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:25.010792 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:24.951236 29868 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.021158 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:25.169344 28404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.034s	sys 0.061s
W20260709 12:17:25.169862 28404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.035s	sys 0.061s
I20260709 12:17:25.008625 28983 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:25.170145 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 552855us
W20260709 12:17:25.176072 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:25.178433 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:25.178812 28439 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:25.018227 29775 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:25.180280 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:24.947650 30059 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:24.906147 30043 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:24.933189 30061 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.459s
I20260709 12:17:25.206409 29471 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:25.193972 30051 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.491s	user 0.001s	sys 0.007s
I20260709 12:17:25.214663 29471 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:25.195369 28985 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.018s	sys 0.022s
I20260709 12:17:25.018460 29777 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:25.224540 28433 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:25.010780 30060 consensus_queue.cc:1059] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.225s
I20260709 12:17:25.227164 28439 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:25.195569 30059 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:25.233321 29471 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:25.234229 29471 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:25.235697 29471 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:25.240576 28285 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
W20260709 12:17:25.237883 29842 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.004s	sys 0.003s
I20260709 12:17:25.239846 29777 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:25.230957 28985 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.018s	sys 0.023s
I20260709 12:17:25.206584 29775 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:25.233287 30042 consensus_meta.cc:287] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.002s
I20260709 12:17:25.213472 29736 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:17:25.175891 30000 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:25.270290 28286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.004s	sys 0.018s
I20260709 12:17:25.268646 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:25.220266 29993 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
W20260709 12:17:25.254896 29842 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.004s	sys 0.004s
I20260709 12:17:25.239861 30058 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:25.275938 28286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.004s	sys 0.018s
W20260709 12:17:25.279989 28364 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.008s	sys 0.020s
W20260709 12:17:25.283694 28364 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.008s	sys 0.020s
W20260709 12:17:25.286839 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 75030us
I20260709 12:17:25.262341 29794 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:25.305049 30064 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
I20260709 12:17:25.305846 30073 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 12:17:25.308137 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:25.305176 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:25.311007 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:25.324128 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:25.331822 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:25.332774 28366 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:17:25.333724 30056 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.492s	user 0.001s	sys 0.001s
I20260709 12:17:25.335642 30029 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:25.339115 29471 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:25.310909 28283 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:25.338127 30040 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:25.343528 30040 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:25.277280 30042 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:25.511603 30076 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.980285 29825 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:25.357925 30019 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:25.337601 29992 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:25.981163 30019 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:25.509779 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 131871us
I20260709 12:17:25.983877 30019 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:25.753337 30037 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.014s	user 0.011s	sys 0.000s
I20260709 12:17:25.509262 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:25.874183 28954 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.183s	user 0.029s	sys 0.028s
I20260709 12:17:25.988848 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1355) took 2569 ms. Trace:
W20260709 12:17:25.428658 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:25.805332 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:25.438614 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.037014 28366 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:25.805140 28407 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.010s	sys 0.028s
I20260709 12:17:25.424067 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:25.579680 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:26.034844 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:25.429001 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:26.037936 28407 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.014s	sys 0.032s
W20260709 12:17:25.979586 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 968877us
W20260709 12:17:26.020620 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:26.039891 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:25.988221 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:25.432008 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:26.040760 28285 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 4: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:26.041239 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1454) took 1181 ms. Trace:
I20260709 12:17:26.042807 28255 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:25.987198 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:25.286784 29796 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:26.040191 28287 rpcz_store.cc:276] 0709 12:17:23.419482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.281753 (+862271us) service_pool.cc:224] Handling call
0709 12:17:24.282106 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.604038 (+321932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.604080 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.604691 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:17:24.731753 (+127062us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:24.745459 (+ 13706us) log.cc:844] Serialized 14 byte log entry
0709 12:17:24.905109 (+159650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.905210 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:25.988575 (+1083365us) raft_consensus.cc:1685] finished
0709 12:17:25.988597 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.988756 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":796782,"prepare.queue_time_us":20844,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":643597,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":16099,"thread_start_us":126655,"threads_started":1,"wal-append.queue_time_us":278702}
W20260709 12:17:26.046151 28954 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.346s	user 0.032s	sys 0.045s
W20260709 12:17:26.044310 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:25.846987 29969 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:25.424966 29929 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:25.440191 30070 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:17:26.152891 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1456) took 1292 ms. Trace:
I20260709 12:17:26.152977 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:25.985198 29992 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.189177 28288 rpcz_store.cc:276] 0709 12:17:24.859984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.579266 (+719282us) service_pool.cc:224] Handling call
0709 12:17:26.152819 (+573553us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:25.346441 30072 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.340s	user 0.001s	sys 0.000s
W20260709 12:17:25.354869 30065 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:17:26.155351 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:25.748469 30068 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.730s	user 0.000s	sys 0.001s
W20260709 12:17:26.039047 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.157423 28292 rpcz_store.cc:276] 0709 12:17:24.859498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.431659 (+572161us) service_pool.cc:224] Handling call
0709 12:17:26.041187 (+609528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.171258 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:26.159484 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 172381us
I20260709 12:17:25.430603 29902 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:25.352155 30066 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.816s
I20260709 12:17:25.344449 30040 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:25.343819 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:26.174281 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1454) took 1665 ms. Trace:
I20260709 12:17:26.174995 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1346) took 2814 ms. Trace:
I20260709 12:17:26.180290 28438 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:26.172044 28190 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 216) took 3490 ms. Trace:
I20260709 12:17:26.164101 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.190128 30033 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:25.806003 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:25.948900 30081 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.987901 29795 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.427064 30060 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:25.359982 30057 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:25.358801 30030 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:25.987661 30025 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.043632 29888 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.149127 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 134) took 1596 ms. Trace:
W20260709 12:17:25.795708 29990 consensus_meta.cc:287] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.942s	user 0.012s	sys 0.000s
I20260709 12:17:26.205580 30089 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:25.928555 28983 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.000s
I20260709 12:17:25.727252 30087 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.345716 30077 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:25.430891 30082 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:17:25.609815 30083 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.262s	user 0.004s	sys 0.000s
I20260709 12:17:25.989254 30015 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.187225 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.188640 28433 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:26.243332 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1357) took 2821 ms. Trace:
W20260709 12:17:26.149047 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 1.46506e+06us
I20260709 12:17:25.714598 29639 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.712970 29898 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:25.353931 30049 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.757s	user 0.006s	sys 0.006s
I20260709 12:17:26.248922 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:26.195269 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.191488 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:26.042200 29998 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:26.239436 30089 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:25.985140 30037 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.266496 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1410) took 1536 ms. Trace:
I20260709 12:17:26.266942 28435 rpcz_store.cc:276] 0709 12:17:24.730599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.730722 (+   123us) service_pool.cc:224] Handling call
0709 12:17:24.731100 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.893314 (+162214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.893348 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.893911 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:17:25.184547 (+290636us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:25.184586 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.184659 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:25.609070 (+424411us) spinlock_profiling.cc:243] Waited 71.6 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:26.266182 (+657112us) raft_consensus.cc:1685] finished
0709 12:17:26.266210 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.266434 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120295,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":17298,"spinlock_wait_cycles":164780288,"thread_start_us":402927,"threads_started":2,"wal-append.queue_time_us":378562}
I20260709 12:17:26.268393 30037 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.196501 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.270092 28358 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:26.222924 30033 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.271057 28437 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:26.244043 30082 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.272464 28283 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.272886 30033 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:25.419441 30013 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.076968 29730 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.274088 29730 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.276081 29730 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.276484 28439 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "114fa146124747358c997ac1008e1b93"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33090
I20260709 12:17:26.273623 30013 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.259555 30049 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.244323 30087 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.279122 30049 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.279805 30049 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.255779 30077 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.260718 29998 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.273605 30082 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.280905 30087 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.283205 29998 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:26.259130 29898 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.285880 29898 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:25.426543 30010 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.714326 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 290) took 1111 ms. Trace:
W20260709 12:17:26.200388 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 155367us
I20260709 12:17:25.430040 29775 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.156216 30019 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:26.179625 29987 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:25.296447 30058 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.200781 28360 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:26.170568 28986 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:26.154632 30036 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.169023 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:26.203681 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.322458 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1459) took 1031 ms. Trace:
I20260709 12:17:25.990041 29814 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.212s
I20260709 12:17:26.156075 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:26.204996 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:26.325654 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 120747us
I20260709 12:17:26.326538 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.182893 29929 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:25.622459 29868 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.212368 28284 raft_consensus.cc:2410] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:26.210836 28287 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:26.209882 28364 rpcz_store.cc:276] 0709 12:17:24.508770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.510687 (+  1917us) service_pool.cc:224] Handling call
0709 12:17:24.510910 (+   223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.511213 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.511240 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.518032 (+  6792us) log.cc:844] Serialized 26 byte log entry
0709 12:17:25.284480 (+766448us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:25.284529 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.284617 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:26.174068 (+889451us) raft_consensus.cc:1685] finished
0709 12:17:26.174089 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.174220 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3297,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":33003,"thread_start_us":733997,"threads_started":1,"wal-append.queue_time_us":1477602}
I20260709 12:17:26.214696 28366 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:26.332818 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1449) took 1596 ms. Trace:
I20260709 12:17:26.333521 28284 rpcz_store.cc:276] 0709 12:17:24.736772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.805610 (+1068838us) service_pool.cc:224] Handling call
0709 12:17:26.332743 (+527133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.315629 30040 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.210414 28190 rpcz_store.cc:276] 0709 12:17:22.681510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.681798 (+   288us) service_pool.cc:224] Handling call
0709 12:17:26.171940 (+3490142us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:24.196055 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:24.264710 (+ 68655us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:24.264764 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:17:24.264785 (+    21us) tablet.cc:662] Decoding operations
0709 12:17:24.266286 (+  1501us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:24.266419 (+   133us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:24.266456 (+    37us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:17:24.267692 (+  1236us) tablet.cc:701] Row locks acquired
0709 12:17:24.267741 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:17:24.267924 (+   183us) write_op.cc:270] Start()
0709 12:17:24.268081 (+   157us) write_op.cc:276] Timestamp: P: 1783599444267851 usec, L: 0
0709 12:17:24.268101 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:17:24.268870 (+   769us) log.cc:844] Serialized 6467 byte log entry
0709 12:17:24.411830 (+142960us) op_driver.cc:464] REPLICATION: finished
0709 12:17:24.537827 (+125997us) write_op.cc:301] APPLY: starting
0709 12:17:24.547882 (+ 10055us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:24.729694 (+181812us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:24.746124 (+ 16430us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:25.307539 (+561415us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:25.307806 (+   267us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 12:17:25.307874 (+    68us) write_op.cc:312] APPLY: finished
0709 12:17:25.308199 (+   325us) log.cc:844] Serialized 113 byte log entry
0709 12:17:25.308388 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:25.308566 (+   178us) write_op.cc:454] Released schema lock
0709 12:17:25.308622 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:26.188616 (+879994us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:26.192606 (+  3990us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":911,"reported_tablets":23,"spinlock_wait_cycles":1408,"child_traces":[["op",{"apply.queue_time_us":101681,"apply.run_cpu_time_us":40764,"apply.run_wall_time_us":1575861,"num_ops":16,"prepare.queue_time_us":544172,"prepare.run_cpu_time_us":6299,"prepare.run_wall_time_us":107919,"raft.queue_time_us":105322,"raft.run_cpu_time_us":22201,"raft.run_wall_time_us":48324,"replication_time_us":143482,"thread_start_us":574090,"threads_started":4,"wal-append.queue_time_us":8471}]]}
I20260709 12:17:26.224352 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.342687 28288 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:26.343196 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1366) took 1610 ms. Trace:
I20260709 12:17:26.343493 28288 rpcz_store.cc:276] 0709 12:17:24.732484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.209189 (+1476705us) service_pool.cc:224] Handling call
0709 12:17:26.343132 (+133943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.224754 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1413) took 1409 ms. Trace:
I20260709 12:17:26.344336 28438 rpcz_store.cc:276] 0709 12:17:24.815043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.815242 (+   199us) service_pool.cc:224] Handling call
0709 12:17:24.815699 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.210104 (+1394405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.224505 (+ 14401us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540026a2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:26.224688 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3137869568}
W20260709 12:17:25.761605 29652 consensus_meta.cc:287] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.891s	user 0.004s	sys 0.006s
I20260709 12:17:26.205719 29902 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:26.226773 28436 rpcz_store.cc:276] 0709 12:17:24.398737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.398861 (+   124us) service_pool.cc:224] Handling call
0709 12:17:24.399224 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.442865 (+ 43641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.442898 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.455572 (+ 12674us) log.cc:844] Serialized 26 byte log entry
0709 12:17:24.540398 (+ 84826us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:24.540442 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.540520 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:25.337883 (+797363us) raft_consensus.cc:1685] finished
0709 12:17:25.337903 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.994754 (+656851us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2684,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":8441,"thread_start_us":62685,"threads_started":1,"wal-append.queue_time_us":223596}
I20260709 12:17:26.230028 29990 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:25.980402 30076 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.230381 28189 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 2 to 3, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:26.232956 30098 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:17:26.234570 29888 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.221603 30038 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.236789 30057 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.226169 30030 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:26.187388 29969 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.365950 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.224499 30060 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:26.222106 28359 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:26.243331 28285 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:26.193953 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:26.243872 28289 rpcz_store.cc:276] 0709 12:17:23.421599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.302037 (+880438us) service_pool.cc:224] Handling call
0709 12:17:24.302676 (+   639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.607157 (+304481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.607194 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.679927 (+ 72733us) log.cc:844] Serialized 26 byte log entry
0709 12:17:24.774878 (+ 94951us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:25.712017 (+937139us) spinlock_profiling.cc:243] Waited 7.98 ms on lock 0x7b180042c2e0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85c227f3c7 00007f85be8919a7 00007f85be8a10bf
0709 12:17:25.712224 (+   207us) log.cc:844] Serialized 14 byte log entry
0709 12:17:25.713286 (+  1062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.713391 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:26.241489 (+528098us) raft_consensus.cc:1685] finished
0709 12:17:26.243124 (+  1635us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.243265 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":471460,"prepare.queue_time_us":109157,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":425803,"raft.run_cpu_time_us":514,"raft.run_wall_time_us":37326,"spinlock_wait_cycles":18357632,"thread_start_us":261883,"threads_started":2,"wal-append.queue_time_us":193764}
I20260709 12:17:26.381835 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1463) took 1088 ms. Trace:
I20260709 12:17:26.222219 30081 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.382316 28285 rpcz_store.cc:276] 0709 12:17:25.293197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.041373 (+748176us) service_pool.cc:224] Handling call
0709 12:17:26.381726 (+340353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.075909 29507 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:26.240043 28983 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:26.262432 28433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:52316
I20260709 12:17:25.352110 30059 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.255520 29825 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.262267 29734 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.288161 28439 raft_consensus.cc:606] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Received request to transfer leadership
W20260709 12:17:26.314463 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 281587us
I20260709 12:17:26.398816 28439 raft_consensus.cc:614] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:26.266451 30089 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:26.318107 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 275527us
I20260709 12:17:26.402649 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.403132 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.279301 30066 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.404246 30066 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.290706 28985 rpcz_store.cc:276] 0709 12:17:24.411720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.413146 (+  1426us) service_pool.cc:224] Handling call
0709 12:17:24.413499 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.413777 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:24.489841 (+ 76064us) log.cc:844] Serialized 14 byte log entry
0709 12:17:25.294964 (+805123us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.300029 (+  5065us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:25.338597 (+ 38568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.422205 (+ 83608us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.515089 (+ 92884us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":106053,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":1811,"raft.queue_time_us":13417,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":15198,"thread_start_us":827026,"threads_started":2,"wal-append.queue_time_us":944330}
I20260709 12:17:26.406745 30066 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.322826 28292 rpcz_store.cc:276] 0709 12:17:25.291516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.203381 (+911865us) service_pool.cc:224] Handling call
0709 12:17:26.322390 (+119009us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.328388 28989 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:26.410567 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:26.411156 28289 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:26.318735 30072 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:17:26.327392 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:26.209897 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.210002 28291 rpcz_store.cc:276] 0709 12:17:23.360715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.127275 (+766560us) service_pool.cc:224] Handling call
0709 12:17:24.137784 (+ 10509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:24.674074 (+536290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.676603 (+  2529us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:24.686178 (+  9575us) log.cc:844] Serialized 26 byte log entry
0709 12:17:24.940135 (+253957us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:24.957521 (+ 17386us) log.cc:844] Serialized 14 byte log entry
0709 12:17:25.163053 (+205532us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4800080358. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c37b0aa1 00007f85c37a7b37 00007f85c37a79ca 00007f85c389f26f 00007f85c389ef37 00007f85c38b355c 00007f85c38b3508 00007f85c38b3488 00007f85c38b3443 00007f85c38b26ec 00007f85c27dca96
0709 12:17:25.350795 (+187742us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b1800151900. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:17:25.351017 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.351112 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:25.595327 (+244215us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b1800151900. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:26.174776 (+579449us) raft_consensus.cc:1685] finished
0709 12:17:26.174798 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.174921 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":565616,"prepare.queue_time_us":48304,"prepare.run_cpu_time_us":7609,"prepare.run_wall_time_us":108817,"raft.queue_time_us":858697,"raft.run_cpu_time_us":3142,"raft.run_wall_time_us":561120,"spinlock_wait_cycles":125620096,"thread_start_us":308498,"threads_started":2,"wal-append.queue_time_us":444384}
I20260709 12:17:26.332758 28991 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:17:26.334658 28441 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.005s	sys 0.003s
W20260709 12:17:26.417127 28441 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.006s	sys 0.003s
I20260709 12:17:26.417582 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:26.417901 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1316) took 1419 ms. Trace:
I20260709 12:17:26.394021 29652 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.422338 28291 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:26.422937 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1368) took 1690 ms. Trace:
I20260709 12:17:26.423249 28291 rpcz_store.cc:276] 0709 12:17:24.732953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.415065 (+1682112us) service_pool.cc:224] Handling call
0709 12:17:26.421998 (+  6933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.422700 (+   702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.422882 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.338079 28283 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:26.279148 30013 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:26.318187 30037 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.345602 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.359002 28432 raft_consensus.cc:2410] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:26.359514 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:26.434504 28288 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.361145 28189 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:26.330703 30042 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.334601 29999 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.364831 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.353853 30087 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.333428 29998 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:26.359272 30015 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.351330 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 297) took 1045 ms. Trace:
I20260709 12:17:26.370786 29929 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.505748 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1344) took 3134 ms. Trace:
I20260709 12:17:26.438480 30032 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.375646 29888 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.376050 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.508881 28436 raft_consensus.cc:2410] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:26.331246 30036 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.492336 29652 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:26.319998 30019 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:26.512137 30112 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:17:26.512537 30058 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.291908 30010 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.378727 29857 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:26.378441 30030 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.380366 30076 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.367560 30038 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.381021 28367 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:17:26.381891 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 188033us
I20260709 12:17:26.393693 28990 raft_consensus.cc:2368] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5.
I20260709 12:17:26.394233 28433 raft_consensus.cc:606] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:26.376173 28984 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:26.404260 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.496423 28986 rpcz_store.cc:276] 0709 12:17:25.305650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.305873 (+   223us) service_pool.cc:224] Handling call
0709 12:17:26.169956 (+864083us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:26.170223 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.321810 (+151587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.351240 (+ 29430us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104866048}
I20260709 12:17:26.408643 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1367) took 1675 ms. Trace:
I20260709 12:17:26.332818 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1464) took 1039 ms. Trace:
W20260709 12:17:26.418848 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 91519.2us
I20260709 12:17:26.419121 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.416587 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 294) took 1127 ms. Trace:
W20260709 12:17:26.435504 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 107829us
I20260709 12:17:26.397280 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1458) took 1089 ms. Trace:
I20260709 12:17:26.507164 30015 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.528563 28359 rpcz_store.cc:276] 0709 12:17:25.307981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.309584 (+  1603us) service_pool.cc:224] Handling call
0709 12:17:26.395182 (+1085598us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.519358 30089 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.438802 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.418293 28441 rpcz_store.cc:276] 0709 12:17:24.998964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.999106 (+   142us) service_pool.cc:224] Handling call
0709 12:17:25.002307 (+  3201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.002568 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:25.622144 (+619576us) log.cc:844] Serialized 14 byte log entry
0709 12:17:26.417531 (+795387us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:26.417585 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:26.417629 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.417694 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.417845 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":432737,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":1362,"thread_start_us":1254832,"threads_started":2,"wal-append.queue_time_us":757349}
I20260709 12:17:26.529877 30089 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.497314 29998 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:26.530874 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 92163.3us
I20260709 12:17:26.531674 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.532042 30089 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.529136 29652 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.484786 28289 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
W20260709 12:17:26.533461 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:26.479734 28283 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1365) took 1708 ms (client timeout 1897 ms). Trace:
I20260709 12:17:26.435665 28189 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
W20260709 12:17:26.413560 28983 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 369) took 1951 ms (client timeout 2101 ms). Trace:
I20260709 12:17:26.435164 29775 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.424597 30063 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.422765 29814 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:26.499583 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 250797us
I20260709 12:17:26.502692 30059 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.513247 30019 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:26.505110 28432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 1405) took 1791 ms (client timeout 2139 ms). Trace:
I20260709 12:17:26.371886 30077 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:26.507248 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 107045us
I20260709 12:17:26.400051 29507 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.444927 30057 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:26.377590 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1457) took 1069 ms. Trace:
I20260709 12:17:26.412381 29825 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.410506 29734 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.552632 28435 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:26.552814 29825 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.506155 28286 rpcz_store.cc:276] 0709 12:17:23.360234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.110861 (+750627us) service_pool.cc:224] Handling call
0709 12:17:24.111223 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:25.283061 (+1171838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.283097 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:25.306060 (+ 22963us) log.cc:844] Serialized 26 byte log entry
0709 12:17:26.201476 (+895416us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:26.413256 (+211780us) log.cc:844] Serialized 14 byte log entry
0709 12:17:26.494673 (+ 81417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.494760 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:26.494806 (+    46us) raft_consensus.cc:1685] finished
0709 12:17:26.494819 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.494965 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":657179,"prepare.queue_time_us":68963,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":578125,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":73767,"thread_start_us":1210874,"threads_started":3,"wal-append.queue_time_us":975815}
I20260709 12:17:26.542680 29775 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.553772 29775 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.554059 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:26.554745 28286 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.555037 29814 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:26.556098 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.556262 29814 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.512995 30036 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.496543 30087 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.441876 30037 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.568990 29990 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.520828 28433 raft_consensus.cc:614] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:26.521771 30119 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:26.436430 30060 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.522159 29996 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.532500 29998 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:26.533990 30117 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.534334 28289 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:26.589490 30123 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:17:26.590243 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1447) took 1854 ms. Trace:
I20260709 12:17:26.590631 28289 rpcz_store.cc:276] 0709 12:17:24.736051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.533631 (+1797580us) service_pool.cc:224] Handling call
0709 12:17:26.533974 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.589921 (+ 55947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.590174 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.534538 29903 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.591642 29999 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:26.534664 28291 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:26.490335 29902 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.592252 29999 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.592876 29902 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.595667 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1445) took 1862 ms. Trace:
I20260709 12:17:26.596072 28291 rpcz_store.cc:276] 0709 12:17:24.735463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.424022 (+1688559us) service_pool.cc:224] Handling call
0709 12:17:26.534483 (+110461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.594512 (+ 60029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.595432 (+   920us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b54001af250. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:26.595587 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127587072}
I20260709 12:17:26.535552 28986 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:17:26.597956 29749 raft_consensus.cc:1068] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Unable to promote non-voter f526369c0d0743b192be6e205540a059: Illegal state: Replica 9aea2391841b4888b4e195e5a1f4623b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:17:26.592329 30082 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:26.468384 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.598781 30117 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.598984 30082 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.601217 30117 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.523967 29857 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:26.536134 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:26.602139 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 66073.8us
I20260709 12:17:26.602716 29996 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:26.536275 28439 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:26.524060 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1459) took 1215 ms. Trace:
I20260709 12:17:26.480880 28284 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:26.533588 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:26.523855 28287 rpcz_store.cc:276] 0709 12:17:25.293599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.149644 (+856045us) service_pool.cc:224] Handling call
0709 12:17:26.332739 (+183095us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.534425 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:26.527751 28987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 298) took 1221 ms. Trace:
W20260709 12:17:26.538918 28283 rpcz_store.cc:269] 0709 12:17:24.732211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.420432 (+688221us) service_pool.cc:224] Handling call
0709 12:17:26.440662 (+1020230us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b540009e2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:26.440785 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538150272}
I20260709 12:17:26.536859 30121 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.535311 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:26.523393 28292 rpcz_store.cc:276] 0709 12:17:24.732718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.407704 (+1674986us) service_pool.cc:224] Handling call
0709 12:17:26.408102 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.408312 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.408339 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:26.408385 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:26.408415 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.408466 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.408580 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.539743 28189 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 2 to 3, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:26.525969 30010 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.639171 28985 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:26.640359 29990 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.641011 29990 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.643252 29999 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:26.644299 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.644150 30121 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.646530 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.605108 30058 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.648546 30058 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.649035 30058 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.637550 30111 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:17:26.651489 30111 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:26.651871 30111 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:26.602356 30119 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:26.637442 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.636929 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.542678 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:26.542558 28432 rpcz_store.cc:269] 0709 12:17:24.713673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.713824 (+   151us) service_pool.cc:224] Handling call
0709 12:17:26.504846 (+1791022us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540026b1d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:26.505032 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3777304960}
I20260709 12:17:26.523765 29868 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.540618 30089 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.524506 28989 rpcz_store.cc:276] 0709 12:17:25.289000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.295324 (+  6324us) service_pool.cc:224] Handling call
0709 12:17:26.168828 (+873504us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762e9c 00007f85bb762a5b 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:26.416506 (+247678us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695181696}
I20260709 12:17:26.548887 29507 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:26.676738 30089 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.550202 30077 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:26.648460 29857 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.677894 30077 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:26.508942 30103 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:17:26.533942 30109 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.678284 30119 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:26.518806 29929 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.554414 28285 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:17:26.557554 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:26.680425 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.545433 28363 rpcz_store.cc:276] 0709 12:17:25.307686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.310202 (+  2516us) service_pool.cc:224] Handling call
0709 12:17:25.310545 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:25.346333 (+ 35788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.346368 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:25.346868 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:17:25.795703 (+448835us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:25.795743 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.795819 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:26.377375 (+581556us) raft_consensus.cc:1685] finished
0709 12:17:26.377394 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.377527 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10811,"raft.queue_time_us":12063,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":90,"thread_start_us":448438,"threads_started":1,"wal-append.queue_time_us":827883}
I20260709 12:17:26.530220 30015 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.535590 30076 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.523671 30038 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.513724 30032 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.565171 28288 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:26.692232 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1462) took 1399 ms. Trace:
I20260709 12:17:26.565483 28286 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:26.571687 29734 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.573364 30081 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.573844 30020 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.695662 30020 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.696362 30036 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:17:26.540607 28983 rpcz_store.cc:269] 0709 12:17:24.457129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.491600 (+ 34471us) service_pool.cc:224] Handling call
0709 12:17:26.398203 (+1906603us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.695588 30121 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.697036 30036 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.699447 30036 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.699527 29999 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:26.701152 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 167626us
I20260709 12:17:26.579814 29969 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.560276 30019 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:17:26.702073 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:26.570705 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 295) took 1275 ms. Trace:
I20260709 12:17:26.583657 30087 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.583514 29775 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.703522 28990 rpcz_store.cc:276] 0709 12:17:25.295678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.295948 (+   270us) service_pool.cc:224] Handling call
0709 12:17:26.570480 (+1274532us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54006e2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:26.570636 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":812886144}
I20260709 12:17:26.704262 30036 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:26.597383 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.606904 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:26.607148 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.705394 28435 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.705901 29775 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.591735 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.618913 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.614697 28987 rpcz_store.cc:276] 0709 12:17:25.306020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.306203 (+   183us) service_pool.cc:224] Handling call
0709 12:17:26.173212 (+867009us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:26.173408 (+   196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.175883 (+  2475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.175917 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:26.183921 (+  8004us) log.cc:844] Serialized 26 byte log entry
0709 12:17:26.355749 (+171828us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:26.355798 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.355889 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:26.527425 (+171536us) raft_consensus.cc:1685] finished
0709 12:17:26.527452 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.527685 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26186,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6291,"spinlock_wait_cycles":86873344,"thread_start_us":148918,"threads_started":1,"wal-append.queue_time_us":279977}
I20260709 12:17:26.592218 28984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 370) took 1231 ms. Trace:
I20260709 12:17:26.706491 30030 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.623881 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:26.707492 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 83712.6us
I20260709 12:17:26.636920 30082 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.636929 28189 catalog_manager.cc:5697] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 2, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 2 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:26.652791 30111 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.720188 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:26.726815 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:26.728134 28189 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:26.676064 29825 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.692637 28288 rpcz_store.cc:276] 0709 12:17:25.292885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.344154 (+1051269us) service_pool.cc:224] Handling call
0709 12:17:26.692151 (+347997us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.734354 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.691669 29507 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.694629 29996 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:26.695066 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 80117.6us
I20260709 12:17:26.703850 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1364) took 1971 ms. Trace:
I20260709 12:17:26.607442 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.632731 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.708591 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.708727 28283 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:26.695101 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.711385 28289 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:26.723332 28286 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:17:26.723498 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.615628 30042 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 12:17:26.725811 28436 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:26.777554 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1451) took 1987 ms. Trace:
I20260709 12:17:26.781145 28283 rpcz_store.cc:276] 0709 12:17:24.790157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.708117 (+1917960us) service_pool.cc:224] Handling call
0709 12:17:26.708501 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.777183 (+ 68682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.777465 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.539886 30106 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:17:26.729761 29999 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:26.730974 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.691591 30109 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:26.771171 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.737001 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:26.761085 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.766952 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:26.737447 28285 rpcz_store.cc:276] 0709 12:17:24.731963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.397522 (+1665559us) service_pool.cc:224] Handling call
0709 12:17:26.418942 (+ 21420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.679103 (+260161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.703597 (+ 24494us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54000f23d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:26.703784 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310476032}
I20260709 12:17:26.776149 28289 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:26.682962 30104 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.615232 30025 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 12:17:26.750813 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:26.782984 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 74443.6us
I20260709 12:17:26.774535 28286 raft_consensus.cc:606] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.767635 29987 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:26.724284 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 181700us
I20260709 12:17:26.728752 28365 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:26.815032 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:26.776990 29990 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:26.785620 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:26.692001 30077 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:26.729872 28189 catalog_manager.cc:5697] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 2, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:26.790607 28437 raft_consensus.cc:2410] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:26.682744 30096 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:26.716883 29795 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.804411 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1457) took 1944 ms. Trace:
I20260709 12:17:26.806365 28292 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:26.809324 28286 raft_consensus.cc:614] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:26.776786 28434 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:26.822324 28289 rpcz_store.cc:276] 0709 12:17:24.860234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.775321 (+1915087us) service_pool.cc:224] Handling call
0709 12:17:26.775677 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.804053 (+ 28376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.804326 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:26.810848 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 73929.7us
I20260709 12:17:26.733685 30117 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:17:26.813941 28255 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.828825 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.808132 29734 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:26.816751 28291 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:26.795405 30087 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:26.789372 30082 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.778277 29507 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.717281 30076 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.725837 30038 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 12:17:26.817855 28189 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:26.844394 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:26.812618 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:26.816759 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 81780.6us
I20260709 12:17:26.830391 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1460) took 1538 ms. Trace:
I20260709 12:17:26.792569 30111 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.831926 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:26.915786 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 83906.6us
I20260709 12:17:26.916566 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.818511 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:26.779743 30119 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.814586 29987 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:26.833925 30125 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.736359 30043 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.904995 28189 catalog_manager.cc:5697] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:26.909122 28435 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:26.706841 28984 rpcz_store.cc:276] 0709 12:17:25.361230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.425893 (+ 64663us) service_pool.cc:224] Handling call
0709 12:17:26.209067 (+783174us) spinlock_profiling.cc:243] Waited 9.04 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:26.223288 (+ 14221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.535232 (+311944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.591913 (+ 56681us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540048fb54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:26.592146 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334900352}
I20260709 12:17:26.823612 30081 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.912189 28286 rpcz_store.cc:276] 0709 12:17:25.292208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.829529 (+1537321us) service_pool.cc:224] Handling call
0709 12:17:26.829897 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.830094 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.830120 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:26.830153 (+    33us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:26.830172 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.830217 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.830330 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.935328 28286 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:26.936074 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1370) took 1579 ms. Trace:
I20260709 12:17:26.936411 28286 rpcz_store.cc:276] 0709 12:17:25.357073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.934710 (+1577637us) service_pool.cc:224] Handling call
0709 12:17:26.935105 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.935787 (+   682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.936004 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.816599 30059 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:17:26.817679 30042 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:26.839552 29775 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:26.845624 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:26.945590 28284 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.941761 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:26.903836 30121 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:17:26.815526 30109 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.843951 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:26.964013 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 120172us
I20260709 12:17:26.964113 29996 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:26.964584 29996 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:26.965008 29996 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:26.965260 30109 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:26.833141 30104 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.958290 30042 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:26.966464 30060 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.966691 30015 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:26.966804 30042 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:26.968827 28283 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:26.946714 28363 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:26.965516 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:26.970489 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:26.928498 28189 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:26.930624 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:26.929766 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:26.938647 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 120179us
I20260709 12:17:26.972358 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:26.939913 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.841706 30082 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.834023 29734 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:26.942193 28291 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:26.915650 30019 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.931747 29998 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:26.948684 30129 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:26.854490 30087 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:26.949594 30124 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:17:26.954624 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:26.958161 30089 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:26.613090 28361 rpcz_store.cc:276] 0709 12:17:25.308268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.343364 (+ 35096us) service_pool.cc:224] Handling call
0709 12:17:25.987631 (+644267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:25.988304 (+   673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:25.988344 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:26.200323 (+211979us) log.cc:844] Serialized 26 byte log entry
0709 12:17:26.321723 (+121400us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:26.329873 (+  8150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.329965 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:26.523813 (+193848us) raft_consensus.cc:1685] finished
0709 12:17:26.523836 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.523990 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":83178,"threads_started":1,"wal-append.queue_time_us":193516}
I20260709 12:17:26.941023 29990 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:26.959494 29507 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:26.962119 29987 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.992115 29749 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.030998 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:26.973202 28189 catalog_manager.cc:5697] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:26.922909 29999 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:26.924160 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:26.979004 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:26.927771 30125 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.979974 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.032488 28363 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:26.979889 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.034206 30125 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.984202 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:26.931150 30106 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:26.986146 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:27.037206 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 53071.3us
I20260709 12:17:27.037456 28437 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.826489 30096 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:26.984206 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1461) took 1691 ms. Trace:
I20260709 12:17:26.941800 30081 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.990532 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:27.038209 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.033497 28435 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.038110 28291 rpcz_store.cc:276] 0709 12:17:25.292558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.918553 (+1625995us) service_pool.cc:224] Handling call
0709 12:17:26.932411 (+ 13858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.983935 (+ 51524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.984139 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:26.980091 30117 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:17:27.003664 28284 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:27.010131 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.012877 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.011616 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:27.047930 28363 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:27.048166 28433 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:27.015797 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:26.978542 29796 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.021502 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:26.982383 30082 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.985496 29969 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.024632 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:26.940012 30043 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:26.973687 28290 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:27.074364 29969 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.987389 29775 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:27.074707 28358 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:17:27.004983 30121 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:27.079118 28432 raft_consensus.cc:2410] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:27.040019 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:26.994874 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.080019 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:27.080638 28291 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:27.040868 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 50430.4us
I20260709 12:17:27.010687 30129 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:27.049793 28366 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:27.047701 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:27.044540 28189 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:27.026961 29857 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:27.043417 30081 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.085520 28189 catalog_manager.cc:5697] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:27.044464 29990 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:27.087242 28189 catalog_manager.cc:5697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:27.069613 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:26.922376 30077 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.030961 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:27.080581 28434 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:27.082587 28284 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.075505 29775 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.083034 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:27.103084 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:27.042382 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:27.104643 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.078598 29825 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:27.092860 28189 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 3 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:27.053416 30019 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.190897 30019 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.191311 30019 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.191984 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:27.195457 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.055117 30131 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:27.084661 29857 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:27.074853 29969 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.097414 28441 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:27.112154 30077 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.112872 30129 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.116686 30015 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.117250 28286 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:27.118463 30115 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:27.120268 29999 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:27.124408 30128 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.183s	user 0.002s	sys 0.000s
I20260709 12:17:27.126199 28984 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:17:27.127689 30043 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.134008 29749 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.180459 30125 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:27.134855 30130 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:17:27.135113 29902 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.000s
I20260709 12:17:27.085841 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.233987 28984 pending_rounds.cc:85] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 4
I20260709 12:17:27.138265 29990 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.230593 30037 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.139688 30104 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.139695 30076 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.082173 30106 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.146984 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:27.149101 28439 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:27.052771 29734 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.159106 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:27.166909 30032 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:27.167559 28290 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1448) took 2431 ms (client timeout 2123 ms). Trace:
W20260709 12:17:27.264678 28290 rpcz_store.cc:269] 0709 12:17:24.736313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.428690 (+692377us) service_pool.cc:224] Handling call
0709 12:17:27.167366 (+1738676us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b540035f2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:27.167483 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1349414784}
W20260709 12:17:27.171696 28365 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1433) took 3683 ms (client timeout 2111 ms). Trace:
W20260709 12:17:27.265906 28365 rpcz_store.cc:269] 0709 12:17:23.488154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.488276 (+   122us) service_pool.cc:224] Handling call
0709 12:17:27.171445 (+3683169us) spinlock_profiling.cc:243] Waited 3.2 s on lock 0x7b540027e554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:27.171580 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7348759296}
I20260709 12:17:27.172513 28362 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:27.129563 30087 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.173775 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.267043 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.267571 28436 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:27.268323 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.176148 30020 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.179309 30119 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:27.183869 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.184280 28288 raft_consensus.cc:2410] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:27.184222 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:17:27.185843 28255 connection.cc:256] RPC call timeout handler was delayed by -1.8447392099999433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1912s for a response
W20260709 12:17:27.272423 28255 connection.cc:256] RPC call timeout handler was delayed by -1.8195392099999026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260709 12:17:27.272797 28255 connection.cc:256] RPC call timeout handler was delayed by -1.7448392100000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20230000000000004s for a response
W20260709 12:17:27.273166 28255 connection.cc:256] RPC call timeout handler was delayed by -1.708839209999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
I20260709 12:17:27.274127 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:27.275269 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:27.276368 28255 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.161727 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.188591 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.189157 29795 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.021701 30058 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:27.189368 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.196853 28284 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:27.200464 28437 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:27.277155 28289 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:27.221980 28291 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:27.211179 29825 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:27.217916 29929 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.238543 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.240247 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.217427 30131 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.246766 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1450) took 2509 ms. Trace:
I20260709 12:17:27.238085 30129 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.138036 30096 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.131819 29996 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:27.249001 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 179490us
I20260709 12:17:27.252549 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.088423 30121 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.253306 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.232576 30115 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:27.249838 28984 pending_rounds.cc:107] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:17:27.258250 28364 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:17:27.259867 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 112971us
I20260709 12:17:27.269507 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.280673 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.303972 28290 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:27.286047 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.286913 28435 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:17:27.288955 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 104839us
I20260709 12:17:27.290306 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.290498 28287 rpcz_store.cc:276] 0709 12:17:24.740894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.692776 (+1951882us) service_pool.cc:224] Handling call
0709 12:17:26.741135 (+ 48359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.775952 (+ 34817us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.775994 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:26.776611 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 12:17:26.919476 (+142865us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:26.919516 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.919588 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:27.246532 (+326944us) raft_consensus.cc:1685] finished
0709 12:17:27.246554 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.246694 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34303,"raft.queue_time_us":35313,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":13187,"thread_start_us":100618,"threads_started":1,"wal-append.queue_time_us":214917}
I20260709 12:17:27.307108 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:27.308317 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 131) took 1063 ms. Trace:
I20260709 12:17:27.306207 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:27.308794 28284 rpcz_store.cc:276] 0709 12:17:26.245285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.307406 (+1062121us) service_pool.cc:224] Handling call
0709 12:17:27.307766 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.307958 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.307985 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.308032 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:27.308064 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.308117 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.308248 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:27.308704 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.310968 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:27.290366 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:27.292934 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.315896 28363 raft_consensus.cc:2410] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:27.264688 30037 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.292507 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.320128 28432 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:27.320437 28358 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:27.295269 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.321024 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:27.321475 28291 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:27.290812 30032 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.321702 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:27.247926 29857 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.326009 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.328238 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1372) took 1348 ms. Trace:
I20260709 12:17:27.328644 28289 rpcz_store.cc:276] 0709 12:17:25.980328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.327421 (+1347093us) service_pool.cc:224] Handling call
0709 12:17:27.327749 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.327937 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.327961 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.328006 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:27.328033 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.328084 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.328183 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.330451 28284 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:27.331012 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1465) took 1014 ms. Trace:
I20260709 12:17:27.331257 28284 rpcz_store.cc:276] 0709 12:17:26.316384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.329845 (+1013461us) service_pool.cc:224] Handling call
0709 12:17:27.330169 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.330769 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.330961 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.334630 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.335479 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.260749 30104 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.340631 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.337286 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.295851 30087 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:27.297935 30010 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:27.300500 30141 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.341706 28360 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:27.347393 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:27.360841 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:27.361359 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.323160 30089 raft_consensus.cc:2764] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:27.324085 28984 raft_consensus.cc:2889] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:27.297780 29903 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.349458 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.287657 29795 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.332744 28361 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 1.
I20260709 12:17:27.327260 30020 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:27.385047 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.326897 29775 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.326737 30066 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:27.365443 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:27.365787 28404 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 2.111s (SENT) [suppressed 15 similar messages]
I20260709 12:17:27.336602 29902 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 12:17:27.328678 30123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.020s	sys 0.027s
I20260709 12:17:27.366892 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:27.336364 30122 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.296371 29929 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.338054 30030 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:27.371093 30146 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:17:27.370748 30082 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.387369 29730 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:27.372262 30143 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:17:27.372696 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:27.395427 30122 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.336555 29749 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.362737 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.348385 29857 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:27.364887 30139 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:17:27.325443 30096 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.385692 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:27.386258 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.359314 30060 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:27.365443 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.368964 29814 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:27.336874 29652 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:17:27.365594 30087 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:27.395432 28987 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:27.368177 30010 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:27.398501 28283 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.347901 30015 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:17:27.401994 28290 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:27.365200 30043 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:17:27.372040 30131 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.414031 30122 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.388460 30066 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.413442 29749 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:27.367993 30141 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.415738 28287 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:17:27.416227 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 50487.7us
I20260709 12:17:27.378655 29825 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.391571 30020 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.423192 28292 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
W20260709 12:17:27.429682 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 44051.9us
I20260709 12:17:27.438833 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:27.438994 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.439781 28292 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
W20260709 12:17:27.407686 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 86746.7us
W20260709 12:17:27.348120 30140 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:17:27.441869 30151 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.405150 30030 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.443765 30141 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.445125 30104 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.394731 30115 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.439651 30066 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.446931 28283 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:27.447953 30060 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.445029 30030 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.440789 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.415971 28983 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:27.450950 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.451608 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:27.451488 30087 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.452857 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.449312 28358 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:27.427738 29814 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.455885 28292 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:27.458446 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.459921 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:27.457060 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1369) took 2100 ms. Trace:
I20260709 12:17:27.434851 29730 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.476150 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.471809 28285 rpcz_store.cc:276] 0709 12:17:25.356723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.919772 (+1563049us) service_pool.cc:224] Handling call
0709 12:17:26.940863 (+ 21091us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.941218 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.941244 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:27.164887 (+223643us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b2c00032f88. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:17:27.165320 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:17:27.261488 (+ 96168us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:27.381354 (+119866us) log.cc:844] Serialized 14 byte log entry
0709 12:17:27.381600 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.381668 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:27.452856 (+ 71188us) raft_consensus.cc:1685] finished
0709 12:17:27.456834 (+  3978us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.456996 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":159649,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":4862,"raft.queue_time_us":4489,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":19960,"spinlock_wait_cycles":6001408,"thread_start_us":186519,"threads_started":2,"wal-append.queue_time_us":199061}
I20260709 12:17:27.476790 28292 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:27.477545 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.435882 29652 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:27.456461 29990 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.452348 29749 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.482638 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:27.486799 28404 leader_election.cc:400] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5.
I20260709 12:17:27.487185 28404 leader_election.cc:403] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:27.459122 29814 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.489787 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:27.490446 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.456666 29996 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.493870 28292 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.480990 29902 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.480569 30043 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:17:27.483240 29990 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.481634 29652 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:27.492367 30025 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.485267 30121 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.474400 30030 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:17:27.479595 30106 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.504638 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.492532 30033 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5.
I20260709 12:17:27.507354 28285 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:27.515753 28292 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:27.505724 30010 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.522819 30082 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.502859 29990 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.524250 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.535426 30019 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.536882 28284 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:27.542913 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.545109 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.546406 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.548267 30066 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.550709 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.502661 30025 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:27.553642 28437 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:27.556865 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.557147 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1374) took 1025 ms. Trace:
I20260709 12:17:27.570180 30028 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.570780 30028 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.573154 30025 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.573311 29969 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.575546 29969 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:27.525071 30145 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:17:27.518761 30058 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.573123 30028 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.580054 30081 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.537616 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1468) took 1220 ms. Trace:
I20260709 12:17:27.581667 28365 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:27.581840 29795 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.583194 30020 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.583316 28989 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:27.562762 28292 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:27.585173 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.586941 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.582866 28284 rpcz_store.cc:276] 0709 12:17:26.317199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.536342 (+1219143us) service_pool.cc:224] Handling call
0709 12:17:27.536698 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.537338 (+   640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.537556 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:27.588302 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 100070us
I20260709 12:17:27.589241 28284 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:27.589681 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:27.589890 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1378) took 1052 ms. Trace:
I20260709 12:17:27.590308 28284 rpcz_store.cc:276] 0709 12:17:26.537883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.588645 (+1050762us) service_pool.cc:224] Handling call
0709 12:17:27.588999 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.589597 (+   598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.589832 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.591734 28284 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:27.566676 28285 rpcz_store.cc:276] 0709 12:17:26.531875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.556210 (+1024335us) service_pool.cc:224] Handling call
0709 12:17:27.556581 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.556783 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.556812 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.556852 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:27.556885 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.556956 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.557075 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.595847 29749 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.596171 30122 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.597015 28434 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.597668 30033 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:27.598155 30033 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:27.598584 30033 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.601127 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:27.599195 30096 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.586009 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1467) took 1269 ms. Trace:
I20260709 12:17:27.603029 28292 rpcz_store.cc:276] 0709 12:17:26.316940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.525801 (+1208861us) service_pool.cc:224] Handling call
0709 12:17:27.562490 (+ 36689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.585662 (+ 23172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.585944 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.609314 28285 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:27.612178 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.613514 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:27.609239 28987 raft_consensus.cc:2393] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:27.615659 28292 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:27.609517 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:27.620857 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.625703 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:27.626515 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.629184 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:27.597977 30019 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.631609 28292 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:27.631757 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:27.635800 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:27.636852 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:27.638808 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.639626 28363 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:27.640385 30015 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:27.640861 30015 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.641198 30015 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:27.643075 28238 auto_rebalancer.cc:893] Removing replace marker failed: Replica 9aea2391841b4888b4e195e5a1f4623b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:17:27.643693 28238 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:17:27.643999 28238 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:17:27.622356 29999 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.648123 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:27.649444 28285 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.646087 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.651044 28363 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 1.
I20260709 12:17:27.629892 30043 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.654529 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.655138 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.656015 28363 raft_consensus.cc:2410] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:27.659458 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.660112 28363 raft_consensus.cc:2376] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4. Re-sending same reply.
I20260709 12:17:27.660941 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.650135 30081 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.648778 29999 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:27.666821 30066 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.668511 28285 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:27.668972 28365 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:27.654973 30043 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.668653 29999 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.645721 30121 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
W20260709 12:17:27.672233 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.680342 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.676656 29749 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.639081 30089 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
W20260709 12:17:27.680298 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43532.2us
I20260709 12:17:27.675794 29902 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.682054 28363 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:27.686791 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:27.662375 30157 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:27.688863 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:27.670125 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.690994 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.692497 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.679405 30043 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.693671 28363 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:27.682256 30025 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.691053 29749 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.699679 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:27.690671 30157 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.689906 30015 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.701617 30157 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:27.701781 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.704484 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.703480 30033 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.708766 30033 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.709529 29902 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:27.709995 29902 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.710351 29902 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.711809 30025 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.713268 30125 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.717821 30122 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.720253 28363 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:27.722975 28363 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:27.719594 30033 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:27.720484 30025 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:17:27.710490 29990 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.729539 28987 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:27.742640 28363 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:27.736109 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:27.744189 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.745451 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.725950 29749 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:17:27.752743 28361 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:27.753173 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:27.726822 30015 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.717443 29814 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.729117 30096 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:17:27.742308 30111 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.719310 29902 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.756073 28360 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:27.731859 30168 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.757725 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:27.750711 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:27.760924 30111 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.766746 29814 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.757532 30058 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:27.767369 29902 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.761325 30168 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.754895 30025 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.751704 30033 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:27.719774 30166 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:17:27.756035 30164 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:27.786278 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:27.759241 28361 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:27.777074 30042 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:27.778812 30168 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.777673 30087 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.801965 30033 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.804548 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:27.778028 30165 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:17:27.796198 29902 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:17:27.808470 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.823879 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:27.789076 29749 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
W20260709 12:17:27.830368 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.779268 30058 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.824837 29990 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.806190 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:27.837263 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:27.828372 30015 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.830070 30096 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:17:27.838984 29902 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:27.841209 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:27.843217 28983 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:27.843391 30111 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.836733 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.787695 30164 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:27.844365 30164 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.845337 28292 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.853135 29999 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.856952 30157 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:27.859463 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:27.863291 28435 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:27.864665 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:27.865249 28435 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.865562 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:27.865806 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.866822 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.867830 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.866631 29999 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.869074 29999 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:27.865302 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.873416 28437 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.868739 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.836074 30087 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.878736 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:27.880307 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.883879 28433 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:27.869846 29999 consensus_queue.cc:1243] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:27.884678 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.864809 30096 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:27.891271 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.900066 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.902210 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.871481 30019 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.895931 28432 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:27.908862 28435 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:27.878265 30037 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.879392 30028 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:27.910740 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:27.885565 30030 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.902595 28292 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:27.870098 30157 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.879215 29795 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.887286 30082 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.930189 30019 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:27.931456 30030 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:27.870800 29749 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:17:27.931887 30030 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:27.932507 29749 consensus_queue.cc:1243] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer f526369c0d0743b192be6e205540a059 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:17:27.933491 28330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.205s	user 0.033s	sys 0.013s
I20260709 12:17:27.933200 30030 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.928248 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:27.896749 29730 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:27.935729 29730 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.935886 30106 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:27.936491 30106 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:27.937412 30106 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.940169 30168 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:27.903337 29775 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:27.931861 30171 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:17:27.910909 30060 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.904290 30104 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:27.887348 29814 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:17:27.922900 30028 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.959247 28436 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:17:27.959255 28330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.231s	user 0.034s	sys 0.013s
W20260709 12:17:27.960827 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 1.23257e+06us
I20260709 12:17:27.944782 30129 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:27.945981 29730 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.911266 30096 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.973594 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1465) took 2427 ms. Trace:
I20260709 12:17:27.974036 28362 rpcz_store.cc:276] 0709 12:17:25.546501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.075625 (+529124us) service_pool.cc:224] Handling call
0709 12:17:27.172117 (+1096492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.267028 (+ 94911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.973394 (+706366us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540027e550. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:27.973534 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2520702336}
I20260709 12:17:27.975384 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.976153 28291 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.976591 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:27.968638 29814 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:27.968241 30129 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:27.977218 29814 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:27.925241 30037 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.978188 29814 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:27.966742 30123 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:17:27.977226 28983 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:27.962855 30028 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.981267 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:27.982533 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.982530 30040 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.983358 30040 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.983479 28361 raft_consensus.cc:2410] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:27.983865 30157 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:27.967923 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:27.968528 30058 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.972911 29929 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:27.973145 30043 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.975462 30096 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:27.986507 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:27.989353 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.989806 30076 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:27.990211 28358 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:27.985813 30111 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:27.990644 30111 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:27.991317 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.991544 30111 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.992822 30111 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:27.993357 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.987445 30040 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.994802 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:27.996088 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:27.996723 28359 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:27.997622 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:27.998322 28436 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:27.998093 30129 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.003310 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.003605 29730 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.004112 28363 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.990367 30076 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:28.006701 30076 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:27.991971 28358 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:28.014019 28285 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:27.985788 30087 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.979350 30037 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:27.981012 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.018699 28289 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:28.019945 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:27.979460 29814 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.022150 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:27.987901 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:28.015309 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.059844 28285 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.028913 30043 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:28.034068 30164 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.036955 30168 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:28.053133 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.047112 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:28.055604 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1371) took 2075 ms. Trace:
I20260709 12:17:28.064008 28286 rpcz_store.cc:276] 0709 12:17:25.979836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.944166 (+964330us) service_pool.cc:224] Handling call
0709 12:17:26.944504 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.220219 (+275715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.047407 (+827188us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54002409d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:28.055509 (+  8102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396427520}
I20260709 12:17:28.065028 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.065593 28286 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.057998 28289 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.972278 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:28.070330 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 82502.4us
W20260709 12:17:28.070999 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45197.6us
I20260709 12:17:28.071203 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:28.072966 28285 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:28.074239 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.074996 28285 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.075708 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:28.083175 28285 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:28.084450 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.085093 28285 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.033556 30028 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.114974 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:28.116159 28363 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:28.117543 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:28.117476 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 210349us
I20260709 12:17:28.118757 28366 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.120011 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:28.120398 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.121162 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:28.123698 28285 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:28.124701 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.087564 30129 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.123185 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:28.091324 30037 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:28.121220 28363 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.127911 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:28.129487 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 57211.4us
I20260709 12:17:28.130928 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:28.067857 30040 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.115526 29868 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.181457 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:28.136619 28289 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:28.185637 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.185777 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:28.186527 28289 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:28.187003 30081 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.187649 30081 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:28.129559 30037 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.135854 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:28.125548 30115 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.157212 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.158329 28363 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:28.137252 28286 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:28.128819 30060 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:28.129473 30119 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.202634 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.202957 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.204221 28363 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.204615 28358 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.157075 29929 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.125571 30104 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:28.126927 30168 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:28.153136 29775 raft_consensus.cc:2764] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:17:28.195484 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 59731.9us
I20260709 12:17:28.187815 28289 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:28.125835 30087 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:17:28.158129 29999 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:28.209911 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.195524 30037 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.203522 30060 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:28.213702 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.198046 28366 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:28.217556 28363 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:28.244649 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.227131 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:28.244972 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.240201 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.240392 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.246861 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:28.240906 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.248373 28435 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.248909 28363 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:28.223722 30187 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:17:28.259265 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.211810 30185 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.257918 28435 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:28.263886 28438 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:28.261457 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:28.270160 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.270932 28438 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:28.272445 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:28.218322 30115 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:17:28.243788 30179 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.112s	user 0.000s	sys 0.002s
W20260709 12:17:28.244029 30182 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.055s	user 0.001s	sys 0.001s
I20260709 12:17:28.250240 29999 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.262163 30060 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.267850 28363 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:28.215560 30119 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:28.213156 30104 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.256376 28358 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
W20260709 12:17:28.304068 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.304455 30119 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.305294 28358 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:28.285905 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:28.273136 28366 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:28.251590 30176 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.308081 30176 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.310397 30176 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:28.277244 30185 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.249600 30168 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:28.311949 30028 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.312177 30168 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.304023 30104 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:28.313040 30185 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:28.299628 30188 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:17:28.313668 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.261866 30129 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.314514 30129 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:28.315475 30129 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.290364 30081 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.319092 28358 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:28.235178 30106 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:17:28.321789 30115 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.322973 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:28.294852 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.323696 28990 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.224093 29929 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.260567 28983 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:28.311045 30121 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.234549 30123 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 12:17:28.331619 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.351125 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.302348 30137 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.031s	user 0.007s	sys 0.001s
I20260709 12:17:28.267678 29730 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.312788 30028 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:28.341954 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 160600us
W20260709 12:17:28.350955 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 89604us
W20260709 12:17:28.298771 30183 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:17:28.357084 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:28.358083 28436 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:17:28.362643 30148 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.972s	user 0.008s	sys 0.005s
I20260709 12:17:28.366021 30028 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.366797 29987 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.368439 30028 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.373119 30030 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.330884 29999 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.290858 29795 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.386240 28990 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:28.386950 28286 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:28.310559 30164 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:28.339653 29825 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:28.430279 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1444) took 1286 ms. Trace:
I20260709 12:17:28.430511 29825 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.431137 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.393095 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56626 (request call id 157) took 1380 ms. Trace:
I20260709 12:17:28.430835 28440 rpcz_store.cc:276] 0709 12:17:27.144136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.220044 (+ 75908us) service_pool.cc:224] Handling call
0709 12:17:27.235879 (+ 15835us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:27.236334 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.236365 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:27.236945 (+   580us) log.cc:844] Serialized 52 byte log entry
0709 12:17:27.287207 (+ 50262us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:27.287250 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.287324 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:28.361970 (+1074646us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:28.429826 (+ 67856us) raft_consensus.cc:1685] finished
0709 12:17:28.429857 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.430195 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1984,"raft.queue_time_us":1089,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":6225,"spinlock_wait_cycles":9364736,"thread_start_us":23740,"threads_started":1,"wal-append.queue_time_us":123677}
W20260709 12:17:28.392298 28441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1334) took 1669 ms (client timeout 2064 ms). Trace:
W20260709 12:17:28.433545 28441 rpcz_store.cc:269] 0709 12:17:26.723034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.723187 (+   153us) service_pool.cc:224] Handling call
0709 12:17:28.392105 (+1668918us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54002db4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:28.392228 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":744215936}
I20260709 12:17:28.395113 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:28.434332 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:28.435317 28986 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:28.400573 28291 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:28.433264 28364 rpcz_store.cc:276] 0709 12:17:27.012391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.012552 (+   161us) service_pool.cc:224] Handling call
0709 12:17:28.392886 (+1380334us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54001d8154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:28.393036 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478449024}
W20260709 12:17:28.403494 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.387754 30122 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.410547 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 153) took 1482 ms. Trace:
I20260709 12:17:28.392598 30106 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:17:28.365581 30104 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.415925 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:28.416770 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:28.417469 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:28.389436 30123 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 12:17:28.391036 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 154650us
I20260709 12:17:28.388351 30040 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:28.443573 28291 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:28.445878 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:28.456775 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.438346 28367 rpcz_store.cc:276] 0709 12:17:26.927799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.929011 (+  1212us) service_pool.cc:224] Handling call
0709 12:17:26.944872 (+ 15861us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:26.945197 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.945226 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:26.955494 (+ 10268us) log.cc:844] Serialized 26 byte log entry
0709 12:17:27.048089 (+ 92595us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:27.048131 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.048200 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:28.267046 (+1218846us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:28.403821 (+136775us) raft_consensus.cc:1685] finished
0709 12:17:28.403865 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.410468 (+  6603us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73665,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":17666,"spinlock_wait_cycles":4397952,"thread_start_us":92164,"threads_started":1,"wal-append.queue_time_us":274757}
I20260709 12:17:28.458768 28291 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:28.412616 30197 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:17:28.427906 30185 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:28.436111 30087 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:28.464509 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.445413 29795 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.478300 30197 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.478132 29795 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:28.478860 30197 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:28.480118 30197 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.455500 30037 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.484486 30037 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.485030 28983 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:28.468925 30087 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.486313 30087 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.486996 30194 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:28.487711 30104 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.445557 30076 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.491119 30020 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.491576 30020 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:28.492403 30020 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.494486 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.466655 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:28.495019 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1474) took 1137 ms. Trace:
I20260709 12:17:28.495481 28360 rpcz_store.cc:276] 0709 12:17:27.357896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.367636 (+  9740us) service_pool.cc:224] Handling call
0709 12:17:28.494743 (+1127107us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b5400125f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:28.494904 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768781184}
I20260709 12:17:28.496039 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:28.496663 28956 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:28.447445 29868 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:28.470976 28986 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:28.447993 30122 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.472148 29929 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:28.472700 29825 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.472848 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1484) took 1097 ms. Trace:
I20260709 12:17:28.467916 28285 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:28.481467 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.503420 28291 raft_consensus.cc:2393] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:28.493468 30020 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.481595 30197 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:28.552103 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 145555us
W20260709 12:17:28.543696 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 61597.1us
I20260709 12:17:28.551704 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.553879 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:28.480885 29795 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.555791 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:28.500348 30196 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.559458 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:28.561165 28364 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:28.564949 28367 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:28.565833 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.566474 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:28.498509 30185 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.623709 30185 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.558202 30104 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:28.624744 30104 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:28.625837 30104 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.582095 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.634367 28291 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:28.583153 30193 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:28.583503 29730 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:28.637859 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.608819 30186 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.402s	user 0.001s	sys 0.000s
W20260709 12:17:28.609930 28283 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 137) took 1621 ms (client timeout 1910 ms). Trace:
W20260709 12:17:28.640872 28283 rpcz_store.cc:269] 0709 12:17:26.988320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.191686 (+203366us) service_pool.cc:224] Handling call
0709 12:17:28.609632 (+1417946us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540009f1d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:28.609846 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473469312}
I20260709 12:17:28.642905 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:28.611078 29996 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:28.645510 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 148916us
I20260709 12:17:28.612375 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 141) took 1559 ms. Trace:
I20260709 12:17:28.613574 30164 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:28.614048 29868 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.526906 28359 rpcz_store.cc:276] 0709 12:17:27.375083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.997426 (+622343us) service_pool.cc:224] Handling call
0709 12:17:28.025194 (+ 27768us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.120017 (+ 94823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.126659 (+  6642us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:28.224258 (+ 97599us) log.cc:844] Serialized 26 byte log entry
0709 12:17:28.427846 (+203588us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:28.440976 (+ 13130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.445053 (+  4077us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:28.472527 (+ 27474us) raft_consensus.cc:1685] finished
0709 12:17:28.472560 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.472757 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19726,"raft.queue_time_us":20127,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":3754,"thread_start_us":125846,"threads_started":1,"wal-append.queue_time_us":198293}
I20260709 12:17:28.560532 30028 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:28.567675 30195 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:17:28.620956 30194 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.352048 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:28.611743 28288 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:28.648181 28290 rpcz_store.cc:276] 0709 12:17:27.053274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.325684 (+272410us) service_pool.cc:224] Handling call
0709 12:17:28.612174 (+1286490us) spinlock_profiling.cc:243] Waited 75.3 ms on lock 0x7b54000f2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:28.612310 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173164928}
I20260709 12:17:28.649993 30164 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.569586 29795 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.654693 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.658658 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:28.654914 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.656785 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.659420 28291 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:28.661208 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1466) took 2344 ms. Trace:
I20260709 12:17:28.654616 28956 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:28.661634 28287 rpcz_store.cc:276] 0709 12:17:26.316687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.344158 (+1027471us) service_pool.cc:224] Handling call
0709 12:17:27.367708 (+ 23550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:27.438246 (+ 70538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.660932 (+1222686us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b5400163d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:28.661140 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109775744}
I20260709 12:17:28.648468 29825 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:28.645875 29996 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:28.667047 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.653489 30195 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:28.670253 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.670989 28283 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:28.672039 28440 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:28.674952 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:28.675494 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:28.676177 28291 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:28.677173 28283 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:28.646077 30060 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.665918 29990 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:28.648216 29730 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.683128 28283 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:28.687346 28283 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:28.649492 29999 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.688200 28366 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:28.673934 30195 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:28.688227 30087 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.690770 30195 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.692066 30195 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.674808 29825 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:17:28.738858 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.696442 28289 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:28.706378 30087 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.684401 29990 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.741684 28360 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:17:28.740051 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.753773 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.743543 30115 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.741318 30081 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:28.749490 29990 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.751616 30205 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.756994 30192 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:17:28.794466 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.830152 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 140132us
I20260709 12:17:28.836498 28284 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:28.796694 29868 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:17:28.838420 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1469) took 1488 ms. Trace:
I20260709 12:17:28.840111 28284 rpcz_store.cc:276] 0709 12:17:27.350110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.592788 (+242678us) service_pool.cc:224] Handling call
0709 12:17:28.836143 (+1243355us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b5800000098. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762eee 000000000036eb39 00007f85c37d61e0 00007f85c389f0c4 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:28.836310 (+   167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:28.838116 (+  1806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.838340 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10145408}
I20260709 12:17:28.761404 29652 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:28.855644 30207 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:17:28.856390 29730 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.857801 30082 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.858218 30082 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:28.858134 28284 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:28.860075 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:28.862839 30210 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:17:28.864661 30020 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:28.864890 30201 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:17:28.855690 29652 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.859086 30082 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.869309 30082 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.840660 29825 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:28.871578 28365 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1482) took 1497 ms (client timeout 1824 ms). Trace:
W20260709 12:17:28.872027 28365 rpcz_store.cc:269] 0709 12:17:27.374567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.586672 (+212105us) service_pool.cc:224] Handling call
0709 12:17:28.871376 (+1284704us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b54001d7754. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:28.871507 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186977152}
I20260709 12:17:28.872385 28362 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:28.873366 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.875885 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.775523 30205 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:28.837628 30104 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:28.876600 28440 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:28.873522 28365 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:28.881078 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:28.865244 30020 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:28.879592 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.885100 30202 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:17:28.886260 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1499) took 1043 ms. Trace:
I20260709 12:17:28.886695 28363 rpcz_store.cc:276] 0709 12:17:27.842650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.324261 (+481611us) service_pool.cc:224] Handling call
0709 12:17:28.324611 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.324965 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.324996 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:28.325531 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:17:28.544001 (+218470us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:28.544042 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.544117 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:28.886030 (+341913us) raft_consensus.cc:1685] finished
0709 12:17:28.886050 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.886195 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6386,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":14460,"thread_start_us":218059,"threads_started":1,"wal-append.queue_time_us":349473}
I20260709 12:17:28.802163 30192 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:28.889369 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:28.902096 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:28.890443 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:28.891031 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:28.904909 28363 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:28.891185 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.848306 30164 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:28.886415 30200 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:17:28.889925 30104 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:28.883473 30203 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:17:28.905053 28362 raft_consensus.cc:2368] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5.
I20260709 12:17:28.910666 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:28.910027 28365 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:28.861756 29999 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.889065 30060 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.905951 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.869530 30122 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:28.916236 28364 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:28.921397 28255 leader_election.cc:400] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5.
I20260709 12:17:28.921890 28255 leader_election.cc:403] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:28.908399 30028 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.921540 30217 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:28.926975 30194 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.897694 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:28.921857 29730 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:28.929782 30010 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.400s	user 0.017s	sys 0.026s
I20260709 12:17:28.937791 28987 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:28.929258 30217 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.940228 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1498) took 1099 ms. Trace:
I20260709 12:17:28.907143 30019 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:17:28.919505 30087 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:17:28.944375 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1500) took 1101 ms. Trace:
I20260709 12:17:28.935674 29730 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.944196 28358 rpcz_store.cc:276] 0709 12:17:27.841015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.319907 (+478892us) service_pool.cc:224] Handling call
0709 12:17:28.320290 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.320669 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.320698 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:28.322072 (+  1374us) log.cc:844] Serialized 26 byte log entry
0709 12:17:28.550723 (+228651us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:28.550786 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.550868 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:28.939993 (+389125us) raft_consensus.cc:1685] finished
0709 12:17:28.940015 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.940158 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":650,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":227188,"threads_started":1,"wal-append.queue_time_us":358579}
I20260709 12:17:28.942437 30205 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:28.935842 28359 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.946255 28360 rpcz_store.cc:276] 0709 12:17:27.842917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.496265 (+653348us) service_pool.cc:224] Handling call
0709 12:17:28.496600 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.749918 (+253318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.944054 (+194136us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54003bb5d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:28.944300 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563023616}
W20260709 12:17:28.939421 30010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.411s	user 0.017s	sys 0.026s
W20260709 12:17:28.942318 28989 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 213) took 1930 ms (client timeout 2101 ms). Trace:
I20260709 12:17:28.956002 30081 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:28.958189 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:28.959357 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.964684 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.952312 29730 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.970168 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.972126 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:28.985934 28432 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:28.986756 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:28.974066 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:28.969363 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.944796 30217 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:28.990885 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.991711 28435 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:28.951411 30019 consensus_queue.cc:1243] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:28.978574 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.995599 28439 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:17:28.974740 28989 rpcz_store.cc:269] 0709 12:17:27.011906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.013584 (+  1678us) service_pool.cc:224] Handling call
0709 12:17:27.158885 (+145301us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:28.935623 (+1776738us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54005bfed4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:28.942215 (+  6592us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738105984}
I20260709 12:17:28.980852 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:28.997696 28432 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:28.974341 30060 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.950387 30028 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.983150 28441 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:28.968394 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:28.994477 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:28.982585 30081 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.005856 30191 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.000809 30060 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.003077 30028 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:29.015771 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.018392 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.003891 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1502) took 1021 ms. Trace:
I20260709 12:17:29.006409 30191 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:29.026154 30191 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.028817 29996 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.051348 29996 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.032980 28434 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:17:29.053429 28434 pending_rounds.cc:85] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
I20260709 12:17:29.053727 28434 pending_rounds.cc:107] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:17:29.053998 28434 raft_consensus.cc:2889] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.055950 30194 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5.
I20260709 12:17:29.039135 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.034919 29734 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.041164 30028 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.058435 30028 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:29.044780 30211 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:17:29.060391 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:29.023453 28361 rpcz_store.cc:276] 0709 12:17:27.982211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.984163 (+  1952us) service_pool.cc:224] Handling call
0709 12:17:28.958010 (+973847us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b5800113798. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:29.003050 (+ 45040us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90952576}
I20260709 12:17:29.047008 30020 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.062494 30020 raft_consensus.cc:2764] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:17:29.062793 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.047587 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1497) took 1254 ms. Trace:
I20260709 12:17:29.063328 28285 rpcz_store.cc:276] 0709 12:17:27.793447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.156859 (+363412us) service_pool.cc:224] Handling call
0709 12:17:29.047325 (+890466us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540020bcd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:29.047511 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624378240}
I20260709 12:17:29.064989 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1494) took 1272 ms. Trace:
I20260709 12:17:29.065388 28289 rpcz_store.cc:276] 0709 12:17:27.792430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.315601 (+523171us) service_pool.cc:224] Handling call
0709 12:17:28.316032 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.739869 (+423837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.064706 (+324837us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54002418d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:29.064922 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874266880}
I20260709 12:17:29.042052 30104 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.078042 28987 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:29.078456 28255 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:29.079787 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.079999 30104 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:29.080585 30104 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:29.080999 30104 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.081179 30111 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:29.081652 30111 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:29.081326 29990 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:29.050041 30223 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:17:29.071197 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.082599 30111 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.082924 28989 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:29.071373 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:29.072011 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:29.084841 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.085330 29652 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.085860 29652 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.088105 29652 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:29.070950 28330 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.073777 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1495) took 1280 ms. Trace:
I20260709 12:17:29.074244 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.074726 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:29.088912 28366 rpcz_store.cc:276] 0709 12:17:27.792934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.306960 (+514026us) service_pool.cc:224] Handling call
0709 12:17:28.307287 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:28.689486 (+382199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.073461 (+383975us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400127d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:29.073684 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875543296}
W20260709 12:17:29.077061 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.090540 30205 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:29.082335 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.056772 30191 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:29.091460 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.083606 30111 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.059026 30028 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.092602 30015 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:29.092795 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:29.093453 29652 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:29.093235 30015 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.094065 29652 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.094514 30028 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.095583 30015 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:29.093149 30221 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.227s	user 0.003s	sys 0.000s
I20260709 12:17:29.097318 30164 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:29.097892 30164 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:29.098843 30164 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.102272 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.091092 29814 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:17:29.103302 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:29.103763 28986 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:29.103343 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.103669 29814 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.104931 29814 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.105283 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.096146 30192 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.107018 30020 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.107755 30020 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:29.103924 29987 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.085098 28987 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 2 LEARNER]: Advancing to term 4
I20260709 12:17:29.109930 30162 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:29.110316 30058 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.110446 30162 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:29.110951 30162 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.110884 30058 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.016162 28257 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.113042 30058 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:29.113894 30082 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:29.114714 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 98679.7us
I20260709 12:17:29.115635 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.109915 29987 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:29.117017 28257 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.116834 29987 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.118695 29987 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.109305 30020 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:17:29.118075 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:29.121011 28983 raft_consensus.cc:2376] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4. Re-sending same reply.
I20260709 12:17:29.121362 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:29.119629 30162 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.096205 29652 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:29.124523 30058 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:29.096110 30015 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:29.088510 30060 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:29.117625 30082 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.126247 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.134260 30058 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:29.134869 30058 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.128535 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.126397 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:29.138823 30208 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.335s	user 0.003s	sys 0.001s
I20260709 12:17:29.139020 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:29.139537 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1373) took 2609 ms. Trace:
I20260709 12:17:29.140511 29652 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.140110 30162 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:29.144411 30162 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:29.146212 29990 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:29.148409 30213 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.272s	user 0.001s	sys 0.001s
I20260709 12:17:29.149839 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1498) took 1326 ms. Trace:
I20260709 12:17:29.150208 28286 rpcz_store.cc:276] 0709 12:17:27.823203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.432149 (+608946us) service_pool.cc:224] Handling call
0709 12:17:28.432484 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.453264 (+ 20780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.453318 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:28.454067 (+   749us) log.cc:844] Serialized 26 byte log entry
0709 12:17:28.756775 (+302708us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:28.756839 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.756934 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:29.147862 (+390928us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:29.149487 (+  1625us) raft_consensus.cc:1685] finished
0709 12:17:29.149510 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:29.149722 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28586,"raft.queue_time_us":5049,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":10669,"spinlock_wait_cycles":8224128,"thread_start_us":302237,"threads_started":1,"wal-append.queue_time_us":421503}
I20260709 12:17:29.139928 28288 rpcz_store.cc:276] 0709 12:17:26.529954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.541638 (+1011684us) service_pool.cc:224] Handling call
0709 12:17:28.611482 (+1069844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.656014 (+ 44532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.139334 (+483320us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540009f1d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:29.139472 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2459450112}
I20260709 12:17:29.150723 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.153544 30015 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.154735 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:29.155737 30089 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.156301 30089 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:29.156879 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.157501 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.167822 30058 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.168417 28367 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:29.168823 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:29.171044 30234 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:29.172590 30234 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.157703 30162 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.157420 30089 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:29.176147 28287 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:29.176390 30162 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.176762 30162 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:29.159125 30231 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:29.177260 28288 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:29.160118 28432 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:29.161211 30205 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.178177 30206 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.178742 30206 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:29.161775 29652 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:29.162344 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.161163 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:29.179610 28437 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:29.161902 30036 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.164479 30121 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.179736 30206 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.180626 30121 consensus_queue.cc:1243] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:29.163055 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:29.180372 30036 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.182247 28441 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.156265 30015 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:29.183627 30015 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.183822 30036 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.184768 28286 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:29.165947 28439 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:29.166996 28438 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:29.175694 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:29.178423 30205 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.189710 30205 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.187647 28290 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.193782 30205 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:29.195385 30076 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:29.195794 30076 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:29.196697 30076 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.197885 30076 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.198405 29652 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.126309 30028 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.183966 30015 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:29.203311 29825 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:29.141880 30020 consensus_queue.cc:1243] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer f526369c0d0743b192be6e205540a059 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:17:29.212924 28290 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:29.212791 29825 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:29.213985 30212 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:29.177819 30231 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.215358 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.216025 30196 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:29.216593 28364 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:29.217240 30219 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:29.217832 28363 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:29.260344 28363 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:29.219353 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 81249.5us
I20260709 12:17:29.219821 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.220885 29868 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:29.165359 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:29.222368 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.219255 28366 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:29.203075 30015 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.226528 28433 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:29.227586 28432 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:29.230608 28359 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:29.231530 29990 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:29.235323 30217 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.234948 28190 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 1 to 3, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 3 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:29.236129 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 87204.2us
I20260709 12:17:29.237762 30162 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:29.238317 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.284801 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.241866 30081 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.292914 30217 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.294329 30081 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.244626 28441 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:29.245397 28439 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:29.247215 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.248807 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.298085 30162 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.253265 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:29.254771 28987 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 4 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:29.177907 30058 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:29.259461 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 89133us
I20260709 12:17:29.303642 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:17:29.266517 30198 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.015s	sys 0.003s
I20260709 12:17:29.261538 28358 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
W20260709 12:17:29.269539 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.271210 28363 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.271548 29730 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:29.306082 28363 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.306685 29730 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.275272 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:29.275528 28438 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:29.307746 28440 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:29.308933 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:29.309517 28440 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.313167 28363 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:17:29.313637 28363 pending_rounds.cc:85] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 5
I20260709 12:17:29.289819 28190 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 1 to 2, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:29.291901 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.314659 29996 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.285279 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:29.256951 30231 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:29.316411 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:29.318082 28190 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 3 to 4, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 4 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:29.322479 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 69303us
W20260709 12:17:29.323304 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 158037us
I20260709 12:17:29.319407 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:29.320523 28440 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:29.327696 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:29.324671 28358 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.336833 28987 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:29.263931 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:29.304448 30198 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.015s	sys 0.004s
I20260709 12:17:29.291319 30060 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:29.340878 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:29.344254 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.344691 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.346403 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:29.347865 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:29.354264 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 90419.5us
I20260709 12:17:29.355332 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.356202 28363 pending_rounds.cc:107] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:17:29.356608 28363 raft_consensus.cc:2889] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:29.353595 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 68327.2us
I20260709 12:17:29.360710 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:29.362578 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.270342 30194 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:17:29.373812 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.374612 28435 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:29.377511 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:29.345309 29996 raft_consensus.cc:2764] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:17:29.352195 30122 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.351401 30193 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:29.365911 30106 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:17:29.391625 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.400527 28433 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:29.401648 28283 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:29.349102 29730 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:29.392201 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.387319 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:29.416157 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.418774 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.364344 28189 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 235) took 6287 ms. Trace:
I20260709 12:17:29.363310 30197 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.362658 29990 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:29.393707 30205 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.375660 30010 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.379712 30036 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:17:29.363315 29999 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.395637 30192 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.353499 29814 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.359166 30058 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:29.397877 28434 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:17:29.459286 28434 pending_rounds.cc:85] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 2
I20260709 12:17:29.459525 28434 pending_rounds.cc:107] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:29.459726 28434 raft_consensus.cc:2889] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.363567 29987 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:29.385636 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:29.365890 30019 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.405110 28284 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:29.413579 28292 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:29.418294 29730 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:29.423457 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:29.319830 30231 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.426004 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:29.454833 30232 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:17:29.383764 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:29.352710 30060 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.365449 29652 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:17:29.429611 28189 rpcz_store.cc:276] 0709 12:17:23.068607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.089835 (+ 21228us) service_pool.cc:224] Handling call
0709 12:17:29.356147 (+6266312us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:27.381159 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:27.449859 (+ 68700us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:27.449897 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:17:27.449910 (+    13us) tablet.cc:662] Decoding operations
0709 12:17:27.451273 (+  1363us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:27.451335 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:27.451366 (+    31us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:17:27.524965 (+ 73599us) tablet.cc:701] Row locks acquired
0709 12:17:27.524992 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:17:27.584180 (+ 59188us) write_op.cc:270] Start()
0709 12:17:27.584308 (+   128us) write_op.cc:276] Timestamp: P: 1783599447584122 usec, L: 0
0709 12:17:27.584332 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:17:27.593407 (+  9075us) log.cc:844] Serialized 5899 byte log entry
0709 12:17:27.759955 (+166548us) op_driver.cc:464] REPLICATION: finished
0709 12:17:27.829618 (+ 69663us) write_op.cc:301] APPLY: starting
0709 12:17:27.829723 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:27.836615 (+  6892us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:27.837395 (+   780us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:27.844739 (+  7344us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:27.845049 (+   310us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 12:17:27.845096 (+    47us) write_op.cc:312] APPLY: finished
0709 12:17:27.910575 (+ 65479us) log.cc:844] Serialized 107 byte log entry
0709 12:17:27.944328 (+ 33753us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:27.955338 (+ 11010us) write_op.cc:454] Released schema lock
0709 12:17:27.961910 (+  6572us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:27.968418 (+  6508us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:27.968452 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":66512,"apply.run_cpu_time_us":5532,"apply.run_wall_time_us":138796,"num_ops":15,"prepare.queue_time_us":112781,"prepare.run_cpu_time_us":49535,"prepare.run_wall_time_us":292173,"raft.queue_time_us":39056,"raft.run_cpu_time_us":20580,"raft.run_wall_time_us":29089,"replication_time_us":175465,"thread_start_us":150353,"threads_started":4,"wal-append.queue_time_us":97691,"wal-append.run_cpu_time_us":19850,"wal-append.run_wall_time_us":1280006}]]}
I20260709 12:17:29.435530 30015 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.436683 29990 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:29.440070 28283 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:29.439292 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.518180 28290 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:29.519379 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:29.520143 28290 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:29.449785 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:29.452610 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:29.422072 30122 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:29.379112 30198 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.082s
I20260709 12:17:29.394269 30119 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:17:29.462848 28292 pending_rounds.cc:85] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 1
I20260709 12:17:29.532027 28292 pending_rounds.cc:107] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:29.532338 28292 raft_consensus.cc:2889] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.533517 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.363710 30076 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:17:29.462889 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 77346.2us
I20260709 12:17:29.421778 30193 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.432178 30125 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.472255 28363 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:29.515694 30019 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:29.471284 28358 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
W20260709 12:17:29.495615 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 72281.9us
I20260709 12:17:29.472393 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:29.406507 30020 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:17:29.450078 30087 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:29.502455 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 118791us
I20260709 12:17:29.575994 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.385507 30235 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 12:17:29.515619 29734 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.460244 30168 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.516193 30231 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.460552 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:29.526592 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.462342 29987 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.461925 30058 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.497822 29730 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.497465 30033 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.497552 30111 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.586436 30081 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.499459 30192 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:29.550864 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.516801 30212 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.554374 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:29.500020 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 80373.4us
I20260709 12:17:29.298085 29795 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:29.592136 28366 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:17:29.591207 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.570464 30019 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.523154 30164 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:17:29.604920 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:29.608034 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 260254us
I20260709 12:17:29.613178 30122 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.622959 28291 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:29.608958 30058 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:29.466822 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.616067 28289 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:29.630038 28291 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:29.630085 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:17:29.619354 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.629325 28290 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:17:29.619190 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 80338.4us
I20260709 12:17:29.621374 28366 pending_rounds.cc:85] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 4
W20260709 12:17:29.619830 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 65560us
I20260709 12:17:29.625128 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.609457 29987 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.607048 28359 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.585508 30195 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:17:29.569083 30076 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.587033 30087 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.653785 28285 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:29.639986 28436 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:29.640547 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.591076 29734 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.637281 28366 pending_rounds.cc:107] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:29.656867 28366 raft_consensus.cc:2889] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.657219 28289 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:29.638340 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.637005 28290 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:29.639055 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:29.625839 29990 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.625629 30125 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.648902 28358 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:17:29.662871 28358 pending_rounds.cc:85] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 2
I20260709 12:17:29.663146 28358 pending_rounds.cc:107] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:29.663398 28358 raft_consensus.cc:2889] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.626231 30019 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.615566 30217 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:17:29.614547 30196 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:17:29.598694 30168 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.636395 30010 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:17:29.655853 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.660770 30119 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:29.686700 30119 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.687121 30119 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.687322 30235 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:29.688232 30235 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:29.688570 30235 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:29.661348 30241 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:17:29.637584 30194 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:29.654994 30104 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 12:17:29.690817 30104 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.692749 28292 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:29.635528 30256 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:29.642773 30197 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.662532 30082 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.674367 30076 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.675618 30205 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.676072 30087 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.677834 30226 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.679713 30191 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:29.701925 30191 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:29.689437 30235 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.703085 30235 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.702880 30191 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.704312 30191 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.682257 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:29.683583 30247 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.167s	user 0.003s	sys 0.000s
I20260709 12:17:29.686036 29734 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:29.698673 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:29.699913 30010 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:29.700191 30245 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:29.714648 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45698.1us
I20260709 12:17:29.706205 28366 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
W20260709 12:17:29.716286 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 45363.1us
I20260709 12:17:29.729100 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.731318 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:29.734282 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.735145 28358 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:29.709491 30060 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.736723 30087 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:29.737177 30087 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:29.737531 30087 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.740332 30259 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.740885 30259 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.741295 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.743177 30259 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:29.744498 30226 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:29.744740 30060 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:29.745486 30020 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:29.725961 30082 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:17:29.745334 30036 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:29.747468 30036 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:29.747905 30255 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:17:29.730937 30076 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:17:29.745333 30060 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.709828 30256 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.753582 30060 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:29.754194 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.719344 30248 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 12:17:29.714712 30228 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:17:29.756745 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.758975 28360 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:29.757243 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:29.711861 30010 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:29.760227 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.766002 28433 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:29.705705 28359 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:29.762468 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:29.759605 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 134300us
I20260709 12:17:29.764910 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:29.715619 30257 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:29.775651 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:29.779338 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.710223 29730 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.778532 28364 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:29.750460 30193 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:29.783406 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.757022 30036 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:29.768774 30076 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:29.695377 30192 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.794353 30106 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:29.795691 30256 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.789384 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.796482 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.796983 30076 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:29.745083 30226 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.798027 30076 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.798642 28367 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:29.798979 30257 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:29.800345 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:29.767266 30104 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.793454 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:29.807624 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.800520 30197 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.789947 28287 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:17:29.817337 28287 pending_rounds.cc:85] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 5
I20260709 12:17:29.817674 28287 pending_rounds.cc:107] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:17:29.818020 28287 raft_consensus.cc:2889] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:29.817538 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 161785us
I20260709 12:17:29.780891 30060 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:29.788422 30020 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.803211 28986 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:29.803488 29730 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.789368 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.783582 30111 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:17:29.801074 28359 raft_consensus.cc:606] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:29.884032 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:29.810322 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:29.814253 28365 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:29.763166 30119 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.829285 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:29.894083 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64920.3us
I20260709 12:17:29.775329 30205 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:17:29.837494 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:29.895289 28364 raft_consensus.cc:2368] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4.
I20260709 12:17:29.822052 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.820343 30060 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.841854 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.849812 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.906718 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.907390 28283 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:29.908233 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.852324 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.909535 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.854864 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:29.804044 30226 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.851253 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.912802 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.912973 28288 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:29.914136 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.804644 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.891642 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:29.891809 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:29.896826 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.840875 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.828687 30256 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:29.910682 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:29.916203 28439 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:29.916363 28288 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:29.838336 30036 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.908705 28283 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:29.921880 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.924016 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:29.917106 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.800042 30257 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:29.942896 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:29.830061 30198 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:29.873546 30193 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:29.942464 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:29.948377 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:29.949081 28439 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:29.949081 28438 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:29.951519 28433 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:29.889396 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.795853 30010 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:29.837335 30028 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
W20260709 12:17:29.965720 30253 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.450s	user 0.006s	sys 0.000s
I20260709 12:17:29.939855 29990 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:17:29.967948 30246 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.968822 30119 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:29.969527 29795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:59772 (request call id 71) took 3642 ms. Trace:
I20260709 12:17:29.903040 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.915956 30060 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:29.903312 29999 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.962692 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:29.976517 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:29.958779 28257 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:29.947031 28360 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:29.978399 28365 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:29.920037 30256 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:29.949918 30234 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.941391 30040 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.913004 30259 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 12:17:29.940857 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:29.949693 30260 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:29.876138 30020 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:29.959951 30193 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.968837 30246 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 12:17:29.954064 30106 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.050696 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:30.051626 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 227560us
I20260709 12:17:30.051651 28360 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:30.086370 28360 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:30.064172 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 105430us
I20260709 12:17:29.947721 30036 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.029402 29996 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.089493 29996 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.090247 30036 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:30.023002 30081 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:30.028487 30256 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:30.048915 30229 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:17:30.026553 28364 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:29.989333 30087 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.075479 28985 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:30.096625 30245 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.097254 30106 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.970016 29795 rpcz_store.cc:276] 0709 12:17:26.321530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.328037 (+1006507us) service_pool.cc:224] Handling call
0709 12:17:27.974607 (+646570us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b540035d020. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be804520 00007f85be88db42 00007f85be88c800 00007f85be8859fa 00007f85c384d4c9 00007f85bcdc9c54 00007f85bcdc9be2 00007f85bcdc9b11 00007f85bcdc9a8c 00007f85bcdc8d42
0709 12:17:27.978880 (+  4273us) log.cc:844] Serialized 494 byte log entry
0709 12:17:28.419046 (+440166us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b18002a4780. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:29.954645 (+1535599us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26629,"raft.queue_time_us":87574,"raft.run_cpu_time_us":2064,"raft.run_wall_time_us":106127,"spinlock_wait_cycles":39315200,"thread_start_us":392271,"threads_started":2,"wal-append.queue_time_us":293551,"wal-append.run_cpu_time_us":2219,"wal-append.run_wall_time_us":1013385}
I20260709 12:17:30.098450 30267 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:30.098997 30019 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.025655 29987 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:29.962666 30198 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.099756 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:30.110273 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 79820.1us
W20260709 12:17:30.124135 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.070655 30260 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:30.133991 30266 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:17:30.075632 30082 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.083773 30028 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.129899 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:30.091888 29996 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:30.140656 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:30.144616 28289 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:30.147117 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.122804 30192 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:30.106328 30122 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:17:30.091908 30081 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.157532 28284 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:30.162673 28288 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:30.170996 28288 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:30.172905 28288 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:30.133430 30267 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:30.126328 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.164748 28363 raft_consensus.cc:2368] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5.
I20260709 12:17:30.131728 30036 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:17:30.131999 29999 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:17:30.154505 30020 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:30.194563 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 59879.6us
I20260709 12:17:30.154074 30260 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.178315 28288 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.198468 28290 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:30.156803 28983 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:30.166232 30192 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:30.166692 30015 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:30.223021 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 91209.6us
I20260709 12:17:30.182147 30267 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:30.241681 28291 pending_rounds.cc:85] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 1
I20260709 12:17:30.246253 28291 pending_rounds.cc:107] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:30.246547 28291 raft_consensus.cc:2889] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:30.248873 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:30.255787 28367 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:30.167600 30082 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.263020 28367 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:30.209762 30283 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:30.224529 29795 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:30.236817 30245 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:30.167932 30028 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.236150 30015 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:30.274073 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 144323us
I20260709 12:17:30.274971 28330 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:30.276307 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:30.212314 30281 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:30.287572 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.270334 30265 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.137s	user 0.001s	sys 0.001s
W20260709 12:17:30.334692 30271 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:17:30.340821 30280 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.208s
I20260709 12:17:30.341359 30081 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:30.279536 29730 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.343387 30285 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:17:30.343705 30015 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.344732 30028 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:30.343245 30106 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:30.345577 30087 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:30.345810 29987 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:30.344883 30015 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:30.346683 30020 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:30.259196 28989 raft_consensus.cc:2955] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:30.274878 29795 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.289885 30281 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.372849 30281 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:30.342964 30276 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:17:30.343169 29730 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:30.335990 30119 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.347153 30020 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.351348 30193 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:30.411729 30226 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:30.419757 30275 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 12:17:30.420279 30270 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 12:17:30.421089 30287 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:30.421552 30274 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 12:17:30.423095 30279 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:17:30.275695 30245 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.360244 30192 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.405647 29730 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.273249 30283 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.413923 30020 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:30.436393 30010 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.424716 30019 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:30.421653 30168 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:30.424223 30226 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:30.442255 30162 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.144s	user 0.014s	sys 0.001s
I20260709 12:17:30.438179 30198 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:30.460983 30273 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 12:17:30.467338 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 379000us
I20260709 12:17:30.470197 30245 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:30.421679 30287 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.493281 30226 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.501157 30226 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:30.463846 30284 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 12:17:30.464568 29795 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:30.500484 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.506579 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e2ac8850682043e5bf8caf7fc64fae05 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:17:30.507586 30191 consensus_meta.cc:287] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.803s	user 0.008s	sys 0.006s
I20260709 12:17:30.519603 28287 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:30.521277 30283 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:30.523716 30287 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:30.503031 30245 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.530813 30245 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.533020 30245 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:30.534152 30282 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.275s	user 0.003s	sys 0.001s
I20260709 12:17:30.524925 30082 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:30.548336 30058 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.906s	user 0.011s	sys 0.000s
I20260709 12:17:30.550150 30256 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:30.552022 30235 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.848s	user 0.010s	sys 0.000s
I20260709 12:17:30.552872 30256 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:30.549028 30058 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.555305 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:30.533159 30260 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.567641 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:30.560148 30235 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.586632 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:30.587978 28432 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:30.574333 28437 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:30.583943 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1409) took 1168 ms. Trace:
I20260709 12:17:30.589365 28284 rpcz_store.cc:276] 0709 12:17:29.415400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.626125 (+210725us) service_pool.cc:224] Handling call
0709 12:17:29.641760 (+ 15635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.169479 (+527719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.583667 (+414188us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54000b18d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:30.583862 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185705728}
I20260709 12:17:30.586784 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:30.591713 28440 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:30.553695 30245 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:30.597648 28434 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 12:17:30.601125 28436 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:30.603498 28435 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:30.594780 30301 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:17:30.600301 30245 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.594012 30235 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.623783 30235 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:30.626348 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:30.621021 30303 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:17:30.690434 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 64153.4us
I20260709 12:17:30.689392 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:30.696269 29868 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:17:30.705413 28440 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
W20260709 12:17:30.707113 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 430871us
W20260709 12:17:30.708052 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.711652 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:30.713773 30245 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:30.715624 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:30.718132 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:30.718772 28286 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:30.721212 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:30.721832 28286 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:30.727104 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:30.689235 30287 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:30.732215 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:30.697018 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:30.698472 30286 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:30.775159 28289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:17:30.775488 30286 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:30.776757 28289 raft_consensus.cc:606] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:30.777140 28289 raft_consensus.cc:614] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:17:30.779735 28362 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.011s	sys 0.023s
I20260709 12:17:30.715054 30256 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.775918 30286 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:30.781121 28362 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.011s	sys 0.024s
I20260709 12:17:30.716331 28284 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:30.617010 30267 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.793952 28292 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:30.867764 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.862874 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.815280 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:30.781208 30060 consensus_meta.cc:287] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.755s	user 0.007s	sys 0.005s
I20260709 12:17:30.797782 30256 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.883019 30060 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.774426 30287 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:30.929844 30302 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:30.930469 30302 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:30.879156 28404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.015s	sys 0.022s
I20260709 12:17:30.863818 30295 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:30.855481 28365 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.018s	sys 0.035s
W20260709 12:17:30.879010 28255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.020s	sys 0.016s
I20260709 12:17:30.923338 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:30.921365 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 179) took 1087 ms. Trace:
I20260709 12:17:30.781522 30058 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:31.269516 28441 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.008s	sys 0.029s
W20260709 12:17:31.470664 28441 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.017s	sys 0.031s
W20260709 12:17:31.469379 28255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.209s	user 0.024s	sys 0.049s
I20260709 12:17:31.471381 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1494) took 1712 ms. Trace:
W20260709 12:17:31.472162 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 1.22333e+06us
I20260709 12:17:31.472044 28434 rpcz_store.cc:276] 0709 12:17:29.759008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.760073 (+  1065us) service_pool.cc:224] Handling call
0709 12:17:29.760409 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:30.598232 (+837823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.460389 (+862157us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b540026cad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:31.471285 (+ 10896us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1924756736}
W20260709 12:17:31.033022 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64507.7us
I20260709 12:17:30.931272 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:31.040465 28292 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:31.473815 30302 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:31.469686 28404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.510s	user 0.030s	sys 0.035s
W20260709 12:17:31.469281 28365 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.307s	user 0.039s	sys 0.054s
I20260709 12:17:31.039515 30235 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.305874 28285 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.006s	sys 0.022s
W20260709 12:17:31.482174 28285 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.006s	sys 0.028s
I20260709 12:17:31.084924 28987 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
W20260709 12:17:31.486914 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 1.54211e+06us
I20260709 12:17:31.488611 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:31.477788 28284 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:31.478427 28292 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:30.927810 30292 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.458801 30298 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.416s	user 0.000s	sys 0.001s
I20260709 12:17:30.925863 29987 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.748997 30245 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.100719 30309 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 12:17:31.603271 29987 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:31.086723 30020 consensus_meta.cc:287] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.000s
I20260709 12:17:31.604516 30020 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:31.606529 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 178) took 1773 ms. Trace:
I20260709 12:17:31.606564 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1422) took 1745 ms. Trace:
I20260709 12:17:31.607055 28290 rpcz_store.cc:276] 0709 12:17:29.860875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.215492 (+354617us) service_pool.cc:224] Handling call
0709 12:17:30.215921 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:30.235594 (+ 19673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.235628 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:30.236275 (+   647us) log.cc:844] Serialized 52 byte log entry
0709 12:17:30.707895 (+471620us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:30.707942 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.708009 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:31.606358 (+898349us) raft_consensus.cc:1685] finished
0709 12:17:31.606378 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:31.606503 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19171,"raft.queue_time_us":23291,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":17626,"thread_start_us":437264,"threads_started":1,"wal-append.queue_time_us":620807}
I20260709 12:17:31.054177 30257 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:31.263321 30106 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.043087 30303 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:17:30.922843 30082 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:31.454402 30305 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:31.654130 30106 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:31.440737 30299 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:17:31.657327 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1418) took 1797 ms. Trace:
W20260709 12:17:31.613309 30226 consensus_meta.cc:287] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.111s	user 0.012s	sys 0.000s
I20260709 12:17:31.287329 30256 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:31.600373 30245 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.969936 30260 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:31.574503 30286 consensus_meta.cc:287] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.003s
I20260709 12:17:31.259040 30198 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:31.309080 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:31.496423 30058 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.096921 30261 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:31.468690 28364 rpcz_store.cc:276] 0709 12:17:29.833585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.133915 (+300330us) service_pool.cc:224] Handling call
0709 12:17:30.150416 (+ 16501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.170994 (+ 20578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.171029 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:30.171508 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 12:17:30.375972 (+204464us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:30.724920 (+348948us) log.cc:844] Serialized 14 byte log entry
0709 12:17:30.790768 (+ 65848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.790862 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:30.790909 (+    47us) raft_consensus.cc:1685] finished
0709 12:17:30.790924 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.921264 (+130340us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80419,"prepare.queue_time_us":549635,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":133,"raft.queue_time_us":96403,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":47284,"thread_start_us":552709,"threads_started":2,"wal-append.queue_time_us":289166}
I20260709 12:17:31.479290 30235 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:30.825098 30281 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:31.204854 30293 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:31.606957 28360 rpcz_store.cc:276] 0709 12:17:29.833301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.087274 (+253973us) service_pool.cc:224] Handling call
0709 12:17:30.087638 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.124685 (+ 37047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.124714 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:30.132581 (+  7867us) log.cc:844] Serialized 26 byte log entry
0709 12:17:30.572037 (+439456us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:30.572083 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.572164 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:31.606299 (+1034135us) raft_consensus.cc:1685] finished
0709 12:17:31.606317 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:31.606465 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36530,"raft.queue_time_us":36774,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6064,"thread_start_us":409720,"threads_started":1,"wal-append.queue_time_us":559210}
I20260709 12:17:31.204550 30283 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.605670 29987 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:31.611636 29652 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:31.038678 30060 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:31.655282 30082 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:31.657805 28291 rpcz_store.cc:276] 0709 12:17:29.859873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.208087 (+348214us) service_pool.cc:224] Handling call
0709 12:17:30.241431 (+ 33344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:30.259931 (+ 18500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.259993 (+    62us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:30.261059 (+  1066us) log.cc:844] Serialized 52 byte log entry
0709 12:17:30.707806 (+446747us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:30.707858 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.707942 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:31.657035 (+949093us) raft_consensus.cc:1685] finished
0709 12:17:31.657062 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:31.657242 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12354,"raft.queue_time_us":23548,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":7030,"thread_start_us":429980,"threads_started":1,"wal-append.queue_time_us":785617}
I20260709 12:17:31.451079 30231 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:31.665270 28367 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.035s	sys 0.061s
I20260709 12:17:31.610014 30287 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:31.577275 30292 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:31.674670 30226 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:31.674881 30020 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:31.615320 30267 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:31.671648 28362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.001s	sys 0.031s
I20260709 12:17:31.475384 30295 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:31.654874 30305 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:31.680210 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:31.686311 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:31.683058 28359 raft_consensus.cc:606] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:31.689402 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:31.926863 28359 raft_consensus.cc:614] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:31.707234 28360 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:31.706403 28362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.002s	sys 0.031s
W20260709 12:17:31.926257 28332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.013s	sys 0.029s
W20260709 12:17:32.011601 28332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.013s	sys 0.030s
I20260709 12:17:32.013052 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1524) took 2046 ms. Trace:
W20260709 12:17:31.924690 28330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.013s	sys 0.028s
W20260709 12:17:32.014024 28330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.086s	user 0.013s	sys 0.028s
I20260709 12:17:32.014175 30287 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.015735 29987 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:32.015470 30287 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.017091 30323 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:31.707917 28358 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:17:31.695441 28367 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.035s	sys 0.061s
W20260709 12:17:31.940438 28406 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.000s	sys 0.004s
W20260709 12:17:32.044003 28441 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.013s
W20260709 12:17:32.044478 28406 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.001s	sys 0.005s
W20260709 12:17:32.044606 28441 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.000s	sys 0.014s
W20260709 12:17:32.045244 28285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.009s	sys 0.003s
I20260709 12:17:32.045395 30020 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:32.045886 28285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.009s	sys 0.003s
W20260709 12:17:31.725756 28437 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.022s	sys 0.023s
W20260709 12:17:32.046886 28437 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.023s	sys 0.023s
I20260709 12:17:32.046389 30020 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.047621 30331 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.703044 29652 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.048599 30331 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.051184 29652 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:30.932782 29730 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.051627 30331 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:31.709408 29814 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.017s	sys 0.027s
I20260709 12:17:31.683127 30058 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.053108 30020 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:32.054975 29814 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.347s	user 0.019s	sys 0.030s
W20260709 12:17:32.055306 28258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.004s	sys 0.004s
W20260709 12:17:32.055806 28258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.005s	sys 0.004s
I20260709 12:17:31.704200 30060 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.770921 30106 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:31.737548 30111 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.138s	user 0.038s	sys 0.047s
W20260709 12:17:32.061137 30111 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.466s	user 0.038s	sys 0.055s
W20260709 12:17:31.743268 30119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.025s	sys 0.022s
I20260709 12:17:31.670799 30192 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:32.078590 30119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.030s	sys 0.026s
I20260709 12:17:32.077137 30323 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.079159 30192 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.080111 30060 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:32.081378 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.707156 30226 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:31.703755 30231 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:31.782042 30235 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:32.094580 30226 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:32.095394 30235 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.096295 30235 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.097102 30332 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:31.750344 30245 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:31.676314 30256 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:32.098035 30332 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.098765 30256 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:31.614233 30257 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:31.676815 30260 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:32.103058 30260 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:32.103698 30260 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.105309 30257 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:31.775068 30281 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:31.695698 30283 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.081447 30323 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:31.673889 30286 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:31.783895 30292 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:31.694208 30293 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.144651 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1507) took 1584 ms. Trace:
I20260709 12:17:32.145213 28441 rpcz_store.cc:276] 0709 12:17:30.559766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.559929 (+   163us) service_pool.cc:224] Handling call
0709 12:17:30.560300 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.560560 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:31.471149 (+910589us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.119998 (+648849us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.132295 (+ 12297us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:32.135489 (+  3194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.140988 (+  5499us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.144466 (+  3478us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16450,"prepare.queue_time_us":1188109,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":38396,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":285,"thread_start_us":1481644,"threads_started":2,"wal-append.queue_time_us":589628}
I20260709 12:17:31.785580 30295 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.147205 30033 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:31.671895 30301 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.008s	sys 0.024s
W20260709 12:17:32.154162 30301 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.370s	user 0.009s	sys 0.024s
I20260709 12:17:32.153957 30033 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:31.789937 30305 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:31.682384 30306 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.829478 30310 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:17:31.836650 30311 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:17:31.835779 30317 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:17:32.179674 30283 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:31.842881 30320 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:17:32.181097 30344 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:31.845148 30322 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:17:31.841022 30330 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:17:32.181991 30344 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:31.831996 30285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.016s	sys 0.016s
W20260709 12:17:31.708545 30206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.010s	sys 0.024s
I20260709 12:17:32.184867 30334 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:31.790664 30205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.009s	sys 0.012s
W20260709 12:17:31.999377 28256 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.015s	sys 0.031s
I20260709 12:17:32.185583 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 184) took 1485 ms. Trace:
W20260709 12:17:31.923496 28286 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.014s	sys 0.027s
I20260709 12:17:32.013870 30324 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.015493 28330 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:32.027690 30082 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.030586 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:32.030941 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 168) took 1463 ms. Trace:
I20260709 12:17:32.059670 29730 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:32.060207 28358 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1513) took 2251 ms (client timeout 2121 ms). Trace:
W20260709 12:17:32.052081 28287 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1524) took 2193 ms (client timeout 1626 ms). Trace:
I20260709 12:17:31.677032 30198 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.082327 30192 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.083017 30060 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.084254 29652 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:32.090919 30058 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:32.091336 28987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 388) took 1364 ms. Trace:
W20260709 12:17:32.091610 28363 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1341) took 2393 ms (client timeout 2404 ms). Trace:
I20260709 12:17:31.675530 30261 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:32.077709 28365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.004s	sys 0.012s
I20260709 12:17:32.107331 30235 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:32.107924 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 174) took 1329 ms. Trace:
I20260709 12:17:32.054610 28364 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:32.111234 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 169) took 1535 ms. Trace:
I20260709 12:17:32.094601 30231 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:32.109676 30256 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:32.133572 28407 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.012s	sys 0.020s
I20260709 12:17:32.122946 30332 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:32.141762 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 526272us
I20260709 12:17:32.018639 28362 rpcz_store.cc:276] 0709 12:17:29.966155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.195922 (+229767us) service_pool.cc:224] Handling call
0709 12:17:30.196350 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.196602 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:30.791784 (+595182us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.011890 (+1220106us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.012028 (+   138us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:32.012209 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.012358 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.012762 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":509381,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":4709,"raft.queue_time_us":4865,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":315,"thread_start_us":1777298,"threads_started":2,"wal-append.queue_time_us":989120}
I20260709 12:17:32.144698 28360 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:32.147297 30293 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:32.167306 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.170077 30306 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.175705 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1501) took 2192 ms. Trace:
I20260709 12:17:32.178325 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 181) took 1562 ms. Trace:
I20260709 12:17:32.170972 30164 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:32.185139 30206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.403s	user 0.028s	sys 0.046s
W20260709 12:17:31.743534 30313 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:17:32.016882 29987 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:32.185966 28256 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.016s	sys 0.031s
W20260709 12:17:32.552071 28407 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.092s	user 0.023s	sys 0.057s
W20260709 12:17:32.471009 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 309887us
I20260709 12:17:32.549759 28285 rpcz_store.cc:276] 0709 12:17:30.575851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.576017 (+   166us) service_pool.cc:224] Handling call
0709 12:17:30.576396 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.576641 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:31.482515 (+905874us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.109846 (+627331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.110710 (+   864us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:32.110835 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.110977 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.111167 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1167686,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":397,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":196,"thread_start_us":1469077,"threads_started":2,"wal-append.queue_time_us":544152}
I20260709 12:17:32.494778 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1435) took 1762 ms. Trace:
W20260709 12:17:32.186287 30205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.010s	sys 0.013s
I20260709 12:17:32.162622 30226 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:32.548839 28363 rpcz_store.cc:269] 0709 12:17:29.698237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.698395 (+   158us) service_pool.cc:224] Handling call
0709 12:17:32.066072 (+2367677us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400103454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:32.091411 (+ 25339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041648000}
W20260709 12:17:32.546239 28358 rpcz_store.cc:269] 0709 12:17:29.780441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.780635 (+   194us) service_pool.cc:224] Handling call
0709 12:17:31.680029 (+1899394us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b5800113798. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:32.032035 (+352006us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5386624}
W20260709 12:17:32.562706 28365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.009s	sys 0.017s
I20260709 12:17:32.570585 30197 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.571698 30197 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.574458 30245 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:32.574743 30324 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.472895 30295 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.575618 30219 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:32.575487 30245 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.576363 30295 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.576514 30219 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.495887 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 226) took 2403 ms. Trace:
I20260709 12:17:32.577891 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 171) took 1779 ms. Trace:
I20260709 12:17:32.578855 30295 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:32.579129 28257 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.3:33721 timed out after 1.690s (SENT) [suppressed 3 similar messages]
I20260709 12:17:32.579605 30295 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.580399 30295 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.547186 28291 rpcz_store.cc:276] 0709 12:17:30.778278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.691574 (+913296us) service_pool.cc:224] Handling call
0709 12:17:31.928967 (+237393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:31.930505 (+  1538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:31.930543 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:31.931670 (+  1127us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.005228 (+ 73558us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:32.005273 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.007716 (+  2443us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:32.107483 (+ 99767us) raft_consensus.cc:1685] finished
0709 12:17:32.107509 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.107679 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80389,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":492,"thread_start_us":69963,"threads_started":1,"wal-append.queue_time_us":145548}
I20260709 12:17:32.582809 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
W20260709 12:17:32.546532 28287 rpcz_store.cc:269] 0709 12:17:29.858844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.876269 (+ 17425us) service_pool.cc:224] Handling call
0709 12:17:32.051473 (+2175204us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b540035fcd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:32.051726 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393881856}
W20260709 12:17:32.586012 28286 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.619s	user 0.019s	sys 0.051s
I20260709 12:17:32.546459 28987 rpcz_store.cc:276] 0709 12:17:30.726673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.726855 (+   182us) service_pool.cc:224] Handling call
0709 12:17:32.090785 (+1363930us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400476b54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:32.091033 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581484800}
I20260709 12:17:32.548148 28283 rpcz_store.cc:276] 0709 12:17:30.567226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.568743 (+  1517us) service_pool.cc:224] Handling call
0709 12:17:30.569097 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.621729 (+ 52632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.621784 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:30.621837 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:30.621874 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.030120 (+1408246us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b54003fa054. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:32.030221 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.030612 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120537984}
W20260709 12:17:32.184907 30285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.020s	sys 0.029s
I20260709 12:17:32.581105 30226 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.186096 30333 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:32.494778 30010 consensus_meta.cc:287] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 2.039s	user 0.005s	sys 0.020s
I20260709 12:17:32.557901 28289 rpcz_store.cc:276] 0709 12:17:30.731711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.498649 (+766938us) service_pool.cc:224] Handling call
0709 12:17:31.578354 (+ 79705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.927260 (+348906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:31.927387 (+   127us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:31.927496 (+   109us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:31.927564 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.494188 (+566624us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54000b0254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:32.494251 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.494517 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800675200}
I20260709 12:17:32.546487 28438 rpcz_store.cc:276] 0709 12:17:29.983302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.983454 (+   152us) service_pool.cc:224] Handling call
0709 12:17:29.983838 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:29.984225 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:29.984251 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:31.665296 (+1681045us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b2c000ef378. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:17:31.665828 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.044989 (+379161us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:32.045097 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.045247 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:32.175081 (+129834us) raft_consensus.cc:1685] finished
0709 12:17:32.175108 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.175485 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43901,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"spinlock_wait_cycles":89161600,"thread_start_us":333175,"threads_started":1,"wal-append.queue_time_us":503832}
I20260709 12:17:32.203948 30033 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:32.585532 30106 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.580495 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:32.473558 30257 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:32.590657 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1516) took 1897 ms. Trace:
I20260709 12:17:32.602875 30330 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:17:32.603371 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:32.205631 30339 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:32.186139 28367 rpcz_store.cc:276] 0709 12:17:30.699987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.700205 (+   218us) service_pool.cc:224] Handling call
0709 12:17:30.700532 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.700900 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.700925 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:30.701388 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.044785 (+1343397us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:32.044921 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.045082 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:32.181640 (+136558us) raft_consensus.cc:1685] finished
0709 12:17:32.181664 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.185389 (+  3725us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49302,"raft.queue_time_us":80,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5392,"thread_start_us":1342725,"threads_started":1,"wal-append.queue_time_us":1061641}
I20260709 12:17:32.397830 30323 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:32.596834 28359 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.578332 28292 rpcz_store.cc:276] 0709 12:17:30.772055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.508246 (+736191us) service_pool.cc:224] Handling call
0709 12:17:31.508602 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.508833 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:32.076663 (+567830us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.471489 (+394826us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.488735 (+ 17246us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:32.528068 (+ 39333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.528164 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.551473 (+ 23309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":383044,"prepare.queue_time_us":262975,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":451,"raft.queue_time_us":331501,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":206,"thread_start_us":365370,"threads_started":2,"wal-append.queue_time_us":435825}
I20260709 12:17:32.580528 30119 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:32.185940 30334 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.627897 30164 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.630995 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:32.632130 30198 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.608703 30330 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:32.570602 30345 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:32.637856 30324 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.639643 30295 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.668182 30281 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:32.668251 30345 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.669775 30292 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.670508 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:32.670686 30267 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.671351 28358 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:32.671742 30345 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.640782 30197 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.672003 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1348) took 1198 ms. Trace:
I20260709 12:17:32.578052 28989 rpcz_store.cc:276] 0709 12:17:30.092620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.095164 (+  2544us) service_pool.cc:224] Handling call
0709 12:17:30.095601 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.239933 (+144332us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:30.529896 (+289963us) log.cc:844] Serialized 14 byte log entry
0709 12:17:31.063526 (+533630us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:31.082250 (+ 18724us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:31.091967 (+  9717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.495286 (+1403319us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400762d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:32.495359 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.495632 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6726,"raft.queue_time_us":22399,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"spinlock_wait_cycles":297551872,"thread_start_us":494902,"threads_started":1,"wal-append.queue_time_us":1202130}
I20260709 12:17:32.610419 30235 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:17:32.674378 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:32.675233 30235 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:32.639909 30312 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.880s	user 0.002s	sys 0.000s
I20260709 12:17:32.646704 30257 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.617961 30332 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:32.648524 30323 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:32.649153 30260 consensus_meta.cc:287] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.545s	user 0.006s	sys 0.009s
I20260709 12:17:32.675983 30235 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.650710 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 178) took 1868 ms. Trace:
I20260709 12:17:32.651381 30219 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.548314 28361 rpcz_store.cc:276] 0709 12:17:30.615429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.616877 (+  1448us) service_pool.cc:224] Handling call
0709 12:17:30.622692 (+  5815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.697368 (+ 74676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.699133 (+  1765us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:32.055629 (+1356496us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b2c0000e678. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:17:32.056363 (+   734us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.130721 (+ 74358us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:32.133818 (+  3097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.136754 (+  2936us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:32.156075 (+ 19321us) raft_consensus.cc:1685] finished
0709 12:17:32.161188 (+  5113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.178079 (+ 16891us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4359,"raft.queue_time_us":67278,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5907,"spinlock_wait_cycles":19096320,"thread_start_us":55981,"threads_started":1,"wal-append.queue_time_us":92087}
I20260709 12:17:32.679782 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:32.601608 28437 rpcz_store.cc:276] 0709 12:17:30.693647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.693808 (+   161us) service_pool.cc:224] Handling call
0709 12:17:30.694130 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.694375 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:32.047570 (+1353195us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.164343 (+116773us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.170466 (+  6123us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:32.545623 (+375157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.553709 (+  8086us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.590575 (+ 36866us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1542,"prepare.queue_time_us":1055782,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":18901,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":133,"thread_start_us":1425894,"threads_started":3,"wal-append.queue_time_us":500133}
I20260709 12:17:32.661439 28367 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.599174 30058 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:32.611033 30344 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.662295 30330 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.669061 30281 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.675997 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:32.646957 30286 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:32.672472 28358 rpcz_store.cc:276] 0709 12:17:31.473617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.670031 (+1196414us) service_pool.cc:224] Handling call
0709 12:17:32.671915 (+  1884us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:32.624119 30245 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:32.616889 30283 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:32.650583 28991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 321) took 2776 ms. Trace:
I20260709 12:17:32.657796 29652 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.652978 30106 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.670460 30292 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.926728 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:32.736646 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 209336us
W20260709 12:17:32.711460 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 57384.5us
I20260709 12:17:32.690465 28289 rpcz_store.cc:276] 0709 12:17:30.782534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.601239 (+1818705us) service_pool.cc:224] Handling call
0709 12:17:32.649888 (+ 48649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.650239 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.650264 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.650324 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:32.650364 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.650445 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.650594 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:32.795740 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 170) took 2032 ms. Trace:
W20260709 12:17:32.874073 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 478309us
I20260709 12:17:32.924733 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1522) took 3050 ms. Trace:
I20260709 12:17:32.842392 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:32.790951 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:32.671311 30267 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:32.967839 28991 rpcz_store.cc:276] 0709 12:17:29.873770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.874812 (+  1042us) service_pool.cc:224] Handling call
0709 12:17:31.459760 (+1584948us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:32.095358 (+635598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.095833 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.095859 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.095920 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:32.095984 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.096052 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.650157 (+554105us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400476b50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:32.650328 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1466349568}
I20260709 12:17:32.844448 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:32.924034 30352 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 12:17:32.790141 30280 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.342s	user 0.019s	sys 0.020s
I20260709 12:17:32.688700 30111 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:33.082540 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 502216us
I20260709 12:17:32.790772 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:32.792637 30164 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:33.086504 28362 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:32.921720 29730 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.790481 30205 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:32.690582 29814 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:33.094481 28363 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:32.792516 30257 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:32.688920 30306 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.099072 28364 raft_consensus.cc:2410] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:32.922127 30293 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:33.087121 30280 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.819s	user 0.029s	sys 0.032s
I20260709 12:17:33.102561 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:33.104753 28441 raft_consensus.cc:2368] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:32.923195 30197 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:33.108690 30197 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.110324 30287 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.112098 30197 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.112738 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.113626 28436 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:33.114219 30261 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:33.012616 28284 rpcz_store.cc:276] 0709 12:17:30.763790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.499221 (+735431us) service_pool.cc:224] Handling call
0709 12:17:31.499529 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.499739 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:32.078221 (+578482us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.795133 (+716912us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.795218 (+    85us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:32.795334 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.795402 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.795576 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":591930,"prepare.queue_time_us":262476,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":1095,"raft.queue_time_us":592904,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"spinlock_wait_cycles":2351872,"thread_start_us":432353,"threads_started":2,"wal-append.queue_time_us":424460}
I20260709 12:17:32.686715 30339 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.100616 28439 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
W20260709 12:17:33.103729 28363 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1355) took 1491 ms (client timeout 1876 ms). Trace:
I20260709 12:17:33.114866 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:33.119879 28363 rpcz_store.cc:269] 0709 12:17:31.603269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.616013 (+1012744us) service_pool.cc:224] Handling call
0709 12:17:33.095044 (+479031us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:32.791208 29734 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.107280 28435 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:32.770690 30332 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.121860 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:33.091809 30323 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:33.097837 30331 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.088209 30111 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:33.131825 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:33.101559 30245 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.843837 30119 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:32.690347 30260 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.132534 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:33.130417 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 1.11525e+06us
I20260709 12:17:33.123216 30261 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.140988 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:32.690160 30333 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:33.145394 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.136229 29730 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.149528 29734 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.151813 29730 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.152944 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:33.135145 30111 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.152258 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1350) took 1677 ms. Trace:
I20260709 12:17:33.157837 28358 rpcz_store.cc:276] 0709 12:17:31.474960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.089570 (+1614610us) service_pool.cc:224] Handling call
0709 12:17:33.152149 (+ 62579us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.131870 30205 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:33.158938 28436 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:33.118480 30058 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.163728 28291 raft_consensus.cc:606] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:33.167842 28291 raft_consensus.cc:614] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:33.168015 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:33.014802 28365 rpcz_store.cc:276] 0709 12:17:29.874380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.161144 (+286764us) service_pool.cc:224] Handling call
0709 12:17:30.161504 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.161747 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:31.479016 (+1317269us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.619062 (+1140046us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.660475 (+ 41413us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:32.681455 (+ 20980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.690382 (+  8927us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:32.924624 (+234242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10085,"prepare.queue_time_us":449484,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":2485,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":20530,"thread_start_us":2400731,"threads_started":2,"wal-append.queue_time_us":663087}
I20260709 12:17:33.122452 30283 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.171036 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1424) took 3231 ms. Trace:
I20260709 12:17:33.170827 28439 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:33.171029 28287 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:33.177255 28441 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:33.180768 30354 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.521s	user 0.001s	sys 0.001s
I20260709 12:17:33.171619 28288 rpcz_store.cc:276] 0709 12:17:29.861343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.218563 (+357220us) service_pool.cc:224] Handling call
0709 12:17:30.218880 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.570366 (+351486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.570399 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:30.571021 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 12:17:30.830136 (+259115us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:30.830188 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.830359 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:32.925438 (+2095079us) raft_consensus.cc:1685] finished
0709 12:17:33.012089 (+ 86651us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.092602 (+ 80513us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18062,"raft.queue_time_us":36115,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":3909,"thread_start_us":591505,"threads_started":2,"wal-append.queue_time_us":878553}
I20260709 12:17:33.186560 28363 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.189363 28360 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:17:33.190302 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.191144 28255 leader_election.cc:400] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5.
I20260709 12:17:33.190289 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 186) took 2479 ms. Trace:
I20260709 12:17:33.191594 28255 leader_election.cc:403] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:17:33.191804 28360 rpcz_store.cc:276] 0709 12:17:30.710532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.188192 (+2477660us) service_pool.cc:224] Handling call
0709 12:17:33.188662 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.189934 (+  1272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.190197 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.192601 29734 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:33.193362 29734 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.194314 28435 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:33.195005 30245 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:33.195458 30353 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.507s	user 0.001s	sys 0.000s
I20260709 12:17:33.196040 30334 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:33.197150 30205 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.198071 30330 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:33.198657 30292 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.216640 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:33.191250 30283 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.218322 30323 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:33.200330 30339 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.218891 30323 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:33.219396 30323 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.219556 29734 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:33.191177 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1351) took 1636 ms. Trace:
I20260709 12:17:33.221837 30281 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:33.193374 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:33.223332 30332 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:33.223590 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:33.201534 28440 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.214731 30246 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.171752 30226 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:33.220047 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 63214.5us
W20260709 12:17:33.209759 28362 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1352) took 1607 ms (client timeout 1825 ms). Trace:
I20260709 12:17:33.215328 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:33.232900 28362 rpcz_store.cc:269] 0709 12:17:31.602210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.602863 (+1000653us) service_pool.cc:224] Handling call
0709 12:17:33.209503 (+606640us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b5400054954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:33.209651 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108995840}
I20260709 12:17:33.213244 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:33.221907 28364 rpcz_store.cc:276] 0709 12:17:31.555193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.690829 (+1135636us) service_pool.cc:224] Handling call
0709 12:17:33.190915 (+500086us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b540007ab54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:33.191103 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36179712}
I20260709 12:17:33.165495 30331 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:33.217525 28365 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:33.236780 28367 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
W20260709 12:17:33.241122 28367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1358) took 1637 ms (client timeout 1968 ms). Trace:
W20260709 12:17:33.241658 28367 rpcz_store.cc:269] 0709 12:17:31.604087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.623417 (+1019330us) service_pool.cc:224] Handling call
0709 12:17:33.241031 (+617614us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.158360 30164 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 12:17:33.254092 28358 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:33.134291 30293 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.236941 30339 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:33.262957 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:33.232177 30333 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:33.260849 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1360) took 1656 ms. Trace:
I20260709 12:17:33.271715 28358 rpcz_store.cc:276] 0709 12:17:31.604648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.165773 (+1561125us) service_pool.cc:224] Handling call
0709 12:17:33.260752 (+ 94979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.235854 29730 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5.
I20260709 12:17:33.243052 30324 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.304499 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:33.311605 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:33.268456 30334 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:33.319089 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46339.5us
I20260709 12:17:33.170024 29814 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.265295 30339 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:33.350229 30339 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.311877 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:33.359771 28436 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.362517 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.363632 28441 raft_consensus.cc:2410] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:33.364730 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.365437 28441 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:33.276829 30344 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:33.329046 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:33.320765 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.332818 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1354) took 1729 ms. Trace:
I20260709 12:17:33.334049 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 175) took 2555 ms. Trace:
I20260709 12:17:33.368212 28435 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:33.338639 30366 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:17:33.367789 28367 rpcz_store.cc:276] 0709 12:17:31.602929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.251290 (+1648361us) service_pool.cc:224] Handling call
0709 12:17:33.260443 (+  9153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.332311 (+ 71868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:33.332351 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:33.332458 (+   107us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:33.332503 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.332572 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.332718 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.343685 28439 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:33.315110 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 57101.1us
I20260709 12:17:33.342792 30260 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:33.244673 30246 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.380045 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:33.388113 30267 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:33.397822 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 182684us
I20260709 12:17:33.389900 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1430) took 2659 ms. Trace:
I20260709 12:17:33.342780 30119 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.402571 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:33.308233 30324 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.367918 28283 rpcz_store.cc:276] 0709 12:17:30.778564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.598770 (+1820206us) service_pool.cc:224] Handling call
0709 12:17:32.635592 (+ 36822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.921389 (+285797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.921431 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:33.086065 (+164634us) log.cc:844] Serialized 26 byte log entry
0709 12:17:33.182834 (+ 96769us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:33.182879 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.182963 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.333792 (+150829us) raft_consensus.cc:1685] finished
0709 12:17:33.333812 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.333974 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":285116,"raft.queue_time_us":286315,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":169,"thread_start_us":48621,"threads_started":1,"wal-append.queue_time_us":207933}
I20260709 12:17:33.406566 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:33.395081 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 176) took 2616 ms. Trace:
I20260709 12:17:33.375566 30334 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:33.379763 30106 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:33.500690 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 53874.4us
W20260709 12:17:33.477285 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 92380.2us
I20260709 12:17:33.582871 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1357) took 1934 ms. Trace:
W20260709 12:17:33.454711 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45388.4us
I20260709 12:17:33.523845 28367 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:33.583458 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:33.629117 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 319920us
W20260709 12:17:33.477236 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 64908.8us
I20260709 12:17:33.632516 28358 rpcz_store.cc:276] 0709 12:17:31.603809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.314565 (+1710756us) service_pool.cc:224] Handling call
0709 12:17:33.382185 (+ 67620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.515078 (+132893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:33.515126 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:33.537766 (+ 22640us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:33.537845 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.537929 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.538120 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18073,"raft.queue_time_us":67531,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":97}
I20260709 12:17:33.384014 29730 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:33.419539 28434 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:33.396752 30192 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:33.510875 30374 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:17:33.638887 28285 rpcz_store.cc:276] 0709 12:17:30.778824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.600175 (+1821351us) service_pool.cc:224] Handling call
0709 12:17:32.646767 (+ 46592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.792940 (+146173us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:33.108021 (+315081us) log.cc:844] Serialized 14 byte log entry
0709 12:17:33.319454 (+211433us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:33.319565 (+   111us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:33.319643 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.394773 (+ 75130us) spinlock_profiling.cc:243] Waited 89.9 ms on lock 0x7b54000affd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:33.394828 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.395000 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29483,"prepare.queue_time_us":366172,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":3211,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":6529,"spinlock_wait_cycles":206771584,"thread_start_us":477019,"threads_started":2,"wal-append.queue_time_us":300841}
W20260709 12:17:33.508502 30357 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.952s	user 0.000s	sys 0.004s
I20260709 12:17:33.644372 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1359) took 2029 ms. Trace:
I20260709 12:17:33.648284 28440 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:33.672966 28436 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:33.480538 30267 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:33.515625 30293 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.522845 30331 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:33.408458 30292 raft_consensus.cc:2764] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:33.675078 28367 rpcz_store.cc:276] 0709 12:17:31.604371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.383514 (+1779143us) service_pool.cc:224] Handling call
0709 12:17:33.417406 (+ 33892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.633448 (+216042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.633680 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.506963 30246 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:33.481698 29652 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:33.680022 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.411660 30058 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:33.481980 30333 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:33.683828 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.479065 30359 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:33.381604 30226 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.295s
I20260709 12:17:33.671340 29730 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:33.690692 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.701892 28438 raft_consensus.cc:2410] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:33.691620 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:33.678241 30267 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.692979 28441 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:33.479121 29734 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:33.417825 30281 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.272s
W20260709 12:17:33.694654 28363 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.002s
I20260709 12:17:33.410409 28286 rpcz_store.cc:276] 0709 12:17:30.730387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.732697 (+  2310us) service_pool.cc:224] Handling call
0709 12:17:30.733031 (+   334us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:30.878259 (+145228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.878300 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:30.878930 (+   630us) log.cc:844] Serialized 52 byte log entry
0709 12:17:32.679455 (+1800525us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:33.096135 (+416680us) log.cc:844] Serialized 14 byte log entry
0709 12:17:33.377829 (+281694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.389254 (+ 11425us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.389345 (+    91us) raft_consensus.cc:1685] finished
0709 12:17:33.389359 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.389676 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63005,"prepare.queue_time_us":327676,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":64,"raft.queue_time_us":63651,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":38037,"thread_start_us":1983822,"threads_started":3,"wal-append.queue_time_us":1441502,"wal-append.run_cpu_time_us":2114,"wal-append.run_wall_time_us":1012196}
I20260709 12:17:33.696080 30333 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.697144 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.709508 28363 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:33.710523 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1361) took 2105 ms. Trace:
I20260709 12:17:33.711544 28363 rpcz_store.cc:276] 0709 12:17:31.604903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.129279 (+1524376us) service_pool.cc:224] Handling call
0709 12:17:33.710448 (+581169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.636729 30111 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:33.687078 28359 consensus_meta.cc:287] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.031s	user 0.014s	sys 0.000s
I20260709 12:17:33.705199 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:33.728535 28436 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:33.729375 28363 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:33.703830 29730 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.731628 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.701538 30106 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:17:33.704450 30267 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.737802 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.741573 28332 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:33.743384 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.747967 28439 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:33.747946 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.749197 28358 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:33.699283 30192 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.751704 28365 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
W20260709 12:17:33.750962 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47954.8us
I20260709 12:17:33.752918 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.730350 28359 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:33.753650 28439 raft_consensus.cc:2410] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:33.756697 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.760273 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.703577 30359 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:33.767645 30372 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:17:33.769243 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.769330 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.769575 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.771035 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.770130 28436 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:33.772758 28441 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:33.758494 30293 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:33.777070 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 180) took 1203 ms. Trace:
I20260709 12:17:33.777536 28292 rpcz_store.cc:276] 0709 12:17:32.573297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.688987 (+115690us) service_pool.cc:224] Handling call
0709 12:17:33.015500 (+326513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.084966 (+ 69466us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:33.260453 (+175487us) log.cc:844] Serialized 14 byte log entry
0709 12:17:33.776643 (+516190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:33.776710 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:33.776770 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.776853 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.776989 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":156856,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":5570,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":5572,"thread_start_us":547038,"threads_started":3,"wal-append.queue_time_us":429856}
I20260709 12:17:33.782820 30305 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.783489 30305 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.785000 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.768625 30292 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:33.789016 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:33.790728 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.794581 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 187) took 2882 ms. Trace:
I20260709 12:17:33.795115 28360 rpcz_store.cc:276] 0709 12:17:30.911906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.192894 (+2280988us) service_pool.cc:224] Handling call
0709 12:17:33.193393 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:33.212533 (+ 19140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:33.212570 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:33.281724 (+ 69154us) log.cc:844] Serialized 26 byte log entry
0709 12:17:33.697857 (+416133us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:33.697901 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.697953 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.794295 (+ 96342us) raft_consensus.cc:1685] finished
0709 12:17:33.794317 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.794491 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38889,"raft.queue_time_us":19124,"raft.run_cpu_time_us":1361,"raft.run_wall_time_us":110198,"thread_start_us":324683,"threads_started":1,"wal-append.queue_time_us":426527}
I20260709 12:17:33.780535 30293 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:33.797904 28437 raft_consensus.cc:2410] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:33.798467 28436 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:17:33.788908 28404 connection.cc:256] RPC call timeout handler was delayed by -1.9191798699998799s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 12:17:33.807601 28435 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
W20260709 12:17:33.810129 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 52739.4us
I20260709 12:17:33.811653 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:33.813177 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:33.814121 30226 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:33.816058 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:33.821339 28404 connection.cc:256] RPC call timeout handler was delayed by -1.9191798699998799s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260709 12:17:33.821903 28404 connection.cc:256] RPC call timeout handler was delayed by -1.6833798699999534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
W20260709 12:17:33.822266 28404 connection.cc:256] RPC call timeout handler was delayed by -1.6662798699999257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
W20260709 12:17:33.822572 28404 connection.cc:256] RPC call timeout handler was delayed by -1.6662798699999257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
W20260709 12:17:33.822857 28404 connection.cc:256] RPC call timeout handler was delayed by -1.6122798700000658s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
I20260709 12:17:33.788228 30292 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:33.785722 30305 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:33.823150 28404 connection.cc:256] RPC call timeout handler was delayed by -1.5726798700000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
W20260709 12:17:33.808036 30353 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.593s	user 0.001s	sys 0.003s
I20260709 12:17:33.828627 30226 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:33.829475 30388 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:33.847736 30388 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.833997 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:33.835325 28438 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:17:33.851410 28438 pending_rounds.cc:85] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 4
I20260709 12:17:33.769028 30334 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.830819 30292 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:33.830605 28404 connection.cc:256] RPC call timeout handler was delayed by -1.537579869999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
W20260709 12:17:33.854362 28404 connection.cc:256] RPC call timeout handler was delayed by -1.4736798700000691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
W20260709 12:17:33.854732 28404 connection.cc:256] RPC call timeout handler was delayed by -1.4349798700000065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
I20260709 12:17:33.854631 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:33.855098 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1838798700000552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24430000000000002s for a response
W20260709 12:17:33.855435 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1838798700000552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24430000000000002s for a response
I20260709 12:17:33.849356 28434 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:33.829546 30205 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:33.819190 30293 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:33.852000 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 96799.4us
W20260709 12:17:33.856781 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 149763us
I20260709 12:17:33.851718 28438 pending_rounds.cc:107] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:33.860038 28438 raft_consensus.cc:2889] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:33.861608 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.862708 28286 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:33.852978 30388 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.866397 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:33.858575 30205 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:33.878831 28441 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:33.870836 28437 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:33.874909 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.875298 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:33.866293 30226 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:33.897570 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:33.901626 30323 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.004s
I20260709 12:17:33.865556 30111 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:33.902163 28284 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:33.896618 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:33.874362 30359 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:33.904966 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:33.906023 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:33.906229 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:33.909543 28358 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:33.907711 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:33.872720 30305 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.914242 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.872653 30381 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:33.880515 30333 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:33.928704 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:33.935679 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.939895 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.940642 28286 raft_consensus.cc:2410] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:33.936721 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:33.942596 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.943615 28441 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:33.941643 28289 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:33.949990 28434 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:33.913682 30226 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:33.936771 30384 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:17:33.959829 28435 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:17:33.961926 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 79638.8us
W20260709 12:17:33.907246 28359 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1349) took 2432 ms (client timeout 1852 ms). Trace:
I20260709 12:17:33.965050 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.965634 28434 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
W20260709 12:17:33.966897 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 104415us
I20260709 12:17:33.963331 28332 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:33.967659 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.934952 30164 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:33.970109 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.971397 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:33.974610 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 158385us
I20260709 12:17:33.970873 28438 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:33.976711 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:33.977211 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:33.977311 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.957882 29734 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.978000 28437 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:33.978034 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.976826 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.981374 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:33.960809 30381 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:33.949767 30331 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:33.964581 30226 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:33.918447 30205 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:33.920286 30323 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:33.981546 28438 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:33.966357 30286 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:33.994267 30286 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:33.982854 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:33.984078 28436 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:33.981947 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:33.988710 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 183) took 3237 ms. Trace:
I20260709 12:17:33.988932 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:34.018110 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:34.003786 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:34.010834 30205 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.011205 30119 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:34.011720 30383 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:34.012272 30235 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:34.012843 30323 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:34.013528 28362 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:34.014420 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:34.014879 30331 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.015631 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:34.014369 30286 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:34.016198 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:34.018186 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:34.019306 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:33.964761 28359 rpcz_store.cc:269] 0709 12:17:31.474337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.022360 (+548023us) service_pool.cc:224] Handling call
0709 12:17:33.906980 (+1884620us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b54003bb0d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:33.907119 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103473408}
I20260709 12:17:34.028030 29730 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.060364 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:34.061980 30235 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.062397 30106 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.062898 30106 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.029083 30192 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.063268 30106 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.063589 30058 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.029331 30164 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.982455 30283 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:34.064646 30283 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.065080 30283 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:34.065433 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 85457.8us
I20260709 12:17:33.988794 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 172) took 3136 ms. Trace:
I20260709 12:17:34.062521 30235 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.017426 30381 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.064679 30164 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.072682 30164 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:33.944876 30305 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:34.043931 28435 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:17:34.041317 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 105558us
I20260709 12:17:34.054863 30245 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.046324 28283 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:34.037400 30119 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:34.030593 30111 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.037585 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1379) took 1357 ms. Trace:
I20260709 12:17:34.069932 28290 rpcz_store.cc:276] 0709 12:17:30.775917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.609025 (+833108us) service_pool.cc:224] Handling call
0709 12:17:31.609346 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.106337 (+496991us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:32.106866 (+   529us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.171708 (+ 64842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.171766 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:32.171823 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.794737 (+622914us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b1800010900. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85c275b58f 00007f85bb79a571 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd
0709 12:17:33.905560 (+1110823us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54003322d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:33.905622 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.912655 (+  7033us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":147790,"prepare.queue_time_us":1932,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":39279,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":65935,"spinlock_wait_cycles":1146272768,"thread_start_us":21977,"threads_started":1,"wal-append.queue_time_us":448265}
I20260709 12:17:34.055094 28286 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:34.061928 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:34.070138 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:34.032204 28366 rpcz_store.cc:276] 0709 12:17:30.697026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.697236 (+   210us) service_pool.cc:224] Handling call
0709 12:17:30.709064 (+ 11828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.709443 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.709470 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:30.719426 (+  9956us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.061207 (+1341781us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b18003fece0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:17:32.173327 (+112120us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:32.174382 (+  1055us) log.cc:844] Serialized 14 byte log entry
0709 12:17:32.176150 (+  1768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.176261 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:33.807661 (+1631400us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b24003d6e68. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6209c4 00007f85bb61eacb 00007f85bb7d9dc9 00007f85bb7d9a55 00007f85bb7d909d 00007f85be40b10e 00007f85be42288c 00007f85be421bb6 00007f85be4217d8 00007f85be42b221
0709 12:17:33.913121 (+105460us) raft_consensus.cc:1685] finished
0709 12:17:33.913145 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.934623 (+ 21478us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18356,"prepare.queue_time_us":295,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":9908,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":32176,"spinlock_wait_cycles":92967424,"thread_start_us":81127,"threads_started":1,"wal-append.queue_time_us":1869862}
I20260709 12:17:34.082257 28440 rpcz_store.cc:276] 0709 12:17:32.673769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.736285 (+ 62516us) service_pool.cc:224] Handling call
0709 12:17:34.003921 (+1267636us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54002e45d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:34.031392 (+ 27471us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259527424}
W20260709 12:17:34.056439 30287 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.007s	sys 0.004s
I20260709 12:17:34.083736 28285 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:34.086390 28288 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:34.084429 28289 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:34.088783 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:34.092096 30192 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:34.093266 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:34.037317 29734 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.093967 28290 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:34.094503 30389 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.094252 30119 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.092489 28432 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:17:34.095620 28432 pending_rounds.cc:85] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 3
I20260709 12:17:34.095923 28432 pending_rounds.cc:107] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:17:34.096163 28432 raft_consensus.cc:2889] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:34.079380 30235 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:34.087237 30287 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.007s	sys 0.004s
I20260709 12:17:34.085222 30111 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.093848 30305 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:34.137641 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:34.138880 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:34.087751 30286 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.094967 30389 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.143743 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:34.145673 28441 raft_consensus.cc:2410] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:34.123464 30164 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:34.177949 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 116101us
I20260709 12:17:34.080295 30267 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.156221 28285 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:34.135352 29730 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.086647 30323 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:17:34.162849 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:34.171361 29734 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:34.172016 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:34.174278 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:34.176213 30339 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.825s	user 0.009s	sys 0.004s
I20260709 12:17:34.094888 30292 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:34.174167 30346 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:17:34.141757 30111 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:34.202648 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 61478.7us
I20260709 12:17:34.150367 30119 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:34.199149 28286 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:34.150826 30389 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.210310 28292 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:34.212160 28285 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:17:34.229957 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 136343us
I20260709 12:17:34.268334 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:34.268959 28286 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:34.237130 29730 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:34.270186 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 181431us
I20260709 12:17:34.266242 28404 leader_election.cc:400] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4.
I20260709 12:17:34.271339 28404 leader_election.cc:403] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:17:34.272115 28290 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
W20260709 12:17:34.274063 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 71415.5us
I20260709 12:17:34.277666 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:34.209570 30339 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:34.262612 30397 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.289868 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:34.290419 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.290761 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.292292 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.292600 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.293099 28982 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:34.293922 30267 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.294480 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56626 (request call id 189) took 1079 ms. Trace:
I20260709 12:17:34.294745 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.295534 28982 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:34.296957 28367 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:34.286778 30164 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4.
I20260709 12:17:34.298462 30332 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:34.298767 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.299867 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.300158 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1518) took 1507 ms. Trace:
I20260709 12:17:34.300540 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.300619 28433 rpcz_store.cc:276] 0709 12:17:32.792171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.795526 (+  3355us) service_pool.cc:224] Handling call
0709 12:17:32.795887 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.145665 (+349778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:33.145697 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:33.145757 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:33.145790 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.299758 (+1153968us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b5400086454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:34.299807 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.299958 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":803905920}
I20260709 12:17:34.301580 30332 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:34.301254 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:34.303697 28362 rpcz_store.cc:276] 0709 12:17:33.237623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.246422 (+  8799us) service_pool.cc:224] Handling call
0709 12:17:34.294242 (+1047820us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54002f44d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:34.294407 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1718541056}
W20260709 12:17:34.294802 28255 connection.cc:256] RPC call timeout handler was delayed by -1.3505602160000763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
W20260709 12:17:34.305083 28255 connection.cc:256] RPC call timeout handler was delayed by -1.2191602160000912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.169s for a response
W20260709 12:17:34.305397 28255 connection.cc:256] RPC call timeout handler was delayed by -1.0508602160000464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
W20260709 12:17:34.305430 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:34.306177 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.319766 28283 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:34.237344 30394 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:17:34.290709 28984 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.286952 29730 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:34.287315 30397 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.333720 28991 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 4 LEARNER]: Advancing to term 5
I20260709 12:17:34.287566 30402 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.337091 29990 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:34.393996 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:34.370678 30197 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.150s	user 0.009s	sys 0.012s
W20260709 12:17:34.397346 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 120007us
W20260709 12:17:34.395089 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 58253.6us
W20260709 12:17:34.395085 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 93922us
I20260709 12:17:34.302776 30332 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.334188 30346 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:34.404940 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.413411 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:34.418282 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 162465us
W20260709 12:17:34.399662 30197 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.179s	user 0.010s	sys 0.012s
I20260709 12:17:34.424153 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.425216 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.330579 30283 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.377179 30402 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:34.365527 30397 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.452042 28991 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 5 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:34.429558 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.507035 28365 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:34.537012 28362 raft_consensus.cc:2410] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:34.430940 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:34.446324 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:43086 (request call id 6) took 2384 ms. Trace:
W20260709 12:17:34.564220 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 121060us
I20260709 12:17:34.560667 28358 raft_consensus.cc:2410] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:34.446321 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:34.557374 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 286252us
I20260709 12:17:34.403791 30331 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.429713 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:34.505985 30293 consensus_meta.cc:287] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.001s
I20260709 12:17:34.565771 28291 rpcz_store.cc:276] 0709 12:17:32.110335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.582637 (+472302us) service_pool.cc:224] Handling call
0709 12:17:34.446021 (+1863384us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54000f2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be881853 00007f85c384f60a 00007f85bcdd57d4 00007f85bcdd5762 00007f85bcdd5691 00007f85bcdd560c 00007f85bcdd48c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:34.446171 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333934464}
I20260709 12:17:34.413480 30346 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.511440 29990 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.510504 30339 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:34.530885 30332 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.503413 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.464926 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.582954 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.507359 30292 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:34.585040 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:34.580777 28363 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:34.599931 28364 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:34.601634 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:34.586263 30408 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:34.602521 30382 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.583539 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:34.421497 30226 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.543083 30402 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.588335 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.589206 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.602783 30408 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:34.599767 28991 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.600270 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:34.596802 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 150606us
I20260709 12:17:34.616653 28362 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:34.617059 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:34.586159 29734 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.603137 30382 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.610213 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:34.436493 30106 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.449357 30389 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.614770 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.584098 30346 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.617851 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:34.581295 30293 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.607411 28986 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 2 LEARNER]: Advancing to term 4
I20260709 12:17:34.609669 28359 raft_consensus.cc:2393] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:34.662631 28363 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:34.597389 30235 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:34.617885 30392 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:34.634049 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:34.657764 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.623987 28358 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:17:34.623267 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 44180.2us
I20260709 12:17:34.610757 30339 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:34.624475 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:34.672425 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:34.673115 28437 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:34.673604 28360 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:34.673533 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 56559.6us
W20260709 12:17:34.637576 30393 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.480s	user 0.007s	sys 0.003s
I20260709 12:17:34.636092 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.627872 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.626634 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.635248 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.634886 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.600154 29730 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.690095 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:34.668859 28984 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:34.692554 28991 raft_consensus.cc:2410] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:34.674629 30392 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:34.733203 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1389) took 1383 ms. Trace:
I20260709 12:17:34.705708 28255 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:34.683513 28436 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:34.627629 30283 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.674471 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.775463 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.776562 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.776736 30324 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.705191 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.778788 28363 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:34.771499 28432 rpcz_store.cc:276] 0709 12:17:33.350328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.350548 (+   220us) service_pool.cc:224] Handling call
0709 12:17:34.007144 (+656596us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b58001d4698. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:34.011686 (+  4542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.123508 (+111822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.732903 (+609395us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b5400415354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:34.733131 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163251328}
I20260709 12:17:34.777645 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.772630 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:34.780831 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 75184.5us
I20260709 12:17:34.624327 30164 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:34.624125 29987 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.782207 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:34.782208 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 179) took 3999 ms. Trace:
I20260709 12:17:34.785138 28287 rpcz_store.cc:276] 0709 12:17:30.782809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.645890 (+1863081us) service_pool.cc:224] Handling call
0709 12:17:32.646349 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.173070 (+526721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.781914 (+1608844us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54000f2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:34.782137 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1200673152}
W20260709 12:17:34.773905 30388 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.866s	user 0.006s	sys 0.006s
I20260709 12:17:34.626426 30106 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:34.771471 30119 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.003s
W20260709 12:17:34.704764 30416 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:17:34.704972 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.695557 30235 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:34.788102 30162 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:17:34.788928 30197 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:34.789286 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1526) took 1589 ms. Trace:
I20260709 12:17:34.815518 28367 rpcz_store.cc:276] 0709 12:17:33.200134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.687371 (+487237us) service_pool.cc:224] Handling call
0709 12:17:34.296755 (+609384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.297392 (+   637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.789078 (+491686us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b54002f44d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:34.789215 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1372187392}
W20260709 12:17:34.789443 30111 consensus_meta.cc:287] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.000s
I20260709 12:17:34.781536 29734 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:17:34.784523 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:17:34.804852 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:34.805102 30398 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.642s	user 0.009s	sys 0.003s
I20260709 12:17:34.805585 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.759471 29814 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.793905 30331 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
W20260709 12:17:34.807642 30415 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
W20260709 12:17:34.810649 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.806792 28358 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:34.829308 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:34.629254 30411 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.157s
I20260709 12:17:34.814814 28437 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:34.785512 30339 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.805153 28432 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:34.817947 30197 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:34.831511 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.834172 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:34.819598 30111 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:34.844280 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.845646 28437 raft_consensus.cc:2368] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:17:34.816607 30162 consensus_queue.cc:1243] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:34.831842 30235 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:34.828904 29814 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.856747 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:34.855638 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 67950.6us
I20260709 12:17:34.857553 30164 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:17:34.860626 30286 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.709s	user 0.009s	sys 0.002s
I20260709 12:17:34.837260 30339 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.843097 30197 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:34.804922 30388 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:34.868670 28440 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:34.854262 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:34.872069 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.872021 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.862375 28986 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 4 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:34.879061 28366 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:34.880486 28358 raft_consensus.cc:2410] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:34.881759 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.883204 28438 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:34.884375 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.884989 28363 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:34.853358 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:34.889940 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1526) took 1064 ms. Trace:
I20260709 12:17:34.926551 28292 rpcz_store.cc:276] 0709 12:17:33.826020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.832150 (+  6130us) service_pool.cc:224] Handling call
0709 12:17:33.832584 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.224613 (+392029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.889669 (+665056us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54003ab454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:34.889864 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863731072}
I20260709 12:17:34.877660 30405 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:17:34.881683 30344 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:17:34.892445 30388 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.843289 29734 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
W20260709 12:17:34.917585 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.876502 30339 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.950183 30408 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:34.892889 30111 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.876868 30389 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.874807 30293 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:34.868963 30235 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:34.960708 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.964507 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.950213 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:34.966785 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:34.967337 30235 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:34.940037 30292 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:34.968117 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.968420 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.969189 28190 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 113) took 8391 ms. Trace:
I20260709 12:17:34.969436 28362 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:34.970171 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:34.970950 28984 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:34.972844 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:34.955266 28983 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.974738 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.976894 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.971728 28190 rpcz_store.cc:276] 0709 12:17:26.577794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.578036 (+   242us) service_pool.cc:224] Handling call
0709 12:17:34.968944 (+8390908us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:29.680649 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:29.781145 (+100496us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:29.783428 (+  2283us) write_op.cc:435] Acquired schema lock
0709 12:17:29.785090 (+  1662us) tablet.cc:662] Decoding operations
0709 12:17:29.954367 (+169277us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:29.961327 (+  6960us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:29.963498 (+  2171us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:17:30.068376 (+104878us) tablet.cc:701] Row locks acquired
0709 12:17:30.071710 (+  3334us) write_op.cc:260] PREPARE: finished
0709 12:17:30.127303 (+ 55593us) write_op.cc:270] Start()
0709 12:17:30.137301 (+  9998us) write_op.cc:276] Timestamp: P: 1783599450093707 usec, L: 0
0709 12:17:30.143945 (+  6644us) op_driver.cc:348] REPLICATION: starting
0709 12:17:30.243187 (+ 99242us) log.cc:844] Serialized 1293 byte log entry
0709 12:17:32.029841 (+1786654us) op_driver.cc:464] REPLICATION: finished
0709 12:17:32.171525 (+141684us) write_op.cc:301] APPLY: starting
0709 12:17:32.175354 (+  3829us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:32.175583 (+   229us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:32.175743 (+   160us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:32.176558 (+   815us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:32.176926 (+   368us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:17:32.176990 (+    64us) write_op.cc:312] APPLY: finished
0709 12:17:32.177388 (+   398us) log.cc:844] Serialized 35 byte log entry
0709 12:17:32.177632 (+   244us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:32.397489 (+219857us) write_op.cc:454] Released schema lock
0709 12:17:32.452965 (+ 55476us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:32.571573 (+118608us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:32.571630 (+    57us) write_op.cc:454] Released schema lock
0709 12:17:32.576845 (+  5215us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88432a 00007f85be809f14 00007f85be809ec3 00007f85be809e38 00007f85be809df3 00007f85be8090dc 00007f85be832e16 00007f85be81acbb 00007f85be80539d 00007f85be808d75
Metrics: {"mutex_wait_us":47288,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":121088,"apply.run_cpu_time_us":4003,"apply.run_wall_time_us":409960,"num_ops":3,"prepare.queue_time_us":339852,"prepare.run_cpu_time_us":17675,"prepare.run_wall_time_us":691202,"raft.queue_time_us":993962,"raft.run_cpu_time_us":13389,"raft.run_wall_time_us":807552,"replication_time_us":1859404,"spinlock_wait_cycles":491099904,"thread_start_us":1731863,"threads_started":4,"wal-append.queue_time_us":164345,"wal-append.run_cpu_time_us":38800,"wal-append.run_wall_time_us":2752613}]]}
I20260709 12:17:34.965210 28358 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:34.837144 28361 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:34.874760 29814 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:34.878293 30346 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:34.995011 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.955592 30382 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:35.002342 28366 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:17:34.993955 30417 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
W20260709 12:17:35.011096 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.018373 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.046875 28358 raft_consensus.cc:2410] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:34.968509 30292 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.051442 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:34.999284 30392 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:35.045271 28982 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5.
I20260709 12:17:35.045636 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:35.056516 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.066391 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:35.070142 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 146544us
I20260709 12:17:35.088214 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:35.091075 30164 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.091531 30164 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.091902 30164 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.103628 29734 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:17:35.105429 30292 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:35.105891 30332 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:35.103875 30162 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:17:35.106423 30332 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:35.107344 30332 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.117983 30332 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.103636 30421 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:35.103780 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.129431 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.134052 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:35.134850 28984 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 1.
W20260709 12:17:35.137936 30397 consensus_meta.cc:287] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.000s
I20260709 12:17:35.138458 30397 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.139528 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:35.140673 30397 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:35.140846 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:35.141455 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:35.141798 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:35.142130 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:35.142274 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.142561 28990 raft_consensus.cc:2368] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:17:35.142932 28439 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:35.143515 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.143869 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:35.144143 28283 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:35.147508 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:35.148284 28440 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:35.143072 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.149986 28292 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:35.151911 28437 raft_consensus.cc:2368] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:35.152514 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:35.140619 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.149921 28284 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:35.154713 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:35.156426 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:35.154433 28291 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:35.158325 28438 raft_consensus.cc:2368] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:35.149233 30421 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:35.160270 30421 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.164428 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:35.165096 28438 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:35.168735 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:35.170234 28440 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:35.171867 28257 connection.cc:256] RPC call timeout handler was delayed by -1.397363076999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
W20260709 12:17:35.172335 28257 connection.cc:256] RPC call timeout handler was delayed by -1.2992630769999778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
I20260709 12:17:35.191016 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:35.194741 29814 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:35.195088 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:35.195753 28435 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:35.196507 28435 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:35.195650 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:35.198594 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:35.199265 28366 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.199765 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:35.200495 28367 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:35.203575 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:35.206404 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 115814us
I20260709 12:17:35.217619 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1536) took 1166 ms. Trace:
I20260709 12:17:35.218114 28360 rpcz_store.cc:276] 0709 12:17:34.051022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.059016 (+  7994us) service_pool.cc:224] Handling call
0709 12:17:34.224387 (+165371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.674170 (+449783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.217332 (+543162us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54003700d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:35.217549 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032632192}
I20260709 12:17:35.220247 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:35.193856 30162 consensus_queue.cc:1243] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:35.217669 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:35.224965 28987 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 5 LEARNER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:17:35.227283 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:35.227977 28440 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.214318 30399 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:35.207882 29730 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:35.214738 30293 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.232079 30339 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.232964 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:35.234293 30402 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.003s
I20260709 12:17:35.234721 29734 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:35.202764 30331 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:17:35.222136 30346 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:35.235476 30399 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.244328 28983 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:35.243096 30428 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:35.264649 30422 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.375s	user 0.001s	sys 0.010s
I20260709 12:17:35.245796 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:35.288561 28366 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:35.291530 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.292325 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.248530 30399 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:35.246385 30402 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.227541 30197 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.241421 29730 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:35.246166 30426 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:35.241763 30293 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.300886 30339 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:35.293378 30226 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:35.307963 28440 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:35.305495 28364 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:35.319805 28360 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
W20260709 12:17:35.357897 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 185076us
I20260709 12:17:35.324082 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.360211 28362 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:35.325909 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.328773 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:35.362578 28991 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:35.328667 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:35.338869 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.342108 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:35.342692 30197 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:35.357188 30293 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.345181 28361 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:35.308516 30162 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:35.345759 29730 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:35.371129 28257 connection.cc:256] RPC call timeout handler was delayed by -1.2056630769998264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
I20260709 12:17:35.345997 30426 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:35.412315 28257 connection.cc:256] RPC call timeout handler was delayed by -1.1845674599999256s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
I20260709 12:17:35.344565 30402 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:35.383934 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:35.413944 28257 connection.cc:256] RPC call timeout handler was delayed by -1.1305674599998383s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
W20260709 12:17:35.415095 28257 connection.cc:256] RPC call timeout handler was delayed by -1.0010657089999313s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2844s for a response
I20260709 12:17:35.416973 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:35.368927 30331 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:35.420781 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.421103 28332 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:35.421876 28364 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:35.413215 30162 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:35.424928 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.383543 28982 raft_consensus.cc:1813] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:35.385190 28359 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:35.425695 28283 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:35.426669 28982 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 3 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:17:35.424516 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.427748 28284 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:35.424584 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:35.425043 30162 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:35.434131 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1543) took 1158 ms. Trace:
I20260709 12:17:35.434710 28286 rpcz_store.cc:276] 0709 12:17:34.275716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.123823 (+848107us) service_pool.cc:224] Handling call
0709 12:17:35.131786 (+  7963us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:35.154147 (+ 22361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.154181 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:35.170491 (+ 16310us) log.cc:844] Serialized 26 byte log entry
0709 12:17:35.360906 (+190415us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:35.360961 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.361053 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:35.433836 (+ 72783us) raft_consensus.cc:1685] finished
0709 12:17:35.433862 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:35.434049 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1555,"raft.queue_time_us":25773,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7848,"thread_start_us":165299,"threads_started":1,"wal-append.queue_time_us":244811}
I20260709 12:17:35.437350 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.438284 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.438798 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:35.439131 28285 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:35.439635 28359 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:35.355786 30164 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.412484 30426 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:35.457540 28332 leader_election.cc:400] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:17:35.458010 28332 leader_election.cc:403] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:35.437239 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.459968 28366 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:35.464479 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.466058 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.466554 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.467732 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.468447 28360 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:35.470216 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:35.470450 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 252860us
I20260709 12:17:35.472640 28439 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:35.456003 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.474841 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.383285 29734 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 12:17:35.473500 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:35.463518 30391 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:35.497108 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.476665 28332 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:35.499059 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.476729 28366 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:35.500818 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:35.501785 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.502440 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.502187 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:35.472301 30226 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:35.499426 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.504077 28286 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:17:35.504700 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 168629us
I20260709 12:17:35.506984 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:35.504110 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.505656 28330 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:35.508819 28289 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:35.465660 30399 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.517124 28365 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:35.517864 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.518607 28284 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:35.520908 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.522994 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:35.523643 28364 raft_consensus.cc:1813] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:35.524209 28364 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 3 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:17:35.521118 28291 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:35.500388 30391 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:35.529256 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.530671 28286 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:35.475498 30402 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:17:35.510573 30339 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.507205 30414 raft_consensus.cc:2764] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:35.533514 28287 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:17:35.535756 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.539574 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:35.542558 28360 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:17:35.544353 28364 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:35.565416 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.517169 29814 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:35.594236 28358 raft_consensus.cc:2368] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:35.526552 29734 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:35.533321 30391 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.597594 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:35.585850 30339 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:35.607661 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.611325 28360 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:35.590086 30435 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.596853 29734 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:35.618304 28364 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:17:35.606153 29814 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.625559 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:35.610527 30339 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.632158 28291 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:35.644270 28286 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:35.624019 30197 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:35.620200 30435 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:35.657588 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 152001us
I20260709 12:17:35.658835 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:35.629928 30443 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:17:35.634604 29814 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.658668 30435 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.662699 30402 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:35.684216 30164 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:35.675954 30438 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:35.712109 30446 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:17:35.718658 29814 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:35.719233 30162 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.720448 30438 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.724664 30426 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.714095 30399 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:35.683954 29734 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.731792 29734 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.732257 30438 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:35.732472 30426 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:35.735298 30162 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:35.735533 29730 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.729871 30226 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:17:35.739046 30435 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.741039 30293 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.742100 30339 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.743989 30419 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:35.743393 30197 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:35.761838 30428 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.762413 30293 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:35.762463 30428 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.756126 30419 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:35.749408 30391 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.766244 30435 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:35.761667 29730 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:35.771792 30419 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:35.778158 30419 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.828683 30428 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:35.759806 30197 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:36.813210 30428 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:36.812564 30391 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:37.164357 30339 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:37.269536 30419 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 1.491s	user 0.008s	sys 0.004s
W20260709 12:17:37.435461 30455 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:17:37.441298 28441 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:37.444579 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 219) took 1792 ms. Trace:
I20260709 12:17:37.445392 28441 rpcz_store.cc:276] 0709 12:17:35.652344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.652529 (+   185us) service_pool.cc:224] Handling call
0709 12:17:35.662389 (+  9860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.441919 (+1779530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.444226 (+  2307us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b5400184054. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:37.444493 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4090580864}
I20260709 12:17:37.447463 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 202) took 2120 ms. Trace:
I20260709 12:17:37.447803 28366 rpcz_store.cc:276] 0709 12:17:35.346808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.536800 (+189992us) service_pool.cc:224] Handling call
0709 12:17:35.537116 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:35.537467 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.537494 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:35.538051 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 12:17:35.616210 (+ 78159us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:35.616250 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.616322 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:37.447226 (+1830904us) raft_consensus.cc:1685] finished
0709 12:17:37.447244 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.447393 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":162,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":77779,"threads_started":1,"wal-append.queue_time_us":117653,"wal-append.run_cpu_time_us":2015,"wal-append.run_wall_time_us":1066625}
W20260709 12:17:37.545912 28365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.952s	user 0.008s	sys 0.020s
W20260709 12:17:37.546597 28365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.957s	user 0.008s	sys 0.020s
I20260709 12:17:37.547544 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 201) took 2249 ms. Trace:
I20260709 12:17:37.547931 28365 rpcz_store.cc:276] 0709 12:17:35.302093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.534863 (+232770us) service_pool.cc:224] Handling call
0709 12:17:35.535136 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.541988 (+  6852us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:35.589308 (+ 47320us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.547107 (+1957799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:37.547174 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:37.547229 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.547308 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.547477 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7062,"prepare.queue_time_us":51296,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":128,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":1957374,"threads_started":1,"wal-append.queue_time_us":177052,"wal-append.run_cpu_time_us":1299,"wal-append.run_wall_time_us":1031588}
I20260709 12:17:37.558102 30331 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:37.558758 30331 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.567272 30331 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:37.569041 30331 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:37.576627 28362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.034s	user 0.015s	sys 0.011s
W20260709 12:17:37.577085 28362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.034s	user 0.015s	sys 0.011s
I20260709 12:17:37.577991 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1557) took 2216 ms. Trace:
I20260709 12:17:37.578408 28362 rpcz_store.cc:276] 0709 12:17:35.361516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.541009 (+179493us) service_pool.cc:224] Handling call
0709 12:17:35.541375 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:35.541788 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.541818 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:35.542447 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.577501 (+2035054us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:37.577556 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.577651 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:37.577698 (+    47us) raft_consensus.cc:1685] finished
0709 12:17:37.577710 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.577916 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6783,"raft.queue_time_us":44584,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":9993,"thread_start_us":2034600,"threads_started":1,"wal-append.queue_time_us":1673418}
I20260709 12:17:37.592269 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.593766 30331 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.594802 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1458) took 1953 ms. Trace:
I20260709 12:17:37.596288 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:37.601482 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:37.602698 28288 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:37.595258 28291 rpcz_store.cc:276] 0709 12:17:35.641244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.641403 (+   159us) service_pool.cc:224] Handling call
0709 12:17:37.590086 (+1948683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.590516 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.590552 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:37.591022 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.594410 (+  3388us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:37.594453 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.594529 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:37.594581 (+    52us) raft_consensus.cc:1685] finished
0709 12:17:37.594594 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.594734 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":3023,"threads_started":1,"wal-append.queue_time_us":914}
I20260709 12:17:37.587282 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:37.604522 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:37.607635 28288 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.607734 30331 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:37.604455 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:37.610930 28291 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.606876 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.616199 28288 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:37.617492 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:37.617970 30421 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:37.618458 28288 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:37.619437 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:37.619606 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:37.621670 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.620076 28288 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:37.622206 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.625335 28291 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:37.625983 30421 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.626555 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:37.627425 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.627898 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:37.629124 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:37.629369 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:37.630101 28283 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:37.630347 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:37.630971 28291 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.631919 30428 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.632498 30421 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:37.632416 30428 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:37.637845 28288 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:37.638761 28291 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:37.639053 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:37.639485 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:37.640033 28257 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.640846 30421 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:37.642140 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:37.642949 30421 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:37.643384 30421 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:37.643774 30421 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.645442 30428 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:37.646956 30428 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.649003 30428 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:37.653398 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:37.654140 28441 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.652063 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:37.655704 28438 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.655364 30421 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:37.657658 30421 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:37.658941 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.662382 28292 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:37.663096 30197 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:37.662281 28441 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:37.664806 30197 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:37.666501 30432 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.667111 30432 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:37.669044 28288 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:37.670115 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.669334 30432 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:37.670372 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:37.670624 28283 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1450) took 2191 ms (client timeout 1539 ms). Trace:
I20260709 12:17:37.671250 28441 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:37.663828 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.671607 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:37.672268 30421 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.672498 28432 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:37.672811 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.673624 30197 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.655421 30428 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:37.671257 28283 rpcz_store.cc:269] 0709 12:17:35.479098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.479209 (+   111us) service_pool.cc:224] Handling call
0709 12:17:37.629240 (+2150031us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b5800000098. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762eee 000000000036eb39 00007f85c37d61e0 00007f85c389f0c4 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:37.670537 (+ 41297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3598720}
I20260709 12:17:37.676506 28435 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:37.669024 28359 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
W20260709 12:17:37.688318 28359 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56626 (request call id 196) took 2392 ms (client timeout 2208 ms). Trace:
W20260709 12:17:37.688632 28359 rpcz_store.cc:269] 0709 12:17:35.296370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.440261 (+143891us) service_pool.cc:224] Handling call
0709 12:17:37.688122 (+2247861us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b5400127354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:37.688238 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4998086528}
I20260709 12:17:37.690486 28435 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.677989 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:37.678628 30391 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.694305 28290 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:37.694167 30391 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:37.694813 28438 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:37.695869 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:37.696637 28438 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:37.696713 30391 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:37.698422 28435 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:17:37.699196 30332 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:17:37.702338 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1428) took 2535 ms. Trace:
W20260709 12:17:37.680790 28367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.082s	user 0.023s	sys 0.040s
W20260709 12:17:37.703104 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:37.703258 28367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.107s	user 0.024s	sys 0.041s
W20260709 12:17:37.703806 28332 connection.cc:256] RPC call timeout handler was delayed by -1.3703928039999482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
I20260709 12:17:37.704175 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 193) took 2437 ms. Trace:
I20260709 12:17:37.704453 30464 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.704876 28367 rpcz_store.cc:276] 0709 12:17:35.266541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.266864 (+   323us) service_pool.cc:224] Handling call
0709 12:17:35.293055 (+ 26191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:35.293333 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:35.496842 (+203509us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.634433 (+2137591us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b18003fed40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:37.703763 (+ 69330us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:37.703830 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:37.703889 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.703967 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.704109 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14481,"prepare.queue_time_us":36669,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":16134,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":9624960,"thread_start_us":2341663,"threads_started":3,"wal-append.queue_time_us":105697,"wal-append.run_cpu_time_us":2454,"wal-append.run_wall_time_us":1120639}
I20260709 12:17:37.705056 30464 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:37.709213 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:37.710137 30391 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:37.710579 30391 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.710924 30391 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.680754 30421 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:37.682809 30197 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.684551 28257 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.714370 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:37.714941 28439 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1423) took 2561 ms (client timeout 1758 ms). Trace:
W20260709 12:17:37.715389 28439 rpcz_store.cc:269] 0709 12:17:35.153464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.153977 (+   513us) service_pool.cc:224] Handling call
0709 12:17:37.714705 (+2560728us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54002e40d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:37.714851 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697001344}
I20260709 12:17:37.716333 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:37.686168 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:37.679242 30292 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.717530 28441 raft_consensus.cc:2410] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:37.717913 30331 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.716678 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:37.719700 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:37.720223 30421 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:37.720911 30419 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:37.719837 30391 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:37.726753 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:37.728464 28439 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:37.697484 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:37.732110 28441 raft_consensus.cc:2368] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4.
I20260709 12:17:37.732254 28438 raft_consensus.cc:2368] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7.
W20260709 12:17:37.735060 28255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.127s	user 0.012s	sys 0.020s
W20260709 12:17:37.735641 28255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.129s	user 0.013s	sys 0.021s
W20260709 12:17:37.736272 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 2.13939e+06us
I20260709 12:17:37.724386 28435 rpcz_store.cc:276] 0709 12:17:35.167000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.167190 (+   190us) service_pool.cc:224] Handling call
0709 12:17:35.176379 (+  9189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.674980 (+2498601us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b4c004b56c0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bcb53a20 00007f85bcb64a29 00007f85bcb2a644 00007f85bcb51f2d 00007f85bcb51b93 00007f85be88218b 00007f85be882cb5 00007f85be87df30 00007f85be87f7de 00007f85be889b49 00007f85be88a416
0709 12:17:37.698975 (+ 23995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.702251 (+  3276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12895488}
I20260709 12:17:37.740254 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:37.728411 30292 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:37.740558 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:37.734721 30419 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:37.745645 30419 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:37.744748 28435 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:37.709790 30464 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:37.754531 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:37.755336 28439 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.758843 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:37.759591 28432 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:37.761415 30391 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:37.763195 30292 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:37.759471 30421 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:37.763919 28255 connection.cc:256] RPC call timeout handler was delayed by -1.77741336400004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
I20260709 12:17:37.764560 30421 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.765201 30421 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:37.765309 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:37.766162 28433 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:37.764618 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1388) took 2661 ms. Trace:
I20260709 12:17:37.772101 30470 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:37.772243 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52609.3us
I20260709 12:17:37.772778 30470 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:37.769873 28363 rpcz_store.cc:276] 0709 12:17:35.102785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.102996 (+   211us) service_pool.cc:224] Handling call
0709 12:17:35.108104 (+  5108us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:35.108501 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.108533 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:35.118103 (+  9570us) log.cc:844] Serialized 52 byte log entry
0709 12:17:35.171001 (+ 52898us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:35.171342 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 12:17:35.171544 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.171621 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:35.537261 (+365640us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b1800037300. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:37.548042 (+2010781us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b1800037300. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:37.764360 (+216318us) raft_consensus.cc:1685] finished
0709 12:17:37.764385 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.764550 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":185,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":2821,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":12463,"spinlock_wait_cycles":22162560,"thread_start_us":52455,"threads_started":1,"wal-append.queue_time_us":70280}
I20260709 12:17:37.766206 30421 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:37.780093 30331 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.787804 30331 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:37.788666 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1391) took 2638 ms. Trace:
I20260709 12:17:37.773242 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:37.789096 28361 rpcz_store.cc:276] 0709 12:17:35.150783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.154708 (+  3925us) service_pool.cc:224] Handling call
0709 12:17:35.174126 (+ 19418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:35.384871 (+210745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.788361 (+2403490us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400126e54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:37.788597 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392734592}
I20260709 12:17:37.788432 30331 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:37.793711 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 72088.4us
I20260709 12:17:37.794647 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:37.795529 28332 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:37.796042 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:37.796280 30464 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:37.796984 30292 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.797346 30292 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:37.797696 30292 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.797936 30468 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.798372 30468 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.798717 30468 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.800079 30464 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:37.801148 30464 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:37.803970 28285 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.188s	user 0.013s	sys 0.025s
W20260709 12:17:37.804600 28285 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.189s	user 0.014s	sys 0.025s
I20260709 12:17:37.805819 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.808017 30414 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:37.808565 30414 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:37.808997 30414 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:37.810045 30331 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:37.810405 30468 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:37.808565 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.816099 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.812214 30332 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.812719 30463 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:37.780568 28439 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:37.818658 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:37.814599 30467 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:37.825176 30331 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:37.833448 30421 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:37.836225 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:37.837543 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1442) took 2385 ms. Trace:
I20260709 12:17:37.843027 28330 leader_election.cc:400] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:17:37.846277 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:37.843116 28285 rpcz_store.cc:276] 0709 12:17:35.452281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.463965 (+ 11684us) service_pool.cc:224] Handling call
0709 12:17:35.473236 (+  9271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:35.495400 (+ 22164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:35.495452 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:35.505953 (+ 10501us) log.cc:844] Serialized 26 byte log entry
0709 12:17:35.593048 (+ 87095us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:37.804973 (+2211925us) log.cc:844] Serialized 14 byte log entry
0709 12:17:37.837145 (+ 32172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.837239 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:37.837287 (+    48us) raft_consensus.cc:1685] finished
0709 12:17:37.837299 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:37.837474 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":161324,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":26487,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":5831,"thread_start_us":2281040,"threads_started":3,"wal-append.queue_time_us":147401,"wal-append.run_cpu_time_us":2347,"wal-append.run_wall_time_us":1059817}
W20260709 12:17:37.848887 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.854800 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:17:37.956681 28255 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 1.965s (SENT) [suppressed 13 similar messages]
W20260709 12:17:37.996354 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.841914 30469 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.981559 30473 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.991822 30477 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:37.841784 30197 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:37.837011 30470 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:37.846809 30339 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.825568 30468 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:38.000607 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 145880us
I20260709 12:17:38.007236 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:37.837278 30332 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:37.837131 30463 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.013063 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:38.000234 30331 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:38.003374 30464 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.026386 30235 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:38.027029 30235 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:38.028434 30235 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.029819 30463 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:38.031085 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.005118 30469 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:38.033505 30469 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:38.035240 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 202688us
I20260709 12:17:38.036566 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:38.005421 30473 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.042105 30473 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:38.042892 30331 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.045768 30235 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.015358 30470 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.050338 30473 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:38.050633 30332 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:38.051594 30473 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:38.052624 30473 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.046298 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:38.057891 28257 leader_election.cc:400] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
W20260709 12:17:38.060868 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.061434 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.060139 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.063529 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:38.064646 28257 leader_election.cc:400] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:38.064872 28257 leader_election.cc:403] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:38.065465 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:38.069514 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:38.068674 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 222582us
I20260709 12:17:38.070255 28332 leader_election.cc:400] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4.
I20260709 12:17:38.055652 30414 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.191414 30414 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:38.060339 30470 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.069149 30468 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.078133 28956 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:38.206470 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:38.207186 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 129149us
I20260709 12:17:38.207721 30485 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:38.083099 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:38.209594 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 126578us
I20260709 12:17:38.212687 30419 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:38.074007 30472 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:38.075913 30467 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
W20260709 12:17:38.081218 28333 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.493s	user 0.000s	sys 0.050s
I20260709 12:17:38.214030 30472 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:38.090320 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 133692us
I20260709 12:17:38.092849 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:38.215099 30467 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.215180 28989 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:38.093655 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:38.216132 28291 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:38.069862 30477 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:38.103659 30464 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:38.104504 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.217918 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.070688 30331 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.219208 30332 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.219504 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.219738 30332 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.220413 28291 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:38.221918 30463 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.222527 30463 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.223539 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.224722 30463 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:38.110275 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:38.124082 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:38.095511 28330 connection.cc:256] RPC call timeout handler was delayed by -2.4299802360001195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
I20260709 12:17:38.126149 30479 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:38.225803 28434 raft_consensus.cc:2410] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:38.227298 30479 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:38.136197 28287 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1443) took 2683 ms (client timeout 2262 ms). Trace:
I20260709 12:17:38.146684 30427 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:38.151594 28186 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 1 to 2, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:38.228628 30427 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:38.229409 28186 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:38.172416 30339 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:38.230628 28330 connection.cc:256] RPC call timeout handler was delayed by -2.3507802359999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
I20260709 12:17:38.230739 30339 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.071645 28332 leader_election.cc:403] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 12:17:38.231199 28330 connection.cc:256] RPC call timeout handler was delayed by -2.2886802360001184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 12:17:38.231638 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 161565us
W20260709 12:17:38.232359 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 108378us
I20260709 12:17:38.066802 30292 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.198616 28292 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:17:38.233142 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.202744 28289 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:38.231876 30391 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:38.234736 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.234505 30391 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.216562 28440 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:38.236791 30391 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:38.214561 30472 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.216025 30467 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:17:38.231585 28330 connection.cc:256] RPC call timeout handler was delayed by -2.2742802360000951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
W20260709 12:17:38.228169 28287 rpcz_store.cc:269] 0709 12:17:35.452709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.464175 (+ 11466us) service_pool.cc:224] Handling call
0709 12:17:38.135820 (+2671645us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b540035f7d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:38.136068 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154228480}
I20260709 12:17:38.204919 28285 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:38.237699 28330 connection.cc:256] RPC call timeout handler was delayed by -2.24638023600005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
I20260709 12:17:38.082257 30478 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:38.207442 28284 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:38.238327 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.229633 30427 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.239120 28438 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.238520 30478 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:38.239691 30478 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.240622 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:38.214633 28333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.626s	user 0.001s	sys 0.051s
I20260709 12:17:38.234555 30292 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:38.239385 30427 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.245983 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.246659 30469 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.247030 28434 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:38.233860 30473 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.247898 28363 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:38.248096 30339 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.249444 28367 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:38.252527 30467 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:38.231783 28186 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:38.247210 30469 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.232762 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:38.238101 28330 connection.cc:256] RPC call timeout handler was delayed by -2.2436802360000456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
W20260709 12:17:38.260512 28330 connection.cc:256] RPC call timeout handler was delayed by -2.236480236000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
W20260709 12:17:38.260911 28330 connection.cc:256] RPC call timeout handler was delayed by -2.2202802360000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:17:38.259697 28954 connection.cc:256] RPC call timeout handler was delayed by -2.6790491459998975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 12:17:38.263262 28954 connection.cc:256] RPC call timeout handler was delayed by -2.6484491460000754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
W20260709 12:17:38.263752 28954 connection.cc:256] RPC call timeout handler was delayed by -1.9204969789998358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260709 12:17:38.264153 28954 connection.cc:256] RPC call timeout handler was delayed by -1.7899969789998522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
I20260709 12:17:38.263968 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:38.264747 28358 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.265457 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:38.266170 28365 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.266141 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:38.277889 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 69480us
I20260709 12:17:38.280465 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:38.279486 28284 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:38.282380 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.283048 28432 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.284116 28439 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:38.284256 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.285027 28437 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.284014 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.291827 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:38.292575 28367 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:38.296813 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:38.297621 28362 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.298703 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:38.299458 28360 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:38.300627 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:38.301951 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.302488 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.303234 28435 raft_consensus.cc:2376] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5. Re-sending same reply.
I20260709 12:17:38.290743 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:38.304522 28364 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.304752 28433 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.305554 28438 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:38.256680 30197 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:38.306888 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:38.312517 30197 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:38.312996 28440 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:38.313715 30197 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.315357 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:38.316045 30332 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:38.316642 30332 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:38.317188 30332 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:38.317027 30421 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:38.316637 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.301254 30477 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:17:38.262946 30428 raft_consensus.cc:2764] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:38.272044 30479 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.327762 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:38.329921 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.333966 30331 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:38.334474 30331 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:38.334738 30463 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:38.335155 30463 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:38.335510 30463 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.336038 30331 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:38.337373 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.339277 30432 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:38.339965 30488 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:38.340435 30488 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:38.340794 30488 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.330647 30464 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:38.343993 30469 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:38.345176 30331 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.331745 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:38.346453 28359 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.346859 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.305903 30419 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:38.282331 30485 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:38.363356 28365 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:38.318074 30339 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:38.372102 28362 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:38.378485 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:38.379305 28362 raft_consensus.cc:2368] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:38.377985 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:38.353278 28364 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:38.318223 30421 raft_consensus.cc:3055] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:38.373687 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:38.375229 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.374543 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.374667 28438 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:17:38.382400 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 122364us
I20260709 12:17:38.382445 28435 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.376426 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:38.384104 28365 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.348584 30464 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:38.385540 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.392225 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.397985 28289 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:38.398473 28367 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:38.399325 28332 leader_election.cc:400] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7.
I20260709 12:17:38.399722 28332 leader_election.cc:403] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:17:38.371512 30197 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.332180 30479 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:38.400846 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.371130 30419 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:38.400740 28440 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:38.401526 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.318164 30332 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.416407 28365 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:38.418865 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.435654 28290 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:38.436750 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.438019 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.438730 28288 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:38.439707 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.419653 28292 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:38.441397 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.437448 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.442132 28292 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:38.442744 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.390578 30421 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:38.422314 28437 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
W20260709 12:17:38.423372 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 157604us
I20260709 12:17:38.414855 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.401397 28360 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:17:38.463677 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 116913us
I20260709 12:17:38.471918 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:38.442807 28286 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:38.443903 28255 leader_election.cc:400] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:38.474696 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.397190 30464 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.477232 28433 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:38.478358 28441 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:38.478679 28330 connection.cc:256] RPC call timeout handler was delayed by -2.1662802360001479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18320000000000003s for a response
W20260709 12:17:38.479095 28330 connection.cc:256] RPC call timeout handler was delayed by -2.0969802360000358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
I20260709 12:17:38.433187 28435 raft_consensus.cc:2410] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:38.444952 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:38.442198 28287 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:38.486943 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:38.487121 30235 raft_consensus.cc:2764] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:38.489126 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.502542 28289 raft_consensus.cc:2410] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:17:38.479398 28330 connection.cc:256] RPC call timeout handler was delayed by -2.08708023600002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
I20260709 12:17:38.491289 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.475752 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.538571 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:38.558933 28290 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:17:38.539124 30430 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:17:38.540632 28288 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:38.547350 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.442773 30331 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:38.548080 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:38.548777 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:38.552877 28358 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:38.424175 30339 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:38.481981 30479 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.378108 30478 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.476809 30468 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:38.559479 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.575450 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:38.576104 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.576709 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.558329 28330 connection.cc:256] RPC call timeout handler was delayed by -2.0177802360001351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
W20260709 12:17:38.577915 28330 connection.cc:256] RPC call timeout handler was delayed by -2.0023219409999911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
W20260709 12:17:38.578261 28330 connection.cc:256] RPC call timeout handler was delayed by -1.9925802360000944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
W20260709 12:17:38.578621 28330 connection.cc:256] RPC call timeout handler was delayed by -1.9241802359999838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
W20260709 12:17:38.578881 28330 connection.cc:256] RPC call timeout handler was delayed by -1.8751719050001157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 12:17:38.579134 28330 connection.cc:256] RPC call timeout handler was delayed by -1.8494802360000904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
W20260709 12:17:38.579412 28330 connection.cc:256] RPC call timeout handler was delayed by -1.7792802359999769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2262s for a response
W20260709 12:17:38.579671 28330 connection.cc:256] RPC call timeout handler was delayed by -1.7607698680001249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 12:17:38.579926 28330 connection.cc:256] RPC call timeout handler was delayed by -1.7200259740000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
W20260709 12:17:38.580181 28330 connection.cc:256] RPC call timeout handler was delayed by -1.7161219409999831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
W20260709 12:17:38.580453 28330 connection.cc:256] RPC call timeout handler was delayed by -1.5911802360001275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24710000000000001s for a response
W20260709 12:17:38.580698 28330 connection.cc:256] RPC call timeout handler was delayed by -1.5294821089999004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 12:17:38.580936 28330 connection.cc:256] RPC call timeout handler was delayed by -1.5122020350002003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
W20260709 12:17:38.581175 28330 connection.cc:256] RPC call timeout handler was delayed by -1.2171698680001555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
W20260709 12:17:38.581460 28330 connection.cc:256] RPC call timeout handler was delayed by -1.1365219409999554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2565s for a response
I20260709 12:17:38.489895 30428 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:38.486981 30470 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:38.581702 28330 connection.cc:256] RPC call timeout handler was delayed by -1.0937558670000271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
I20260709 12:17:38.583256 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.541581 28359 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
W20260709 12:17:38.587958 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 112288us
I20260709 12:17:38.587759 30339 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:38.594322 30468 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:38.594614 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.595685 30468 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:38.597133 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 206652us
I20260709 12:17:38.597869 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.597891 28332 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:38.602130 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.575032 30481 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:38.633237 28361 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:38.634964 30508 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:38.600135 28186 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 1 to 3, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:38.589936 30331 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:38.588320 30467 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:17:38.584592 30428 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:38.638526 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:38.599125 30506 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:38.547088 30472 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.648947 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.626675 30498 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.649121 28186 catalog_manager.cc:5697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 2 to 3, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:38.637912 30496 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:38.637895 30487 raft_consensus.cc:2764] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:38.654567 28361 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:38.659987 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 72566.2us
I20260709 12:17:38.650174 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:38.644953 30414 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.669001 30498 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:38.685895 30498 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.640270 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:38.690344 28985 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:38.692526 30488 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.565249 30419 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.655082 30331 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:38.694424 28441 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:38.695319 30483 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:38.695937 30483 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:38.696573 30472 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:38.696913 30483 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.543128 30421 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.557104 30485 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.702462 28291 raft_consensus.cc:2410] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:38.644851 30339 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.707424 30478 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:38.652545 30428 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.669150 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:38.713922 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.714828 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:38.715849 28432 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:38.746425 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:38.747129 28432 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:38.715716 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:38.748335 28439 raft_consensus.cc:2410] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:17:38.716082 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.708212 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.730708 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 258885us
I20260709 12:17:38.745291 28438 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:38.740929 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.761307 28438 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
W20260709 12:17:38.762935 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 160926us
I20260709 12:17:38.751017 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.770162 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:38.764107 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:38.681555 28989 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.683884 30414 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:38.771871 28432 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:38.772318 28441 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:38.668404 30512 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:38.784626 30487 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:38.785167 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:38.785717 30487 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:38.786049 28358 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 2: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:38.786139 30487 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:38.771167 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:38.781244 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.787478 30487 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.788925 30487 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:38.806197 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 150450us
I20260709 12:17:38.738668 30506 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:38.812867 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:38.744902 30504 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:38.818111 30523 raft_consensus.cc:2764] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:38.770099 30427 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.777448 30472 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:38.830024 30427 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.831342 30506 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.830483 30472 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:38.833624 30472 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.807446 30421 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.823849 30512 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:38.836791 30512 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.824544 30467 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.826813 30331 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.808295 30508 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 12:17:38.829504 30473 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.004s
I20260709 12:17:38.842171 30473 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4.
I20260709 12:17:38.829823 30504 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:38.842903 30488 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:38.843633 30504 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:38.803145 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.841625 30508 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:38.847604 30508 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:38.848693 30508 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:38.858302 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 87909us
I20260709 12:17:38.859305 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:38.781755 30414 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.869658 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:38.806666 30332 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.874728 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.875803 28435 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:38.882097 28364 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
W20260709 12:17:38.914834 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 127324us
I20260709 12:17:38.916103 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:38.914924 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 104272us
I20260709 12:17:38.918748 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:38.919668 30432 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:38.920254 30432 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:38.921283 30432 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:38.814445 30463 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:38.939553 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:38.905702 30332 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:38.909870 30508 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.911602 30468 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:38.946594 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:38.905329 30509 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:38.949183 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.958019 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 309157us
I20260709 12:17:38.929108 30427 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:38.959147 28441 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:38.961342 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.963586 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:38.932775 30470 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:38.981302 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.948333 30332 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.907409 30507 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 12:17:38.989853 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:38.994405 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:39.000772 28358 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:38.993330 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 77297.1us
I20260709 12:17:39.002130 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:38.983960 30427 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:38.997908 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:38.988662 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:38.973608 30526 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:39.027320 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:17:39.026361 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.959561 30479 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.020874 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:17:39.051898 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.994676 30529 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:39.001776 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:39.052855 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 92089.7us
I20260709 12:17:39.003211 30427 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.047322 30526 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:39.056854 30481 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7.
I20260709 12:17:39.070680 30526 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:39.080447 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08002ed860 after lost signal to thread 27119
I20260709 12:17:39.062513 30470 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.058475 30498 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:39.082485 30520 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:17:39.081192 30524 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:17:39.094338 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800327a80 after lost signal to thread 28224
W20260709 12:17:39.114115 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 158686us
I20260709 12:17:39.115552 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:39.127089 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08002cf4a0 after lost signal to thread 28316
W20260709 12:17:39.134027 29012 debug-util.cc:398] Leaking SignalData structure 0x7b080017d360 after lost signal to thread 28391
W20260709 12:17:39.134557 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08001b4be0 after lost signal to thread 28392
W20260709 12:17:39.135998 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08003ce7e0 after lost signal to thread 28465
W20260709 12:17:39.147233 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 145552us
W20260709 12:17:39.148314 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:39.154022 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08002296c0 after lost signal to thread 29015
W20260709 12:17:39.154959 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800351720 after lost signal to thread 29016
I20260709 12:17:39.150945 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.151212 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.167883 28292 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:39.150051 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.171680 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.175693 28433 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:17:39.161369 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.148139 30467 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:39.148876 30532 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:39.151659 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:39.217195 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 162668us
I20260709 12:17:39.066574 30517 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.120s
I20260709 12:17:39.171463 30529 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.173727 30463 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.151648 30468 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:39.149030 30507 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:39.228436 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.154120 30514 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.107s
I20260709 12:17:39.232079 30518 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.232405 30292 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:39.233610 30510 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:39.269663 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:39.238463 30535 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:17:39.238992 30463 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:39.230799 30432 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:17:39.205176 30532 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.246505 30468 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:39.252058 30501 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.695s	user 0.005s	sys 0.000s
I20260709 12:17:39.254766 30479 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:39.253568 30464 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.731s	user 0.014s	sys 0.000s
I20260709 12:17:39.256708 30414 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.260900 28989 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
W20260709 12:17:39.253571 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.262126 30518 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:39.271664 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 96571.3us
I20260709 12:17:39.269647 30510 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:39.294661 28991 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:39.274169 30463 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.281091 30467 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:39.280866 30532 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:39.285210 30534 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:17:39.281442 30541 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:17:39.269279 30488 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
W20260709 12:17:39.283676 30536 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:17:39.276780 28359 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:39.291980 30464 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:39.301061 30427 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:39.313100 28361 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:39.315083 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:39.301895 30463 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.276844 30525 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:17:39.302416 30432 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:39.366452 30432 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:39.271647 30507 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.372546 28365 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:39.362641 30391 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:39.383324 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 68330.8us
W20260709 12:17:39.381350 30428 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.607s	user 0.019s	sys 0.012s
I20260709 12:17:39.380442 30545 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:39.363125 30472 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:17:39.412446 30489 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:39.414544 30432 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.416141 30526 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.418413 30555 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.384374 30391 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.425419 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:39.487614 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 62266.8us
I20260709 12:17:39.403275 30559 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:39.470192 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 135136us
W20260709 12:17:39.470744 30549 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:17:39.471803 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 320213us
W20260709 12:17:39.473775 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 211195us
I20260709 12:17:39.543782 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.543536 30555 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.403970 30498 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.546025 30419 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:39.546239 30516 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.492s	user 0.000s	sys 0.002s
I20260709 12:17:39.546856 30559 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.420939 30518 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.549579 28988 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:39.334201 30510 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.422909 30545 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:17:39.479333 30543 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:17:39.468071 30428 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:39.528427 30550 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:17:39.570706 28359 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:39.572201 30538 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:17:39.486517 30519 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.448s	user 0.002s	sys 0.000s
W20260709 12:17:39.529500 30546 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:17:39.528110 30473 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:39.488798 30391 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:39.580993 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.546054 30525 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:39.579994 30538 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:39.597787 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:39.598898 30473 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.610684 30510 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:39.646458 30526 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.616248 30538 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.601801 30545 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.621210 28359 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:39.623167 28363 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:39.649186 30545 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:39.641443 30542 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 12:17:39.651028 30542 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:39.651420 30542 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:39.623376 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:39.623974 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.624380 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.654328 28362 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:39.626693 30503 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.629210 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:17:39.640908 30554 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:17:39.641093 30473 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.662979 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1440) took 1357 ms. Trace:
I20260709 12:17:39.663539 28436 rpcz_store.cc:276] 0709 12:17:38.305176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.371526 (+ 66350us) service_pool.cc:224] Handling call
0709 12:17:38.397707 (+ 26181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:38.561853 (+164146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:38.561891 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:38.583752 (+ 21861us) log.cc:844] Serialized 26 byte log entry
0709 12:17:38.850974 (+267222us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:38.851019 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.851101 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:39.419165 (+568064us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:39.662632 (+243467us) raft_consensus.cc:1685] finished
0709 12:17:39.662655 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:39.662898 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123596,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":18037,"spinlock_wait_cycles":72765312,"thread_start_us":355189,"threads_started":2,"wal-append.queue_time_us":346255}
I20260709 12:17:39.641691 30464 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:39.643640 30507 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:39.666296 30526 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:39.672161 30332 consensus_meta.cc:287] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.676s	user 0.008s	sys 0.006s
I20260709 12:17:39.657913 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:39.637699 30414 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:39.652336 30542 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.657513 28286 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
W20260709 12:17:39.681408 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.682730 28991 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:39.683503 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.684542 28290 raft_consensus.cc:2376] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5. Re-sending same reply.
I20260709 12:17:39.685606 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.686468 28290 raft_consensus.cc:2410] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:39.653028 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:39.703503 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:39.699206 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.704463 28290 raft_consensus.cc:2410] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:39.698530 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.708904 28404 leader_election.cc:400] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5.
I20260709 12:17:39.710398 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:39.710141 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.711241 28288 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.711539 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:39.711680 30564 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.712162 28283 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:39.712255 30564 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.714547 30564 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:39.715124 30498 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:39.716816 30500 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:39.680816 30507 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:39.687971 30526 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:39.718303 28986 raft_consensus.cc:2410] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:39.695358 30542 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:39.721527 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:39.720715 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:39.722045 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.722697 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:39.725831 28367 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:39.726089 28363 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:39.728633 28360 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:39.718261 30507 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.731068 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.731709 28367 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:39.732227 30507 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:39.730288 30428 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.728209 30526 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:39.692880 30463 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.733294 30526 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.735505 30526 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:39.736083 30488 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:39.736483 30488 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:39.738729 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.737336 30488 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.739861 30488 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.740116 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.740909 28360 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:39.743291 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.732693 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.689815 30503 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:17:39.687192 30332 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.747925 30391 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.694639 30473 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.754112 28366 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:17:39.754158 28288 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:39.755326 28367 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:39.758415 28360 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:39.757630 28359 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:39.761240 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:39.764756 28404 connection.cc:256] RPC call timeout handler was delayed by -2.9431193349998921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 12:17:39.767364 28404 connection.cc:256] RPC call timeout handler was delayed by -2.9107193350000671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
W20260709 12:17:39.767650 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.767993 30560 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:39.768473 30560 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.765689 30526 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.769356 30526 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.770206 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1553) took 1365 ms. Trace:
I20260709 12:17:39.770604 28284 rpcz_store.cc:276] 0709 12:17:38.404467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.412973 (+  8506us) service_pool.cc:224] Handling call
0709 12:17:39.769977 (+1357004us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400164254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:39.770121 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2689142528}
I20260709 12:17:39.770964 30391 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:39.771440 30526 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.771862 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.773108 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.768821 30560 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.773694 28284 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:39.773684 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.775410 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:39.775665 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:39.767745 28404 connection.cc:256] RPC call timeout handler was delayed by -2.9098193350000656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 12:17:39.785786 28404 connection.cc:256] RPC call timeout handler was delayed by -2.7046193349999612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
I20260709 12:17:39.778122 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1557) took 1361 ms. Trace:
W20260709 12:17:39.786263 28404 connection.cc:256] RPC call timeout handler was delayed by -2.7010193349999554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
I20260709 12:17:39.779808 30332 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.782171 30503 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.783871 28332 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.784080 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.786617 28434 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:39.786991 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:39.787784 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:39.789784 28358 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:39.789808 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.790287 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:39.790843 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:39.790931 28434 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:39.786531 28291 rpcz_store.cc:276] 0709 12:17:38.416269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.425009 (+  8740us) service_pool.cc:224] Handling call
0709 12:17:39.777928 (+1352919us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54003f9654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:39.778059 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319064832}
I20260709 12:17:39.791402 28432 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.790908 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:39.792194 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.792845 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:39.793139 28291 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 4: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:39.793368 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:39.786645 28404 connection.cc:256] RPC call timeout handler was delayed by -2.6596193349998885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
W20260709 12:17:39.794092 28404 connection.cc:256] RPC call timeout handler was delayed by -2.6317193350000707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
W20260709 12:17:39.794445 28404 connection.cc:256] RPC call timeout handler was delayed by -2.6317193350000707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
W20260709 12:17:39.794724 28404 connection.cc:256] RPC call timeout handler was delayed by -2.5291193349999048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
I20260709 12:17:39.773766 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:39.795023 28404 connection.cc:256] RPC call timeout handler was delayed by -2.5291193349999048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
I20260709 12:17:39.795900 30502 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:39.796458 30502 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:39.781554 30473 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:39.797369 30502 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:39.800386 30530 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.770s	user 0.001s	sys 0.004s
I20260709 12:17:39.750182 30527 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:17:39.802462 28441 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:39.802834 30527 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:39.803246 30527 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:39.803596 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:39.804241 28441 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:39.801591 28440 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:39.795310 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.807737 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.807999 28332 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.810410 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.810850 28358 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:39.812093 28288 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:39.811476 28366 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:39.813720 28358 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:39.816859 28436 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:39.818118 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:39.817819 28432 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:39.818702 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:39.795356 28404 connection.cc:256] RPC call timeout handler was delayed by -2.4499193350000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
W20260709 12:17:39.819640 28404 connection.cc:256] RPC call timeout handler was delayed by -2.4364193349999823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
I20260709 12:17:39.819819 28432 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:39.819804 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:39.820597 28404 connection.cc:256] RPC call timeout handler was delayed by -2.4310193349999736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21930000000000002s for a response
W20260709 12:17:39.820919 28404 connection.cc:256] RPC call timeout handler was delayed by -2.206919335000066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24420000000000003s for a response
W20260709 12:17:39.821138 28404 connection.cc:256] RPC call timeout handler was delayed by -1.8460193349999372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2843s for a response
W20260709 12:17:39.821291 28404 connection.cc:256] RPC call timeout handler was delayed by -1.430219334999947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3305s for a response
I20260709 12:17:39.822050 28291 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:39.823383 28440 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:39.765985 30532 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.827206 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:39.828080 28982 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:39.828362 30532 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.814371 30478 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.808970 30503 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:39.832494 30572 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:17:39.834429 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.835314 28288 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:39.835521 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:39.834318 28985 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:39.836400 30463 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.836838 28366 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:39.818265 30473 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:39.838029 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.838248 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1448) took 1255 ms. Trace:
I20260709 12:17:39.836952 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:39.836200 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.838737 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.839021 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.832518 30503 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:39.843134 28987 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:39.849504 28284 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.839789 30559 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:39.847178 30478 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:17:39.857746 30559 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:39.858146 30559 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:39.840585 28438 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:39.808970 30526 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.859800 30526 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.861867 30555 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:39.840731 30470 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.840920 28283 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:39.804499 30391 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.842686 28433 rpcz_store.cc:276] 0709 12:17:38.582811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.582958 (+   147us) service_pool.cc:224] Handling call
0709 12:17:39.838044 (+1255086us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b54003cc654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:39.838186 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1355626880}
I20260709 12:17:39.877612 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:39.878976 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:39.880194 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:39.880815 28433 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.842531 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.850579 28440 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:39.844437 28288 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.881760 28290 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:39.883046 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.870342 28257 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:39.822729 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.839630 28441 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:39.884584 28283 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:39.854866 30529 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:39.844033 28991 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:39.841696 30473 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.843012 30432 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.845060 30414 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:17:39.848306 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:39.874562 28987 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:39.887244 28988 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:39.888757 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:39.891439 30331 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:39.876796 30470 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.896137 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:39.897055 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:39.859014 30559 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.804100 30527 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.897789 30560 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.854871 30547 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:17:39.898778 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:39.899000 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:39.900689 28284 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:39.908051 28288 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:39.907608 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:39.909689 28283 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [2.3-4.4]
I20260709 12:17:39.911722 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:39.910383 30555 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:39.947995 30555 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:39.948495 30555 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:39.910845 30197 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:39.950016 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:39.950773 28289 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:39.950065 30197 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.952102 28289 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:17:39.929145 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:39.924201 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 75980.7us
I20260709 12:17:39.921643 30472 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.927052 28441 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
I20260709 12:17:39.937914 28437 raft_consensus.cc:2368] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:17:39.960170 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 1579) took 1247 ms. Trace:
I20260709 12:17:39.913111 30526 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.943477 30559 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.927094 28290 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:39.946259 28284 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:39.950248 30560 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:39.935504 28439 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:39.932618 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:39.957535 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50035.8us
I20260709 12:17:39.953680 28435 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:39.924647 28433 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:39.955814 28283 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:39.966552 30332 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:39.960557 28437 rpcz_store.cc:276] 0709 12:17:38.712583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.746218 (+ 33635us) service_pool.cc:224] Handling call
0709 12:17:39.911526 (+1165308us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b58001d4698. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:39.960087 (+ 48561us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9358208}
W20260709 12:17:39.961915 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 139241us
I20260709 12:17:39.971736 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:39.970093 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.974074 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.970875 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.976131 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.927048 30502 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:39.903081 30473 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.977247 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:39.964332 30472 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:39.964236 30503 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:39.964272 30517 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:39.975545 30332 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:39.979785 28360 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:39.962872 30529 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:17:39.980407 28435 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:39.985142 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:39.998729 28435 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:39.986642 28290 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:17:39.987735 28437 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:39.999364 28433 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:39.976649 28332 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:39.987809 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:39.993072 28284 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:39.993662 28358 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:39.996533 28359 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:39.998476 30332 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.000037 30503 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:39.988920 30479 raft_consensus.cc:2764] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:40.006557 30472 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:39.975677 30339 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:40.012104 30339 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:40.013327 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1578) took 1301 ms. Trace:
I20260709 12:17:39.978080 30478 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:17:40.013818 28438 rpcz_store.cc:276] 0709 12:17:38.712317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.768834 (+ 56517us) service_pool.cc:224] Handling call
0709 12:17:39.840385 (+1071551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.859616 (+ 19231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.013070 (+153454us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003cc650. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:40.013245 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2463476992}
I20260709 12:17:39.978001 30564 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:17:40.015053 30538 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.006851 30503 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:39.989953 30331 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:17:40.017145 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 137535us
I20260709 12:17:40.017809 28360 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:39.989706 30514 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:17:40.019301 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:40.020198 28434 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:40.014599 30478 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:40.021479 30478 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:40.022326 28285 raft_consensus.cc:2376] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5. Re-sending same reply.
I20260709 12:17:40.022625 30478 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.001703 28433 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:39.988793 30560 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:17:40.026585 30478 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:39.998853 30522 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:40.046401 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:39.999306 30517 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.028568 28435 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.029042 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:40.056679 28435 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:39.976181 30391 consensus_queue.cc:1059] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:17:40.029419 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:40.058261 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:40.022184 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.028455 30479 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:40.031651 28255 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:40.035396 28982 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:40.035887 30197 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:40.036073 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.070188 28292 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:40.038136 28362 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 5. Ops: []
I20260709 12:17:40.039196 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:40.071146 28988 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
I20260709 12:17:40.040004 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.041919 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.073252 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.074608 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.075812 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.076395 28290 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:40.027303 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:40.078130 28433 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:40.051487 28987 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:40.052274 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 75701.8us
I20260709 12:17:40.053458 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:39.997578 30502 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:40.080709 30502 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.088429 28987 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:40.043990 30547 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:40.089623 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:40.091188 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:40.093189 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.096076 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.096719 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.097291 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:40.102218 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:40.116492 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.101073 28290 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:40.119318 28289 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:40.119097 28288 raft_consensus.cc:2376] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4. Re-sending same reply.
I20260709 12:17:40.059621 30517 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.122444 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:40.129289 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:40.058104 30529 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:40.089522 30503 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:40.108218 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.091790 30469 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:40.134984 28365 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:40.174856 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:40.175545 28365 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:40.176864 28365 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:40.135649 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:40.136448 28987 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:40.178031 28988 raft_consensus.cc:2376] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6. Re-sending same reply.
I20260709 12:17:40.138276 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:40.130553 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:40.159348 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.131264 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.180445 30498 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:40.180301 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49820.8us
I20260709 12:17:40.181535 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.182219 28287 raft_consensus.cc:2376] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5. Re-sending same reply.
I20260709 12:17:40.182512 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:40.183131 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.183887 28287 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:40.180351 28284 raft_consensus.cc:2368] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7.
I20260709 12:17:40.186233 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.187378 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.058107 30522 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.125931 30571 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:17:40.172557 30526 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:40.127055 30463 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:40.117762 30547 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:40.189428 30463 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:40.092079 30331 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:17:40.194340 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 97136.4us
I20260709 12:17:40.195262 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:40.194662 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.120347 30464 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:40.202952 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:40.173857 30469 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:40.132354 30479 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:40.092365 30542 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 12:17:40.136062 30529 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.225404 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:40.259802 28991 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:40.132515 30506 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.160679 30532 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:40.127871 30513 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:40.266144 30513 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:40.267150 30339 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:40.158985 30500 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:40.268198 30500 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.268591 30500 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:40.268988 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.262979 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:40.262143 30479 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.271809 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.101436 30472 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
W20260709 12:17:40.255970 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.274119 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.246991 30503 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:40.277907 30539 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.105s
I20260709 12:17:40.199086 30463 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.226069 30414 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:40.278469 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.279479 30551 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:17:40.279444 28285 raft_consensus.cc:2376] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5. Re-sending same reply.
I20260709 12:17:40.281180 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.277662 30469 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.282357 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:40.282723 30579 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:17:40.282907 30469 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.283489 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.185039 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.092192 30566 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.114s
I20260709 12:17:40.284663 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.190577 30477 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 12:17:40.261149 30571 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:40.248886 30428 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.303380 28285 raft_consensus.cc:2368] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4.
W20260709 12:17:40.304476 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 122017us
I20260709 12:17:40.306177 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.329725 28287 raft_consensus.cc:2376] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4. Re-sending same reply.
I20260709 12:17:40.286846 30513 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:40.294363 30479 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.337903 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.338835 28285 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:40.340950 28285 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:40.316612 30542 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:40.341905 30542 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:40.342370 30555 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.343007 30542 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:40.344545 30542 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.299223 30503 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:40.297858 30547 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.346482 30503 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:40.227066 30332 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:40.347847 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:40.321159 30500 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.349203 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:40.349481 30332 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:40.322211 28291 raft_consensus.cc:2410] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:40.322726 30522 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:40.197288 30197 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:17:40.262797 30529 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:40.296962 30532 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:40.352392 28287 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:40.353184 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:40.355966 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1486) took 1205 ms. Trace:
I20260709 12:17:40.358628 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:40.360396 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:40.359918 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:40.361886 28365 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.356326 28286 rpcz_store.cc:276] 0709 12:17:39.150537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.150672 (+   135us) service_pool.cc:224] Handling call
0709 12:17:40.355780 (+1205108us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b540009ecd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:40.355891 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399715200}
I20260709 12:17:40.324230 30560 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.367352 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:40.454857 30560 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:40.457178 30532 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.458555 30500 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:40.355822 28987 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:40.371927 28285 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:40.471843 28361 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:40.515452 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:40.519651 28361 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:40.404424 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:40.417392 28365 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:40.545019 28363 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:40.367628 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:40.367355 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:40.565987 28991 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:40.566629 30500 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:40.567934 30560 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:40.577450 30582 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:17:40.586920 30555 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:40.463794 30463 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.587702 30569 raft_consensus.cc:1064] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:17:40.588712 30470 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:40.592468 30473 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.591s	user 0.002s	sys 0.005s
I20260709 12:17:40.457535 30479 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.354806 30547 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.350953 30571 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:40.626549 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.636699 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:40.634482 30498 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:40.350696 30428 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:40.651849 30492 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:40.665323 30581 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.334s	user 0.000s	sys 0.003s
I20260709 12:17:40.601336 30517 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.677100 28432 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:40.678032 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:40.772843 30517 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:40.773136 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 95184.9us
W20260709 12:17:40.676645 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 481460us
I20260709 12:17:40.774072 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:40.738675 30197 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.775297 30197 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.776099 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:40.667861 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.777284 30197 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:40.628911 30485 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
W20260709 12:17:40.709488 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.634303 30526 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:40.717388 28256 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.008s	sys 0.060s
I20260709 12:17:40.645404 30528 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:40.778311 28256 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.008s	sys 0.060s
I20260709 12:17:40.682232 30414 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:40.724572 30483 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.700s	user 0.030s	sys 0.025s
I20260709 12:17:40.728873 30587 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:40.737726 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.788475 28285 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:40.739289 28438 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:40.740167 28435 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:40.791563 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:40.741822 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:40.740617 30492 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:40.743089 30498 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.744222 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:40.745289 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:40.759948 30479 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:40.762912 30463 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:40.763833 30571 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:40.731642 30473 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:40.718742 30507 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:17:40.780630 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.781833 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.773454 30517 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:40.716914 30464 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:40.787772 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:40.792754 28433 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:40.793179 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:40.792701 28438 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:40.778503 30528 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.510715 28436 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:41.511031 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:41.315265 30595 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:41.138604 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.511837 28439 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:41.512733 30586 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:41.512811 30564 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:40.786612 30483 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.767s	user 0.030s	sys 0.025s
I20260709 12:17:41.328020 30591 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.804798 30469 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.512959 30597 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.514986 30597 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:41.165639 30532 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.012s	sys 0.025s
W20260709 12:17:41.516189 30532 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.053s	user 0.015s	sys 0.053s
I20260709 12:17:41.513589 30586 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:40.845417 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:41.518424 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:41.499719 30529 consensus_meta.cc:287] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.100s	user 0.004s	sys 0.007s
I20260709 12:17:41.518671 28283 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:41.519359 30586 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:41.513701 30564 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:41.519578 30463 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:41.520190 30463 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:41.519145 30529 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:41.024309 28404 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.045s	sys 0.077s
I20260709 12:17:41.522320 30597 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:41.521427 30564 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:41.122467 30492 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:40.934340 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 140792us
I20260709 12:17:40.789382 30566 consensus_queue.cc:1243] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:41.285080 30487 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.525076 30492 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.524562 30464 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:41.043360 28988 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.012s	sys 0.050s
I20260709 12:17:40.778178 30526 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.525676 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:41.553259 28988 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.095s	user 0.019s	sys 0.073s
I20260709 12:17:40.856339 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:40.804731 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:41.298563 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:41.509560 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 733460us
I20260709 12:17:41.554100 28284 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
W20260709 12:17:41.509158 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 724803us
I20260709 12:17:41.525954 28286 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.524646 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:41.522377 28404 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.060s	sys 0.107s
I20260709 12:17:41.554066 28292 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:41.556435 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 1.204e+06us
W20260709 12:17:41.314481 28985 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 1.100s	user 0.006s	sys 0.006s
I20260709 12:17:41.553295 28436 raft_consensus.cc:2410] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:41.510349 28434 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.528420 28367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:40.811309 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:41.557121 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:41.557576 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.561666 28288 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.562101 28285 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:41.562469 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:41.561256 28367 raft_consensus.cc:606] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:41.563254 28367 raft_consensus.cc:614] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:40.926642 28363 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:41.529378 28441 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:41.525462 30487 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:41.529187 30503 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.071s	user 0.002s	sys 0.012s
I20260709 12:17:41.552940 28287 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.528268 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1562) took 1814 ms. Trace:
I20260709 12:17:41.069993 30485 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:41.578790 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54256.2us
I20260709 12:17:41.523689 30529 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:41.579883 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:41.627239 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 1.50839e+06us
W20260709 12:17:41.639649 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 59842.6us
I20260709 12:17:41.580909 28292 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:41.030051 30467 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.586429 28362 rpcz_store.cc:276] 0709 12:17:39.714167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.715821 (+  1654us) service_pool.cc:224] Handling call
0709 12:17:40.037932 (+322111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:40.070665 (+ 32733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.528072 (+1457407us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400102f50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:41.528203 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740022656}
I20260709 12:17:41.566315 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1589) took 1208 ms. Trace:
I20260709 12:17:40.832370 30593 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:40.933786 30478 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:40.843618 30428 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:41.565866 28286 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:40.788168 30587 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:41.615183 28291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1572) took 1848 ms (client timeout 2166 ms). Trace:
I20260709 12:17:41.551234 30527 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:41.586097 30503 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.002s	sys 0.012s
I20260709 12:17:41.595378 28288 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:41.642395 30479 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:41.612612 30483 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:41.643203 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:41.640614 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:41.643661 30488 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:41.630522 30585 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.907s	user 0.005s	sys 0.003s
I20260709 12:17:41.561028 28985 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:41.624441 30569 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:41.588197 30566 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:41.588011 30512 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:17:41.568506 30332 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:41.588865 30517 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:41.569954 30507 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 12:17:41.648417 28363 rpcz_store.cc:276] 0709 12:17:40.357282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.361299 (+  4017us) service_pool.cc:224] Handling call
0709 12:17:41.566147 (+1204848us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:41.680370 30529 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.569561 30564 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:17:40.831990 30551 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.758s
I20260709 12:17:40.935552 30590 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.829056 30560 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.604413 30463 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:41.640604 28332 leader_election.cc:400] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:17:41.514606 30591 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:41.671615 28291 rpcz_store.cc:269] 0709 12:17:39.766624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.822703 (+ 56079us) service_pool.cc:224] Handling call
0709 12:17:41.614934 (+1792231us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54000f23d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:41.615109 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087277440}
I20260709 12:17:41.698012 30566 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:41.673089 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:41.652571 30467 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.679363 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:41.723237 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43955.8us
I20260709 12:17:41.580320 30532 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.727555 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:41.727663 30532 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:41.607643 30487 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:41.728446 28363 raft_consensus.cc:606] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:41.511806 30595 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.674475 30479 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.701201 30517 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:41.483667 30547 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.747s	user 0.001s	sys 0.007s
I20260709 12:17:41.703804 28287 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
W20260709 12:17:41.710454 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.713550 30590 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.655897 30506 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:17:41.657725 30586 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:41.686569 30428 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:41.703653 30419 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.029s	sys 0.041s
I20260709 12:17:41.715163 28332 leader_election.cc:403] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:17:41.778898 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 138475us
I20260709 12:17:41.057175 30414 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:41.717664 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44640.7us
I20260709 12:17:41.731616 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:41.783543 28437 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:41.656246 30485 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:41.698439 30332 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:41.728754 28392 heartbeater.cc:646] Failed to heartbeat to 127.26.123.62:36149 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.26.123.62:36149 timed out after 15.000s (SENT)
I20260709 12:17:41.736692 30479 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.716291 30560 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.690609 30488 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:41.755717 30197 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.806715 30197 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.809017 30197 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:41.756620 30595 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:41.758301 30532 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:41.657841 30478 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.811340 30485 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:41.812132 30478 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:41.718739 30566 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:41.730686 30339 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.812675 30591 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:41.813059 30339 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:41.813558 30339 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.814543 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4036b67205e41c597049a10a71e9467"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:41.814940 30432 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:41.730135 30463 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:41.815522 30432 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:41.816176 30603 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:17:41.769347 28360 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:41.737361 30517 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:41.776675 28186 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 222) took 14930 ms (client timeout 15000 ms). Trace:
W20260709 12:17:41.785177 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.789079 28438 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:41.790444 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:41.791566 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:41.792191 28432 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:17:41.794327 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 62829.9us
I20260709 12:17:41.819034 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:41.818588 28439 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:41.820578 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:41.822456 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:41.717418 30529 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:41.778007 30419 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.029s	sys 0.041s
I20260709 12:17:41.686520 30483 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:41.823763 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:41.826464 28362 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:41.827010 28439 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:41.828519 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:41.799105 30547 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:17:41.832123 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:41.835106 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:41.836824 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:41.801589 30597 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:41.842654 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:41.782778 30586 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.844877 28432 raft_consensus.cc:2410] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:17:41.804086 30560 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.845608 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:41.809010 30488 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:41.848289 28363 raft_consensus.cc:606] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:41.820554 30532 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:41.850968 28363 raft_consensus.cc:614] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:41.826153 30593 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.830777 30463 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.802973 30551 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:17:41.833102 30503 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.826303 30595 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:41.852588 30599 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:17:41.842598 30528 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:41.842907 30428 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.888347 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:41.888214 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 333448us
W20260709 12:17:41.818117 28186 rpcz_store.cc:269] 0709 12:17:26.846253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.946450 (+100197us) service_pool.cc:224] Handling call
0709 12:17:41.776578 (+14830128us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:39.066750 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:39.087049 (+ 20299us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:39.087105 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:17:39.087123 (+    18us) tablet.cc:662] Decoding operations
0709 12:17:39.106606 (+ 19483us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:39.152147 (+ 45541us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:39.155737 (+  3590us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:17:39.334133 (+178396us) tablet.cc:701] Row locks acquired
0709 12:17:39.370593 (+ 36460us) write_op.cc:260] PREPARE: finished
0709 12:17:39.382625 (+ 12032us) write_op.cc:270] Start()
0709 12:17:39.410572 (+ 27947us) write_op.cc:276] Timestamp: P: 1783599459379061 usec, L: 0
0709 12:17:39.411520 (+   948us) op_driver.cc:348] REPLICATION: starting
0709 12:17:39.601139 (+189619us) log.cc:844] Serialized 2074 byte log entry
0709 12:17:39.954438 (+353299us) op_driver.cc:464] REPLICATION: finished
0709 12:17:40.206115 (+251677us) write_op.cc:301] APPLY: starting
0709 12:17:40.225912 (+ 19797us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:40.286823 (+ 60911us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:40.300914 (+ 14091us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:40.570070 (+269156us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:40.570289 (+   219us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:17:40.570331 (+    42us) write_op.cc:312] APPLY: finished
0709 12:17:40.570586 (+   255us) log.cc:844] Serialized 47 byte log entry
0709 12:17:40.570774 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:40.570894 (+   120us) write_op.cc:454] Released schema lock
0709 12:17:40.570938 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:40.571307 (+   369us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:40.571335 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":915025,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":205363,"apply.run_cpu_time_us":4011,"apply.run_wall_time_us":379986,"num_ops":5,"prepare.queue_time_us":233639,"prepare.run_cpu_time_us":36839,"prepare.run_wall_time_us":704996,"raft.queue_time_us":151501,"raft.run_cpu_time_us":28160,"raft.run_wall_time_us":90284,"replication_time_us":542761,"thread_start_us":424145,"threads_started":4,"wal-append.queue_time_us":180923,"wal-append.run_cpu_time_us":24472,"wal-append.run_wall_time_us":1805691}]]}
I20260709 12:17:41.855887 30597 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:41.932695 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:41.894963 30526 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:41.904770 30532 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:41.904256 30463 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:41.906371 30509 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:41.805967 30587 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:41.969684 28360 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1585) took 1623 ms (client timeout 1882 ms). Trace:
I20260709 12:17:41.938830 30551 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:41.895658 30595 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.939047 30487 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.857429 30593 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:41.933544 30597 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:41.979816 30600 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
W20260709 12:17:41.976744 28360 rpcz_store.cc:269] 0709 12:17:40.346252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.350585 (+  4333us) service_pool.cc:224] Handling call
0709 12:17:41.969472 (+1618887us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400103454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:41.969609 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2969855872}
W20260709 12:17:42.027976 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 119929us
I20260709 12:17:42.034037 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.030138 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:17:42.036195 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:42.038604 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 201842us
I20260709 12:17:42.037439 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:42.034791 28439 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:41.980873 30551 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:41.956106 30473 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:42.016160 28434 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
W20260709 12:17:42.046818 28434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1476) took 1304 ms (client timeout 1536 ms). Trace:
W20260709 12:17:41.981930 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 154071us
I20260709 12:17:42.051957 30611 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.052456 30507 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:42.052615 30611 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.053798 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.053939 30507 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:42.055267 30611 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.055433 30507 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:42.055862 30611 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.056087 30507 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:42.056859 30611 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.057575 30611 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.057427 30507 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.059090 30507 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.994570 30467 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:42.047258 28434 rpcz_store.cc:269] 0709 12:17:40.743728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.743997 (+   269us) service_pool.cc:224] Handling call
0709 12:17:42.046747 (+1302750us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:42.022435 30547 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:41.982530 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 423327us
I20260709 12:17:42.042517 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:42.011286 30587 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.062364 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:42.068086 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:42.044502 28988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 337) took 1687 ms. Trace:
I20260709 12:17:42.026499 30597 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:42.100831 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.101215 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:42.069492 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.019008 30463 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:42.097415 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:42.024623 28985 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 259) took 2117 ms (client timeout 1661 ms). Trace:
I20260709 12:17:41.978924 30532 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.101583 28434 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:42.040480 30503 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:42.104372 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.102660 28440 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:42.108769 28988 rpcz_store.cc:276] 0709 12:17:40.351400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.352256 (+   856us) service_pool.cc:224] Handling call
0709 12:17:40.352605 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:40.402426 (+ 49821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:40.402460 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:40.458088 (+ 55628us) log.cc:844] Serialized 26 byte log entry
0709 12:17:41.553779 (+1095691us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:41.553818 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.553881 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:42.038140 (+484259us) raft_consensus.cc:1685] finished
0709 12:17:42.038161 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:42.038286 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24962,"raft.queue_time_us":34704,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":53681,"thread_start_us":1095281,"threads_started":1,"wal-append.queue_time_us":1189077}
I20260709 12:17:42.066188 28982 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:17:42.114043 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51806.2us
I20260709 12:17:42.112504 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.107993 28257 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.115787 28440 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
W20260709 12:17:42.114423 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 74191us
I20260709 12:17:42.018059 30487 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.118101 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:42.118422 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.044956 30478 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:42.045255 30492 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:42.103694 30498 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:42.065979 30419 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:17:41.983491 30509 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.110522 30488 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.104019 30590 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.112226 30616 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:42.115584 30586 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:42.103344 30467 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:17:42.096061 30547 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.124080 30529 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.124961 30478 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:42.152305 30332 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.153409 30504 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:42.167266 30504 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:42.167841 30593 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:42.154021 30602 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:17:42.146884 30616 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.169889 30339 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.156128 30618 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:42.155427 30487 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.172772 30339 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.118674 30503 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.174088 30547 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.142860 30551 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.175556 30503 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:42.175761 30551 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.176716 30479 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.141602 30498 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.179082 30432 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:42.131646 30610 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:42.186066 30610 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.186589 30610 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:42.188611 30608 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:17:42.142908 30488 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.123803 30605 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:42.190443 30605 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:42.191865 30488 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.191632 30605 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.139654 30483 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:42.179776 30498 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:42.180840 30517 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.165948 30478 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.179186 30479 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:42.170934 30509 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.172554 30616 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:42.139000 30615 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.169350 30512 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:42.149859 30614 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.059s
W20260709 12:17:42.210279 30613 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:17:42.213406 30517 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:42.201200 30571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.007s	sys 0.000s
I20260709 12:17:42.222329 30595 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:42.229094 30571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.008s	sys 0.000s
I20260709 12:17:42.275241 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 217) took 1492 ms. Trace:
W20260709 12:17:42.276547 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 158383us
W20260709 12:17:42.287755 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.281252 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:42.293337 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 231816us
I20260709 12:17:42.276336 28283 rpcz_store.cc:276] 0709 12:17:40.782622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.519599 (+736977us) service_pool.cc:224] Handling call
0709 12:17:41.519977 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:41.520351 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:41.520381 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:41.520902 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:17:41.993687 (+472785us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:41.993757 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.993838 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:42.274163 (+280325us) raft_consensus.cc:1685] finished
0709 12:17:42.274265 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:42.274791 (+   526us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3031,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":472355,"threads_started":1,"wal-append.queue_time_us":597688}
W20260709 12:17:42.308145 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 200075us
I20260709 12:17:42.296248 28360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:42.327806 28360 raft_consensus.cc:606] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:42.337057 28360 raft_consensus.cc:614] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:42.343307 30488 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.344606 30488 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:42.104326 28985 rpcz_store.cc:269] 0709 12:17:39.878094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.883107 (+  5013us) service_pool.cc:224] Handling call
0709 12:17:41.977781 (+2094674us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54004c2554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:41.995162 (+ 17381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506661760}
I20260709 12:17:42.168316 30428 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.182s
W20260709 12:17:42.350195 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 229389us
I20260709 12:17:42.351334 28437 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:17:42.357575 30331 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:42.358237 30331 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:42.358907 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.360086 30331 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:42.362528 30626 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:17:42.170389 30504 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.370518 30463 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.374243 30622 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.375456 30571 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:42.375310 30512 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:42.377820 30512 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.378831 30532 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:42.383359 30488 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:42.366268 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 85301.1us
W20260709 12:17:42.390934 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 281743us
I20260709 12:17:42.394325 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 260) took 2514 ms. Trace:
I20260709 12:17:42.395006 28986 rpcz_store.cc:276] 0709 12:17:39.879163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.883660 (+  4497us) service_pool.cc:224] Handling call
0709 12:17:39.894308 (+ 10648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.814603 (+1920295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:41.814634 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:41.814679 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:41.814721 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.393496 (+578775us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b540048fb54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:42.393598 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:42.393987 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5138,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":8605,"spinlock_wait_cycles":4415425920}
I20260709 12:17:42.397171 28982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 261) took 2267 ms. Trace:
I20260709 12:17:42.397990 28982 rpcz_store.cc:276] 0709 12:17:40.129920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.172318 (+ 42398us) service_pool.cc:224] Handling call
0709 12:17:41.982986 (+1810668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:42.118508 (+135522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.396614 (+278106us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54004c2550. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:42.396929 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4122148096}
I20260709 12:17:42.403443 28360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:42.404465 28360 raft_consensus.cc:606] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:42.405712 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.406694 28363 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:17:42.408536 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.409575 28367 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:42.410683 30485 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:42.414969 30566 consensus_meta.cc:287] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.589s	user 0.008s	sys 0.004s
I20260709 12:17:42.415541 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:42.415731 30566 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.417781 28433 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.419494 30566 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:42.202912 30560 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:42.422488 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.423703 30560 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.391981 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:42.448645 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:17:42.462311 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.463608 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:42.466552 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 69484.2us
I20260709 12:17:42.455127 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:42.473160 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.474938 28257 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.487102 30615 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:17:42.488171 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.489033 30414 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.492604 30504 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.493374 30332 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.415222 30485 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.487430 30632 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:42.499953 30632 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.500349 30632 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:42.502640 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 110961us
I20260709 12:17:42.503139 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.503749 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:42.504279 30532 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.504730 30532 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.505096 30532 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.506734 28433 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:42.504413 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:42.519330 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44630.3us
W20260709 12:17:42.522061 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.523622 30473 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:42.530118 30627 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.503072 30566 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:42.530845 30627 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:42.535009 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.539077 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.539834 30478 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.540668 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.532645 30627 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.532846 30414 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.546501 30414 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.549161 30414 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:42.551687 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 478635us
I20260709 12:17:42.551326 30611 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.553875 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:42.554186 30611 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:42.555589 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 421933us
I20260709 12:17:42.555893 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:42.558979 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.559854 28439 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:42.551610 30478 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:42.563839 30485 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.566249 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.533131 30566 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:42.568632 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.523453 30597 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.602535 30597 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:42.603821 30571 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.606652 30488 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.607439 30488 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.610044 30571 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.573355 30483 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:42.612196 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 43734.7us
W20260709 12:17:42.572604 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 68968.4us
I20260709 12:17:42.612404 30512 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.613209 30512 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.613649 30512 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.614364 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:42.615490 30478 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.617213 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.578300 30642 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:42.618120 30642 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:42.580353 30473 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:42.618944 30473 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:42.619511 30473 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.583395 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.622399 28367 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:17:42.623493 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.604204 30331 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:42.585342 30507 consensus_meta.cc:287] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.005s
I20260709 12:17:42.586277 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.592693 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:42.589476 28187 catalog_manager.cc:5697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 2 to 4, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 4 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:42.595333 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:42.596479 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.597417 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:42.598506 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:42.599869 30566 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.571698 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.603300 28441 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:42.610476 30488 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:42.579325 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 58006.4us
I20260709 12:17:42.582115 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:42.624643 28440 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 4: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:42.623749 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.618647 30642 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.632606 28433 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:42.636319 28359 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.638799 28436 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:42.636514 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 70446.4us
I20260709 12:17:42.612475 30571 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.641206 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.646322 28332 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:42.648027 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:42.646829 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:42.649049 28291 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.650144 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.651418 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:42.649319 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:42.656347 30428 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:42.660926 28330 leader_election.cc:400] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7.
I20260709 12:17:42.647552 30332 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.657096 30428 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:42.672446 28434 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:42.671897 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:42.658597 30532 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.671291 30488 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:17:42.676790 28187 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 3 to 4, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 4 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:42.703275 28187 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 4, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 4 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:42.706312 30428 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.707544 30513 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.677187 30332 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.714354 28440 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.715607 30332 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:42.716958 30605 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:42.717674 30605 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:42.718796 30605 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.680604 30571 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:42.730758 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44586.8us
I20260709 12:17:42.729698 28437 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:42.730737 28440 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:42.722877 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:42.693177 30610 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.736954 28439 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:42.704914 28187 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:42.741006 28291 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:42.741041 28359 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
W20260709 12:17:42.746304 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 112721us
I20260709 12:17:42.746784 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.747448 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:42.747673 28366 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:42.748533 28364 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:42.748669 30478 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.749145 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:42.750324 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.751132 28361 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:42.749043 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.754096 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:42.755151 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:42.756019 30488 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:42.756482 30488 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.756889 30488 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:42.764379 30627 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.765802 30488 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.770485 28332 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:42.766537 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 80435.3us
I20260709 12:17:42.767102 30597 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:42.772665 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.733074 30571 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:42.777577 28436 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:42.680620 30624 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:42.756557 30485 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:42.694368 30632 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.786291 28187 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 4, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 4 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:42.791496 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:42.779296 30571 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.802621 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.803329 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.805178 30560 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:42.805827 30560 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:42.806879 30560 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:42.807478 30512 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.787613 30414 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:42.803642 28187 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:42.826022 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:42.830823 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44145us
I20260709 12:17:42.804136 30624 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:42.816846 28982 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:42.847007 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.831065 30512 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:42.852396 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 103341us
I20260709 12:17:42.857954 28986 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:42.859355 30332 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:42.860105 30332 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:42.836798 30595 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:42.866011 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:42.868211 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:42.872825 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.860026 28187 catalog_manager.cc:5697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:42.877233 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:42.830349 30473 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.915405 28359 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:42.915616 28434 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.910658 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:42.915705 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 62067us
I20260709 12:17:42.920192 30622 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.924513 30622 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.924947 30622 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.924117 28187 catalog_manager.cc:5697] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 3 to 4, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 4 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:42.930496 30624 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.931627 30473 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:42.935115 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:42.937796 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:42.938789 28435 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:42.939036 28436 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:42.939648 30614 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.940356 30614 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.941172 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:42.942162 28361 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:42.942624 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:42.861305 30332 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.937687 28360 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:42.945818 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.943238 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:42.905404 30595 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:42.909806 30532 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:42.873637 30414 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:17:42.967440 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:42.968380 28361 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.953905 28991 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 2.
I20260709 12:17:42.984579 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:42.983354 30473 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.946238 30566 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:42.999459 28359 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:17:43.002975 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 66929.5us
I20260709 12:17:42.991305 30610 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:43.019750 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:43.024870 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 75481.6us
W20260709 12:17:43.036545 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.040206 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:43.041355 28287 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:43.041805 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:43.020298 28187 catalog_manager.cc:5697] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 4, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 4 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:43.002786 30627 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:43.040726 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:43.045349 28285 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:17:43.052297 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.053766 30506 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
W20260709 12:17:43.055462 30648 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:17:43.057363 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:43.045764 30615 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:43.089952 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:43.090173 30473 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:43.090888 28287 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.030730 30414 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:17:43.181824 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 244048us
I20260709 12:17:43.190904 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:43.191745 28438 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.192574 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:43.181602 28187 catalog_manager.cc:5697] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
W20260709 12:17:43.205067 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 96699.1us
I20260709 12:17:43.210014 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:43.193148 30615 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:43.224983 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.111871 30595 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:43.231797 28361 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
W20260709 12:17:43.224864 30642 consensus_meta.cc:287] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.000s
W20260709 12:17:43.287510 30649 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 12:17:43.289979 30509 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.328657 28291 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:43.299618 28287 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:43.286019 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:43.279572 28187 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:43.333519 28434 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:43.335845 28436 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:43.338068 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:43.287259 28255 connection.cc:256] RPC call timeout handler was delayed by -1.2066040669999438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
I20260709 12:17:43.294111 30671 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:43.248819 30566 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:43.349426 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:43.351588 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:43.279274 30512 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.289927 30637 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.248371 30611 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.353423 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:43.353880 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:43.357983 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 137227us
W20260709 12:17:43.360240 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.393630 30633 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:17:43.394052 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:43.363818 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:43.394692 30564 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:43.400379 30622 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.317703 30615 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:17:43.403498 30653 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:17:43.705504 28434 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:43.353814 28289 raft_consensus.cc:1813] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:43.706943 28289 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 3 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:17:43.446933 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:43.447788 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:43.448571 30517 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.738131 30517 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.740759 30517 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:43.345993 30671 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:43.742528 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:43.742278 30671 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.743408 30671 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:43.744966 30571 consensus_meta.cc:287] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.927s	user 0.012s	sys 0.000s
I20260709 12:17:43.357872 30509 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.745446 30571 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.747176 28363 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:43.747542 30509 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:43.748175 30509 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.748669 30509 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.451442 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:43.453186 30637 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.364002 28361 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:43.329234 30642 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:43.749958 30571 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:43.751791 30642 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:43.507145 28438 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:17:43.753031 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:43.693118 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 289893us
I20260709 12:17:43.763610 30637 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:43.766616 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 183381us
I20260709 12:17:43.768316 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:43.449468 30551 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.770390 30551 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.772744 30551 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:43.774050 28287 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
W20260709 12:17:43.777312 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 327113us
I20260709 12:17:43.778368 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:43.779848 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:43.786392 28187 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 3 to 5, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 5 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:43.788218 28187 catalog_manager.cc:5697] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 1 to 3, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:43.790004 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:43.791689 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:43.792301 28362 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:43.793200 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:43.795740 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1450) took 1050 ms. Trace:
I20260709 12:17:43.796195 28358 rpcz_store.cc:276] 0709 12:17:42.745708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.745952 (+   244us) service_pool.cc:224] Handling call
0709 12:17:42.746387 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:42.746987 (+   600us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:17:42.747021 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:42.747694 (+   673us) log.cc:844] Serialized 26 byte log entry
0709 12:17:42.985233 (+237539us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:42.985280 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.985361 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:43.795369 (+810008us) raft_consensus.cc:1685] finished
0709 12:17:43.795390 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:43.795535 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1473,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"thread_start_us":237062,"threads_started":1,"wal-append.queue_time_us":363692}
I20260709 12:17:43.454538 30512 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.733407 28286 raft_consensus.cc:2368] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6.
I20260709 12:17:43.733163 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:43.799038 28290 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:43.799860 30512 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:43.800469 30512 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:43.801229 28437 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.699059 30560 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.700038 30627 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.757110 30564 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:43.858103 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.865547 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:43.884742 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:43.859369 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:43.910817 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:43.703259 30611 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:43.924422 30622 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:43.867024 30595 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.864420 30627 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:43.934031 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:44.147590 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:43.979503 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.009599 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1495) took 1683 ms. Trace:
I20260709 12:17:44.068189 28291 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:43.961968 28358 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:43.963618 28187 catalog_manager.cc:5697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:44.149272 28440 rpcz_store.cc:276] 0709 12:17:42.329510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.637864 (+308354us) service_pool.cc:224] Handling call
0709 12:17:44.009322 (+1371458us) spinlock_profiling.cc:243] Waited 65.4 ms on lock 0x7b5400416254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:44.009497 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150531456}
I20260709 12:17:44.085465 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 241) took 2061 ms. Trace:
W20260709 12:17:44.149869 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 381656us
W20260709 12:17:44.088599 30654 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.945s	user 0.008s	sys 0.000s
I20260709 12:17:44.128262 30478 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:44.147055 28986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.134s	sys 0.306s
I20260709 12:17:44.164731 30478 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:44.164848 28986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.134s	sys 0.306s
I20260709 12:17:44.090267 28982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 266) took 1365 ms. Trace:
I20260709 12:17:44.074836 30571 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.167768 28187 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:44.168361 30571 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.169480 28187 catalog_manager.cc:5697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 3 to 4, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 4 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:44.171144 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:44.170892 28187 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:44.173053 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:44.173382 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:44.174161 28984 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:44.174475 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:44.174976 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:44.175609 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:44.176167 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:43.973313 30678 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.178651 28990 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.944057 30414 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:44.145638 30509 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.126396 30611 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.174301 28983 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:43.943854 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 268) took 1068 ms. Trace:
I20260709 12:17:44.175639 28989 raft_consensus.cc:1813] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:44.195494 28989 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 3 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:17:44.197351 30675 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:44.198107 30675 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:44.199102 30675 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.202771 28990 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:44.157843 28432 rpcz_store.cc:276] 0709 12:17:42.024207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.024366 (+   159us) service_pool.cc:224] Handling call
0709 12:17:42.040844 (+ 16478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:42.123928 (+ 83084us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:42.480304 (+356376us) log.cc:844] Serialized 14 byte log entry
0709 12:17:42.686472 (+206168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:42.694755 (+  8283us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:42.725777 (+ 31022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.084936 (+1359159us) spinlock_profiling.cc:243] Waited 82.8 ms on lock 0x7b54003cbc54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:44.085005 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.085239 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5935,"prepare.queue_time_us":190020,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":73,"raft.queue_time_us":17664,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":29258,"spinlock_wait_cycles":190544128,"thread_start_us":220451,"threads_started":2,"wal-append.queue_time_us":254955,"wal-append.run_cpu_time_us":2710,"wal-append.run_wall_time_us":1268408}
I20260709 12:17:44.167829 28982 rpcz_store.cc:276] 0709 12:17:42.743421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.744952 (+  1531us) service_pool.cc:224] Handling call
0709 12:17:42.745509 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:42.848715 (+103206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.089919 (+1241204us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b54006e2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:44.090182 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97300224}
I20260709 12:17:44.201820 28985 rpcz_store.cc:276] 0709 12:17:42.786962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.795532 (+  8570us) service_pool.cc:224] Handling call
0709 12:17:42.873463 (+ 77931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.088369 (+214906us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:43.187561 (+ 99192us) log.cc:844] Serialized 14 byte log entry
0709 12:17:43.337771 (+150210us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:43.764412 (+426641us) log.cc:844] Serialized 26 byte log entry
0709 12:17:43.764636 (+   224us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:43.764677 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.764764 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:43.812443 (+ 47679us) raft_consensus.cc:1685] finished
0709 12:17:43.819337 (+  6894us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:43.846289 (+ 26952us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13642,"prepare.queue_time_us":91318,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":120946,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":17093,"spinlock_wait_cycles":6068480,"thread_start_us":277371,"threads_started":2,"wal-append.queue_time_us":260215}
W20260709 12:17:44.187841 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 406131us
I20260709 12:17:44.166312 30622 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.214932 30622 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.212026 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1460) took 1242 ms. Trace:
I20260709 12:17:43.909981 30560 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.212800 28361 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:44.115861 30637 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.213703 28440 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:44.212517 28257 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.015s	sys 0.024s
W20260709 12:17:44.233348 28257 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.015s	sys 0.024s
I20260709 12:17:44.234503 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.235675 28330 leader_election.cc:400] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4.
I20260709 12:17:44.216039 28364 rpcz_store.cc:276] 0709 12:17:42.969254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.098026 (+128772us) service_pool.cc:224] Handling call
0709 12:17:43.150265 (+ 52239us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:43.248097 (+ 97832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:43.248151 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:43.322449 (+ 74298us) log.cc:844] Serialized 52 byte log entry
0709 12:17:43.352315 (+ 29866us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:43.863442 (+511127us) log.cc:844] Serialized 14 byte log entry
0709 12:17:44.102187 (+238745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.102280 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.211769 (+109489us) raft_consensus.cc:1685] finished
0709 12:17:44.211788 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.211949 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":238415,"prepare.queue_time_us":430565,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":349319,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":55852,"thread_start_us":258322,"threads_started":3,"wal-append.queue_time_us":854738}
I20260709 12:17:44.242080 28440 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: [2.5-6.7]
W20260709 12:17:44.245007 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.237880 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 270) took 1357 ms. Trace:
I20260709 12:17:44.245931 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.248135 28364 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.249337 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.250017 28289 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:44.250615 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1451) took 1496 ms. Trace:
I20260709 12:17:44.251053 28365 rpcz_store.cc:276] 0709 12:17:42.746162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.749309 (+  3147us) service_pool.cc:224] Handling call
0709 12:17:42.749685 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:42.750240 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:42.750277 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:42.750814 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:17:42.951938 (+201124us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:42.951976 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.952038 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.087701 (+1135663us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:44.240497 (+152796us) raft_consensus.cc:1685] finished
0709 12:17:44.240521 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.242740 (+  2219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131,"spinlock_wait_cycles":36863872,"thread_start_us":200664,"threads_started":1,"wal-append.queue_time_us":330056}
I20260709 12:17:44.252104 28330 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:44.252991 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.252127 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 233) took 1313 ms. Trace:
W20260709 12:17:44.256129 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.256313 30622 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.257213 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:44.255589 28283 rpcz_store.cc:276] 0709 12:17:42.938659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.949350 (+ 10691us) service_pool.cc:224] Handling call
0709 12:17:44.251303 (+1301953us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b5800000098. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:44.251554 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:44.251767 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:44.251793 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:44.251850 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:44.251881 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.251943 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.252044 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5037824}
I20260709 12:17:44.189929 30678 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.259130 28290 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:44.256908 28289 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:44.259191 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.240280 30637 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.261185 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.263248 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.263337 28284 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:44.263890 28364 raft_consensus.cc:2410] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:44.264729 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:44.265473 28287 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:44.266443 28358 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:44.267041 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.268540 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.269668 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.271556 28289 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:44.272197 30656 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.232s	user 0.006s	sys 0.003s
I20260709 12:17:44.256588 30564 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:44.272821 30614 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.269934 30488 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.278867 28284 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:44.279950 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.281509 30627 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.273464 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.288746 28361 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:44.284691 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 101259us
I20260709 12:17:44.260782 30483 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:44.292354 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.294536 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.297721 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1598) took 1103 ms. Trace:
I20260709 12:17:44.298125 28365 rpcz_store.cc:276] 0709 12:17:43.197190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.295558 (+1098368us) service_pool.cc:224] Handling call
0709 12:17:44.295895 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:44.297330 (+  1435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:44.297364 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:44.297410 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:44.297444 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.297508 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.297635 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.300101 28290 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:44.301939 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:44.302597 30671 consensus_meta.cc:287] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.559s	user 0.013s	sys 0.000s
I20260709 12:17:44.303520 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:44.301909 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.304941 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1600) took 1096 ms. Trace:
I20260709 12:17:44.305239 28364 rpcz_store.cc:276] 0709 12:17:43.208624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.290885 (+1082261us) service_pool.cc:224] Handling call
0709 12:17:44.304874 (+ 13989us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:44.306470 28358 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:44.307210 30488 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:44.307643 30488 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:44.307719 30627 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:44.308588 30627 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:44.308964 30627 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:44.309873 30627 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.311357 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:44.307025 30564 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:44.308040 30488 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:44.312803 28361 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:44.293866 30560 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:44.324661 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:44.312155 30627 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.291679 30622 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
W20260709 12:17:44.328253 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.319960 30647 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.652s	user 0.001s	sys 0.011s
I20260709 12:17:44.333681 28988 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:44.334388 28367 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:44.316542 30610 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:44.336256 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:44.337428 28440 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:44.342756 28361 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:44.344960 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:44.330631 30566 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:44.333798 30642 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.350299 30688 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:44.357787 30688 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.358237 30688 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.338058 30610 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:44.370993 28367 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:44.374564 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:44.370105 30610 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.380488 30590 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:44.381026 30590 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.381460 30590 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:44.386301 28988 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:17:44.365541 30587 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:44.390641 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 52679.3us
I20260709 12:17:44.376925 30532 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:44.366451 30614 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:17:44.377488 30428 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:44.405560 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1631) took 1110 ms. Trace:
I20260709 12:17:44.366118 30642 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.247038 28986 rpcz_store.cc:276] 0709 12:17:42.879376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.905145 (+ 25769us) service_pool.cc:224] Handling call
0709 12:17:42.923360 (+ 18215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.145074 (+221714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:43.156179 (+ 11105us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:43.192502 (+ 36323us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.165474 (+972972us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:44.165534 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.166925 (+  1391us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.214373 (+ 47448us) raft_consensus.cc:1685] finished
0709 12:17:44.214395 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.235919 (+ 21524us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79299,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5490,"thread_start_us":1076601,"threads_started":2,"wal-append.queue_time_us":1017623}
I20260709 12:17:44.406093 28432 rpcz_store.cc:276] 0709 12:17:43.295612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.214892 (+919280us) service_pool.cc:224] Handling call
0709 12:17:44.215258 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:44.215542 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:44.289478 (+ 73936us) log.cc:844] Serialized 14 byte log entry
0709 12:17:44.403342 (+113864us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:44.403396 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:44.405284 (+  1888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.405370 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.405498 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15811,"prepare.queue_time_us":73911,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":37901,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":4214,"thread_start_us":171023,"threads_started":3,"wal-append.queue_time_us":85155}
I20260709 12:17:44.374161 30566 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.415970 30532 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.417387 30566 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:44.412570 30587 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.420014 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:44.420639 28988 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:44.421252 28440 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.418948 30532 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:44.411674 28284 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:44.405989 30463 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:44.430059 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.408753 30678 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.411850 30331 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:44.440994 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:44.448292 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1139882819998093s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
I20260709 12:17:44.419414 30428 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:44.448688 28404 connection.cc:256] RPC call timeout handler was delayed by -1.0590882819999479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
I20260709 12:17:44.450623 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:44.453755 30642 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:44.450047 28954 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 1.713s (SENT) [suppressed 37 similar messages]
I20260709 12:17:44.464591 28988 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:44.437618 30637 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.441078 30331 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:44.480370 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1601) took 1246 ms. Trace:
I20260709 12:17:44.482019 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.484974 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.481493 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.491537 28440 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:44.492799 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.493168 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.493999 28283 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:44.496157 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:44.479943 30694 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:17:44.453866 30697 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:44.483754 28358 rpcz_store.cc:276] 0709 12:17:43.234989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.310258 (+1075269us) service_pool.cc:224] Handling call
0709 12:17:44.310631 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:44.311009 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:44.311369 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 12:17:44.384951 (+ 73582us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b18001bbe00. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:44.415363 (+ 30412us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b18001bbe00. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be42723a 00007f85be891a2a
0709 12:17:44.480002 (+ 64639us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:44.480062 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:44.480118 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.480185 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.480304 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":963,"prepare.run_wall_time_us":49,"raft.queue_time_us":1352,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":4109,"spinlock_wait_cycles":10984064,"thread_start_us":45334,"threads_started":1,"wal-append.queue_time_us":156215}
I20260709 12:17:44.499946 28988 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:44.440174 30463 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.503259 28432 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:44.507704 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 254) took 1735 ms. Trace:
I20260709 12:17:44.508224 28437 rpcz_store.cc:276] 0709 12:17:42.772448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.787183 (+ 14735us) service_pool.cc:224] Handling call
0709 12:17:42.815179 (+ 27996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.801662 (+986483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.507395 (+705733us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b5400347454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:44.507617 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2194967296}
W20260709 12:17:44.513136 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 63156.8us
I20260709 12:17:44.514984 28285 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:44.502921 28288 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:44.515987 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:44.513994 28954 leader_election.cc:336] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 1.871s (SENT)
I20260709 12:17:44.518615 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:44.518679 28289 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:44.519771 28990 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:44.512239 30697 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.524235 28285 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:44.524160 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.523090 30488 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.531080 30488 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:44.533819 28283 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:44.534765 28432 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:44.536983 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1496) took 1915 ms. Trace:
I20260709 12:17:44.536324 28289 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:44.537446 28439 rpcz_store.cc:276] 0709 12:17:42.622096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.638498 (+ 16402us) service_pool.cc:224] Handling call
0709 12:17:44.536726 (+1898228us) spinlock_profiling.cc:243] Waited 87.8 ms on lock 0x7b540026a2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:44.536906 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201929472}
I20260709 12:17:44.514524 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:44.518558 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:44.553584 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 227) took 1151 ms. Trace:
I20260709 12:17:44.549460 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1446) took 1886 ms. Trace:
I20260709 12:17:44.554086 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.521536 30622 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:44.547228 30615 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:17:44.554126 28364 rpcz_store.cc:276] 0709 12:17:43.402488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.333418 (+930930us) service_pool.cc:224] Handling call
0709 12:17:44.344108 (+ 10690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.361162 (+ 17054us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:44.398791 (+ 37629us) log.cc:844] Serialized 14 byte log entry
0709 12:17:44.446093 (+ 47302us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:44.446612 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.446728 (+   116us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:44.446970 (+   242us) log.cc:844] Serialized 14 byte log entry
0709 12:17:44.447166 (+   196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.447247 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.553280 (+106033us) raft_consensus.cc:1685] finished
0709 12:17:44.553306 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.553495 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24199,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":1625,"raft.queue_time_us":12726,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":4663,"spinlock_wait_cycles":46464,"thread_start_us":19237,"threads_started":1,"wal-append.queue_time_us":36176}
I20260709 12:17:44.554561 28363 rpcz_store.cc:276] 0709 12:17:42.663395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.663611 (+   216us) service_pool.cc:224] Handling call
0709 12:17:42.692540 (+ 28929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.747560 (+1055020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.549154 (+801594us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003bb0d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:44.549382 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2424973696}
I20260709 12:17:44.526873 30560 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:44.553287 30697 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:44.565101 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 51127.8us
I20260709 12:17:44.571623 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.571630 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1625) took 1343 ms. Trace:
I20260709 12:17:44.565541 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.572431 28436 rpcz_store.cc:276] 0709 12:17:43.228091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.233626 (+  5535us) service_pool.cc:224] Handling call
0709 12:17:43.335660 (+102034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.336257 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.571355 (+1235098us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b54002e40d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:44.571538 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39399808}
I20260709 12:17:44.574460 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.574854 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:44.577047 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.563784 30614 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.579010 28436 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:44.562881 30587 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:44.580868 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.582811 28290 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:44.583498 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.583650 28439 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:44.582216 28288 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:44.579952 28330 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:44.584456 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.566140 30632 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.588457 28436 raft_consensus.cc:2410] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:44.593403 30517 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:44.534991 30331 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.597203 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:44.566064 30622 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.590950 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.598400 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:44.630954 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.631548 30622 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:44.631686 28288 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:44.632716 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:44.633338 28288 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.592985 30483 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:44.634251 28362 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:44.634944 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1626) took 1406 ms. Trace:
I20260709 12:17:44.635382 28435 rpcz_store.cc:276] 0709 12:17:43.228453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.233778 (+  5325us) service_pool.cc:224] Handling call
0709 12:17:43.318918 (+ 85140us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.319740 (+1000822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:44.319779 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:44.332439 (+ 12660us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.422939 (+ 90500us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:44.422982 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.423054 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.634684 (+211630us) raft_consensus.cc:1685] finished
0709 12:17:44.634709 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.634874 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1020261,"raft.queue_time_us":1008780,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":7812,"thread_start_us":52570,"threads_started":1,"wal-append.queue_time_us":154833}
I20260709 12:17:44.603870 28432 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:44.641511 30622 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.603513 30590 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:44.642899 30658 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.589s	user 0.037s	sys 0.043s
I20260709 12:17:44.643958 30590 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:44.616349 30517 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.646070 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:44.692101 30485 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:44.693418 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47420.4us
I20260709 12:17:44.694949 30610 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.697924 30590 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:44.698312 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:44.699044 30696 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.203s	user 0.003s	sys 0.000s
I20260709 12:17:44.698750 30590 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.699334 30485 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.701296 30590 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:44.619206 30688 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.620204 30671 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:44.702603 28982 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:44.620364 28330 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:44.622474 30331 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:44.704092 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 83857.8us
I20260709 12:17:44.704680 30671 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:44.623665 30428 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:44.706950 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.707614 28438 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:44.708490 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.709161 28438 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:44.712273 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:44.605131 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.625983 30615 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:44.713160 28991 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:44.641763 30414 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:44.715293 30671 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:44.646385 30571 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.716150 30671 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:44.647114 30622 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.648268 30673 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.718340 30673 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.723443 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1442) took 2130 ms. Trace:
I20260709 12:17:44.724046 28437 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:44.726140 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1627) took 1497 ms. Trace:
I20260709 12:17:44.727510 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:44.650646 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.650048 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.653234 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.655764 30560 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:44.729123 28363 raft_consensus.cc:2410] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:44.729398 28432 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:44.722518 30622 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.729565 30560 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.723831 28360 rpcz_store.cc:276] 0709 12:17:42.592536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.592686 (+   150us) service_pool.cc:224] Handling call
0709 12:17:42.593098 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:42.947175 (+354077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.723166 (+1775991us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b540027db54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:44.723355 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791976704}
I20260709 12:17:44.729133 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.720777 28991 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:44.731832 28438 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:44.732411 30560 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.603680 30678 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:44.583550 30614 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.657706 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:44.662487 30655 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.607s	user 0.002s	sys 0.008s
I20260709 12:17:44.666713 30687 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.733134 30571 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:44.733843 30687 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.734707 30614 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.659155 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:44.667891 30651 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.569s	user 0.009s	sys 0.002s
I20260709 12:17:44.668908 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:44.616771 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.729080 28287 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:44.736826 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 247) took 2179 ms. Trace:
I20260709 12:17:44.666563 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:44.685086 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:44.687783 30478 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.736562 30687 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:44.737982 28290 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:44.738740 30485 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:44.739081 28985 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.602360 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.721643 30414 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:44.737797 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 84111.4us
I20260709 12:17:44.737259 28441 rpcz_store.cc:276] 0709 12:17:42.557531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.664051 (+106520us) service_pool.cc:224] Handling call
0709 12:17:42.671104 (+  7053us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:42.709953 (+ 38849us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:42.709989 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:42.710619 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:17:42.850635 (+140016us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:42.850677 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.850750 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.657397 (+1806647us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:44.736524 (+ 79127us) raft_consensus.cc:1685] finished
0709 12:17:44.736546 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.736760 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10657,"raft.queue_time_us":36076,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":69077888,"thread_start_us":139567,"threads_started":1,"wal-append.queue_time_us":291136}
I20260709 12:17:44.740446 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:44.741549 30414 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.726560 28434 rpcz_store.cc:276] 0709 12:17:43.228748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.959696 (+730948us) service_pool.cc:224] Handling call
0709 12:17:44.130669 (+170973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.334247 (+203578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:44.334277 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:44.334733 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.442737 (+108004us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:44.442782 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.442850 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.698295 (+255445us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:44.725845 (+ 27550us) raft_consensus.cc:1685] finished
0709 12:17:44.725868 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.726080 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":225889,"raft.queue_time_us":209165,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2578,"spinlock_wait_cycles":6092672,"thread_start_us":91716,"threads_started":1,"wal-append.queue_time_us":153545}
I20260709 12:17:44.651687 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:44.742302 30488 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:44.651801 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.742889 30488 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:17:44.742722 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 91134us
I20260709 12:17:44.739558 30478 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.742835 30695 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:44.744220 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:44.743332 28359 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:44.747855 30571 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.748600 30473 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:44.749859 30551 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:44.750562 30551 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:44.751528 30551 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.749927 30473 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:17:44.753007 30551 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.753993 30595 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:44.754395 30595 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:44.755280 30595 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.717060 30671 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.692042 30463 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:44.757112 30678 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:44.720721 30673 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.743244 30414 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:44.758628 30414 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.747304 30488 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.759429 30414 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.758594 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:44.667034 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:44.736974 30685 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:44.764722 30685 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:44.766057 30671 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.765777 30685 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.756570 30595 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.767098 28332 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:44.735523 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 76471.7us
I20260709 12:17:44.771528 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:44.776249 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.778702 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 77937.5us
I20260709 12:17:44.704035 30688 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:44.786717 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 189638us
I20260709 12:17:44.787340 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:44.789944 28332 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:44.795306 28289 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 12:17:44.810562 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.823470 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52027.8us
I20260709 12:17:44.826308 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:44.824708 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:44.829005 28365 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:44.832582 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:44.833690 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:44.786386 30678 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:44.798419 30428 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:44.837502 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:44.697624 30610 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:44.842644 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:44.823344 30615 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:44.851308 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.831187 30687 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:44.834059 30331 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:44.871124 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:44.869844 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 80000.8us
I20260709 12:17:44.872078 28332 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:44.875488 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 208567us
I20260709 12:17:44.799476 30560 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:17:44.843523 30685 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:44.862731 30697 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.883788 30697 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:44.884598 30478 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.885175 30478 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:44.878369 30428 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:44.854910 30678 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.886096 30687 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:44.886844 30675 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:44.887305 30675 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.887423 30478 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:44.887141 30687 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:44.862775 30615 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.888140 30615 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.874735 30331 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:44.883685 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.882647 30685 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.870283 30587 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:44.897114 30331 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.897567 30723 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:44.914038 30517 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:44.914629 28985 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:17:44.915577 30685 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.880836 30571 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:44.876859 30590 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:44.922941 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1599) took 1718 ms. Trace:
I20260709 12:17:44.921447 30485 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:44.921401 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.910023 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:45.124526 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:44.953655 28288 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
W20260709 12:17:45.139017 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 262748us
I20260709 12:17:44.887707 30675 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:45.140012 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 257396us
I20260709 12:17:45.073501 28434 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:45.031522 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:44.923842 28363 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:45.138547 28365 rpcz_store.cc:276] 0709 12:17:43.204418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.299056 (+1094638us) service_pool.cc:224] Handling call
0709 12:17:44.299402 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.829553 (+530151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.922651 (+ 93098us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54003d2f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:44.922849 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217303424}
W20260709 12:17:45.103529 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 230423us
I20260709 12:17:45.142103 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:45.120656 30712 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:17:44.993759 30720 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:17:45.144836 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.898608 30488 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:44.902956 30624 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:44.948875 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.921283 30483 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.158500 28359 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:45.005708 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:45.122383 30678 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:45.172974 28435 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:45.007055 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:45.024747 30697 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:45.139592 30590 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.172091 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:44.899088 30622 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.271s
I20260709 12:17:44.886790 30428 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:45.110319 30615 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:45.052608 30414 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:45.245977 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:45.146221 30625 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:45.437945 28360 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:45.150346 30478 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.175818 28991 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:45.178012 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:45.177567 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:45.304268 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:45.172035 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:45.438293 28366 raft_consensus.cc:2410] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:45.260332 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:17:45.392148 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 242944us
W20260709 12:17:45.409866 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 267856us
I20260709 12:17:45.140667 30614 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:45.224665 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49554.7us
I20260709 12:17:45.156399 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:45.141916 30587 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:45.436985 28358 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.445384 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:45.448437 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:45.462232 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:45.462651 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:45.468289 28441 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:45.453526 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.469586 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.469219 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:45.444386 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:45.158263 30485 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:45.472765 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:45.473654 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:45.476404 28437 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:45.205785 30517 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.299s
I20260709 12:17:45.474309 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:45.443725 30590 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:45.552294 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.507460 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.546885 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.208992 30627 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:45.158008 30688 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.348s
I20260709 12:17:45.482726 28363 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:45.448635 30625 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:45.562137 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 48514.6us
I20260709 12:17:45.571414 28440 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:45.513283 28364 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:45.504908 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:45.508956 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:45.574424 28366 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.463142 30587 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:45.505523 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 245308us
I20260709 12:17:45.224211 30723 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:45.559700 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:45.505551 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:45.435254 30488 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:45.397413 30687 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.505522 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:45.436062 30624 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:45.455291 30483 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:45.505263 28984 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:45.580690 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:45.507972 30673 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:45.562258 30463 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:45.507128 30571 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:45.595129 28359 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:45.575798 30625 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:45.594504 30701 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:17:45.597030 28435 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:45.597612 28441 raft_consensus.cc:2410] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:45.466600 30478 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:45.593245 30624 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.610884 28367 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:45.698311 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:45.700418 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:45.593983 30642 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:17:45.594285 30605 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:17:45.561856 30721 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:17:45.661669 30571 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:45.704263 28367 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:45.705058 30571 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:45.637270 30627 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:45.705811 30627 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.705761 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:45.706642 30627 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.706920 28367 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:45.709829 28366 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:45.711778 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:45.675482 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 200519us
I20260709 12:17:45.666395 30723 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:45.714501 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:45.610733 30483 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:45.683678 28358 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:45.609674 28361 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:45.720395 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:45.721210 30716 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:45.582566 30722 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 12:17:45.716915 28366 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:45.611063 30331 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:45.595278 30687 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:45.716813 30627 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:45.603073 30488 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:45.598344 30726 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.184s
I20260709 12:17:45.749223 28440 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:17:46.146495 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.896106 28290 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:45.992635 28285 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:17:45.936036 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.155097 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 273) took 1043 ms. Trace:
I20260709 12:17:46.151463 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1471) took 1102 ms. Trace:
I20260709 12:17:45.882025 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:46.108397 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 93208.9us
W20260709 12:17:46.166750 28437 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.010s	sys 0.011s
W20260709 12:17:46.167333 28437 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.011s	sys 0.011s
I20260709 12:17:45.763522 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:45.830871 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.847600 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.155685 28990 rpcz_store.cc:276] 0709 12:17:45.112229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.112435 (+   206us) service_pool.cc:224] Handling call
0709 12:17:45.112869 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.119333 (+  6464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:45.119363 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:45.476546 (+357183us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.708263 (+231717us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:45.708643 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 12:17:45.708882 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.708973 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:46.142402 (+433429us) raft_consensus.cc:1685] finished
0709 12:17:46.142421 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.155025 (+ 12604us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1411,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":1773,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":373,"thread_start_us":118885,"threads_started":1,"wal-append.queue_time_us":569482}
I20260709 12:17:46.157171 28366 rpcz_store.cc:276] 0709 12:17:45.049628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.716191 (+666563us) service_pool.cc:224] Handling call
0709 12:17:45.716538 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:46.151125 (+434587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.151381 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:46.205355 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 441951us
I20260709 12:17:46.008152 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:45.896404 28292 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:46.205534 28367 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:46.208364 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 766036us
W20260709 12:17:46.208725 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47553.2us
I20260709 12:17:46.002825 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.210075 28289 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:46.008541 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:45.949276 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 376636us
I20260709 12:17:46.211169 28291 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:46.211374 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:46.230136 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.231544 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.208711 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:45.837037 30736 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
W20260709 12:17:45.815552 30730 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 12:17:46.139791 30743 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:17:45.890064 30414 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.235724 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.241124 28283 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
W20260709 12:17:46.016398 30737 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 12:17:46.145166 30428 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.831777 30571 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.993642 30615 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:45.770136 30463 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:46.203516 30483 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.249481 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:46.255283 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 45767.2us
I20260709 12:17:45.815131 30738 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.145195 30627 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:46.245065 30685 consensus_meta.cc:287] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.071s	user 0.014s	sys 0.000s
W20260709 12:17:46.238447 30723 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.016s	sys 0.037s
I20260709 12:17:46.139979 30624 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.270555 30685 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.271371 30605 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:46.258356 30428 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:46.266566 30723 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.016s	sys 0.037s
W20260709 12:17:46.287405 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:46.278561 30735 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.554s	user 0.002s	sys 0.010s
W20260709 12:17:46.294549 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.258942 30749 raft_consensus.cc:2764] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:46.296051 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.267109 30722 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:46.297894 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 265) took 1367 ms. Trace:
I20260709 12:17:46.301385 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1466) took 1316 ms. Trace:
I20260709 12:17:46.243556 30731 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.048s
W20260709 12:17:46.293066 30675 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.090s	user 0.013s	sys 0.000s
I20260709 12:17:46.301959 28362 rpcz_store.cc:276] 0709 12:17:44.985206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.985349 (+   143us) service_pool.cc:224] Handling call
0709 12:17:44.985753 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.986043 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:44.986070 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:45.430888 (+444818us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.678862 (+247974us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:45.678904 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.679004 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:46.301106 (+622102us) raft_consensus.cc:1685] finished
0709 12:17:46.301133 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.301309 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":116362,"threads_started":1,"wal-append.queue_time_us":316819}
I20260709 12:17:46.301772 28439 rpcz_store.cc:276] 0709 12:17:44.930605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.981901 (+ 51296us) service_pool.cc:224] Handling call
0709 12:17:44.993041 (+ 11140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:45.435742 (+442701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:45.435774 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:45.435815 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:45.435848 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.297455 (+861607us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b54003cbc54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:46.297513 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.297700 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1017618176}
I20260709 12:17:46.271602 30742 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:46.291971 30605 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:46.309705 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:46.310084 30605 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:46.261372 30483 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.311879 30747 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:46.312538 30747 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.313853 30675 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.314277 30463 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.315196 30564 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:46.315608 30564 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:46.316087 30675 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.307322 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:46.316665 30564 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.307555 30739 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:46.318080 28365 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:46.318260 30739 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:46.318722 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:46.323498 28439 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.326046 28432 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.319687 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1474) took 1215 ms. Trace:
I20260709 12:17:46.333685 28365 rpcz_store.cc:276] 0709 12:17:45.106040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.156089 (+ 50049us) service_pool.cc:224] Handling call
0709 12:17:46.319471 (+1163382us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b5400127d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:46.319602 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1983268736}
W20260709 12:17:46.345052 30590 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.001s
I20260709 12:17:46.346330 30564 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.319307 30739 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.350668 30331 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:46.353245 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.259070 30571 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.319028 30624 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:46.354525 30747 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:46.355424 30571 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:46.267946 30738 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.356813 28287 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:46.364140 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:46.379696 28437 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.301855 30722 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:46.405963 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1473) took 1335 ms. Trace:
I20260709 12:17:46.406451 28363 rpcz_store.cc:276] 0709 12:17:45.070283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.156703 (+ 86420us) service_pool.cc:224] Handling call
0709 12:17:46.405702 (+1248999us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b5400125f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:46.405882 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105283968}
I20260709 12:17:46.319484 30742 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:46.320191 30687 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.320135 30483 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.320844 30734 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:46.414019 28432 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:46.432927 28439 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
W20260709 12:17:46.434643 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 142412us
I20260709 12:17:46.442888 28432 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:17:46.443243 28432 pending_rounds.cc:85] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 2
I20260709 12:17:46.345664 30590 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.411545 28982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 275) took 1284 ms. Trace:
I20260709 12:17:46.406647 30738 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.456231 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1527) took 1003 ms. Trace:
I20260709 12:17:46.443423 28432 pending_rounds.cc:107] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:46.432933 30571 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:46.457320 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1470) took 1426 ms. Trace:
I20260709 12:17:46.435673 30734 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:46.460819 28361 rpcz_store.cc:276] 0709 12:17:45.031009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.031168 (+   159us) service_pool.cc:224] Handling call
0709 12:17:46.457109 (+1425941us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54003700d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:46.457252 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062415744}
I20260709 12:17:46.460822 28432 raft_consensus.cc:2889] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:46.464883 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 255333us
I20260709 12:17:46.411434 30742 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:46.459739 28433 rpcz_store.cc:276] 0709 12:17:45.453295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.559617 (+106322us) service_pool.cc:224] Handling call
0709 12:17:45.559967 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:45.560185 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:45.887875 (+327690us) log.cc:844] Serialized 14 byte log entry
0709 12:17:46.455902 (+568027us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:46.455957 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:46.456003 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.456063 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.456166 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":690038,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":4514,"thread_start_us":477138,"threads_started":2}
I20260709 12:17:46.465385 30571 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.472113 30571 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:46.464677 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 233175us
W20260709 12:17:46.466518 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 217133us
I20260709 12:17:46.468566 28361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:46.473500 28361 raft_consensus.cc:606] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.473909 28361 raft_consensus.cc:614] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:46.479624 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.479658 30722 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.457769 30551 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.487578 30551 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.458065 30483 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:46.495101 30483 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.498775 30685 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.499907 30734 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.501221 30734 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:46.501394 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.502064 30759 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.491858 30551 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.502686 30759 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.497623 30483 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.505717 30747 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:46.506512 30747 consensus_queue.cc:1243] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:17:46.507012 30675 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:46.507476 30675 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:46.563767 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:46.564086 30770 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:46.501896 30625 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.840s	user 0.014s	sys 0.001s
I20260709 12:17:46.571741 30564 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.572655 30615 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:46.572424 30564 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.573088 30615 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.573516 30615 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.573760 28332 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:46.574110 30770 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.574916 30622 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:46.575429 30564 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:46.508514 30675 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:46.578094 30760 consensus_queue.cc:1175] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 6 status { last_received { term: 6 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 6 index: 4 } }
W20260709 12:17:46.565550 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 59155.1us
I20260709 12:17:46.579345 30723 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.579540 30483 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.580015 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.580260 30483 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.579968 30723 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.581243 30770 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:46.581571 30571 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:46.572186 30625 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.582523 30483 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.579487 30747 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.583494 30747 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:46.570876 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 91384.9us
I20260709 12:17:46.584661 30723 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:46.582855 28440 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:46.585458 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.586050 30747 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:46.562959 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 82530.5us
I20260709 12:17:46.562769 30722 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:46.588850 30759 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.541620 28361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:46.570170 30773 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:46.564987 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.591466 28361 raft_consensus.cc:606] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.591986 28361 raft_consensus.cc:614] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:46.592453 30564 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:46.593922 30564 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.594400 30564 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:46.593294 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.594656 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:46.595286 30685 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:46.596174 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:46.596683 30762 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:46.596846 28363 raft_consensus.cc:606] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.597378 28363 raft_consensus.cc:614] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:46.597600 30762 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:46.576622 30622 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:46.599524 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.508414 30590 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.583527 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:46.602910 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.599524 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:46.583935 30625 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:46.610589 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.611763 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:46.661301 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:46.663344 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:46.456019 28982 rpcz_store.cc:276] 0709 12:17:45.124359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.124721 (+   362us) service_pool.cc:224] Handling call
0709 12:17:45.125174 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.125552 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:45.125583 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:45.504739 (+379156us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.654119 (+149380us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:45.654175 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.654269 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:46.358220 (+703951us) raft_consensus.cc:1685] finished
0709 12:17:46.364081 (+  5861us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:46.408713 (+ 44632us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":142439,"threads_started":1,"wal-append.queue_time_us":242004}
I20260709 12:17:46.473166 30742 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.664711 28433 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:17:46.664774 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 65361.7us
I20260709 12:17:46.604979 30770 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:46.669484 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 86063.5us
I20260709 12:17:46.671762 28438 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:46.669884 28432 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:46.670544 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:46.667207 30759 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:46.672735 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:46.676098 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.677873 30747 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:46.678026 30770 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.678653 30747 consensus_queue.cc:1243] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:46.669255 30463 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:46.679563 30727 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:46.680176 30727 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.673779 30759 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.685956 30625 raft_consensus.cc:2764] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:17:46.687577 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:46.674433 28433 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:46.690220 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.690838 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:46.679145 30747 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:46.692101 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:46.692838 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.693800 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:46.694392 28366 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:46.696537 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:46.697160 28366 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:46.701249 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:46.701699 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:46.701411 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.702534 28367 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:46.702497 28288 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:46.561849 30605 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.729394 28361 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:46.704787 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:46.706038 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:46.737349 28434 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:46.673069 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 73646.2us
I20260709 12:17:46.742224 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.742710 28284 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:46.743043 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.665606 30590 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.743737 28288 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:46.742728 28287 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.743156 28285 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:46.744781 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.745944 28288 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:46.746521 28366 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:46.749439 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.749366 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:46.750564 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:46.751114 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:46.751313 28433 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:46.742353 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.702589 30759 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:46.757771 28404 connection.cc:256] RPC call timeout handler was delayed by -2.3414085680001335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 12:17:46.758025 30770 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:46.758275 28404 connection.cc:256] RPC call timeout handler was delayed by -2.2694085680002445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
W20260709 12:17:46.758634 28404 connection.cc:256] RPC call timeout handler was delayed by -2.2163085680001586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
W20260709 12:17:46.758929 28404 connection.cc:256] RPC call timeout handler was delayed by -2.1839085680001062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
W20260709 12:17:46.759274 28404 connection.cc:256] RPC call timeout handler was delayed by -2.169508568000083s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
W20260709 12:17:46.759563 28404 connection.cc:256] RPC call timeout handler was delayed by -2.16140856800007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1773s for a response
W20260709 12:17:46.759815 28404 connection.cc:256] RPC call timeout handler was delayed by -2.1029085680002026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 12:17:46.760095 28404 connection.cc:256] RPC call timeout handler was delayed by -2.0435085680001066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1904s for a response
W20260709 12:17:46.760375 28404 connection.cc:256] RPC call timeout handler was delayed by -2.0327085680000891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 12:17:46.760632 28404 connection.cc:256] RPC call timeout handler was delayed by -1.9841085680002379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
W20260709 12:17:46.760870 28404 connection.cc:256] RPC call timeout handler was delayed by -1.9832085680002365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19710000000000003s for a response
W20260709 12:17:46.761125 28404 connection.cc:256] RPC call timeout handler was delayed by -1.9184085680001317s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 12:17:46.761372 28404 connection.cc:256] RPC call timeout handler was delayed by -1.881508568000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
W20260709 12:17:46.761969 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 70366.4us
I20260709 12:17:46.763478 30762 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.763648 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.764180 30762 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.765805 30746 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:46.702003 30747 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:46.701860 30463 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:46.767571 30615 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:46.768730 30463 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.769240 30759 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:46.761632 28404 connection.cc:256] RPC call timeout handler was delayed by -1.8347085680002237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
W20260709 12:17:46.770741 28404 connection.cc:256] RPC call timeout handler was delayed by -1.8302085680002165s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2141s for a response
W20260709 12:17:46.771220 28404 connection.cc:256] RPC call timeout handler was delayed by -1.809508568000183s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
W20260709 12:17:46.771588 28404 connection.cc:256] RPC call timeout handler was delayed by -1.7420085680000739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
I20260709 12:17:46.773496 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.779141 28365 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:46.759054 30727 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.600335 30622 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.550443 30485 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.787616 30615 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.789585 28362 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:46.801172 30485 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.790135 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.802634 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:46.789984 30749 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.793430 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.807716 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.803527 28292 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:46.816953 30752 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:46.818040 30752 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.818272 28284 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:46.749341 30610 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:46.775532 30722 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:46.835290 28362 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:17:46.828703 28287 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:46.837762 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:46.830026 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.750794 30769 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:46.826694 28437 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:46.838907 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:46.833089 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:46.839679 28361 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.840708 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:46.809755 30587 consensus_meta.cc:287] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.153s	user 0.010s	sys 0.003s
I20260709 12:17:46.842063 30587 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.843887 28439 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:46.839116 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 252472us
I20260709 12:17:46.848006 28367 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:17:46.848376 28367 pending_rounds.cc:85] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 6
I20260709 12:17:46.848615 28367 pending_rounds.cc:107] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:17:46.848850 28367 raft_consensus.cc:2889] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:46.852252 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.854545 28361 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:17:46.837996 30610 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.856688 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:46.864900 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.858237 30605 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:46.858451 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:46.865648 28284 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:46.870747 28434 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
W20260709 12:17:46.867271 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.869092 28330 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.872635 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:46.873669 28361 raft_consensus.cc:2410] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:17:46.876381 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 73835.7us
I20260709 12:17:46.836221 30734 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:17:46.879844 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.838857 30722 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:46.885246 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:46.886059 28990 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:46.888335 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:46.888983 28990 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:46.891429 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:46.877614 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:46.835654 30738 consensus_queue.cc:1059] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:17:46.866775 30773 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.866359 30673 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:46.875037 30483 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:46.875505 30723 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:46.893728 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:46.904938 28284 raft_consensus.cc:2410] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:46.880580 30762 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:46.907199 28990 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:17:46.891758 30722 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:46.910920 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:46.908902 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:46.892129 30571 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:46.856742 30747 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:46.922091 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:46.922531 30747 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.925354 30610 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:46.933662 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:46.934283 30781 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:46.935003 30781 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:46.905942 30773 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.905678 30615 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:46.939937 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:47.134687 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:47.027282 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 118451us
I20260709 12:17:47.250398 28436 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
W20260709 12:17:47.063068 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 182033us
I20260709 12:17:47.188779 28291 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:47.092315 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:47.172279 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1562) took 1266 ms. Trace:
W20260709 12:17:47.221585 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 320681us
I20260709 12:17:46.939762 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:47.202463 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:47.253607 28439 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.189450 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1534) took 1265 ms. Trace:
I20260709 12:17:47.253494 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:17:47.089443 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.144495 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:47.016651 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:47.142534 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:47.254288 28440 rpcz_store.cc:276] 0709 12:17:45.924181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.144182 (+220001us) service_pool.cc:224] Handling call
0709 12:17:46.144593 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:46.585919 (+441326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.189138 (+603219us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54003cb254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:47.189371 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007475072}
I20260709 12:17:47.024544 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:47.112195 28364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1478) took 1930 ms (client timeout 1528 ms). Trace:
W20260709 12:17:47.256314 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 111889us
I20260709 12:17:46.935151 30622 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:46.939359 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:47.253937 28438 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:47.138501 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:47.057631 30779 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:17:47.257555 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:46.999536 30769 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:47.298472 28437 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:47.114504 30614 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:47.174952 30428 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:47.274693 28432 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:47.274055 28283 rpcz_store.cc:276] 0709 12:17:45.905879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.906029 (+   150us) service_pool.cc:224] Handling call
0709 12:17:47.172070 (+1266041us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b54000b1dd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:47.172183 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78963968}
I20260709 12:17:47.281728 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:47.325322 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 43682.9us
W20260709 12:17:47.208232 30782 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.150s	user 0.000s	sys 0.002s
I20260709 12:17:47.150599 30675 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:47.271626 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.202997 30488 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.935554 30587 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:47.148221 30752 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:46.954087 30721 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.290607 28434 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:47.070621 30722 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.095556 30759 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:47.293109 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:47.295277 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.329276 30770 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:47.114251 30478 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:46.967053 30632 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.135993 28982 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:46.934090 30732 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.355s
W20260709 12:17:47.297830 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.093031 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:47.258096 30738 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:47.299906 28332 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:47.207604 28190 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 4, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 1 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:17:47.348613 28358 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:17:47.305907 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 54916.5us
I20260709 12:17:46.997648 30775 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:47.081274 30723 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.986814 30749 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:47.314324 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 60930.8us
I20260709 12:17:47.314953 30614 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:47.315232 30642 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:47.316521 30769 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:46.967849 30571 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:47.317230 30773 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:47.145678 30590 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.164s
I20260709 12:17:47.211083 30765 consensus_queue.cc:1059] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 12:17:47.319479 30615 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:47.319509 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:47.382921 28437 raft_consensus.cc:2410] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:17:47.274820 28435 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
W20260709 12:17:47.256695 28364 rpcz_store.cc:269] 0709 12:17:45.175928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.448095 (+272167us) service_pool.cc:224] Handling call
0709 12:17:47.106442 (+1658347us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5400102f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:47.106624 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23128320}
I20260709 12:17:47.326685 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:47.384940 28440 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:47.326936 28439 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:47.330417 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:47.290499 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:47.070950 30564 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.160657 30761 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.163s
I20260709 12:17:47.391772 28438 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:47.392051 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:47.071465 30781 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.333369 28187 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 236) took 9133 ms. Trace:
I20260709 12:17:47.330969 30685 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:17:47.290980 30622 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:47.394778 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 64456.5us
I20260709 12:17:47.329233 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:47.396070 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66927.5us
I20260709 12:17:47.397078 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:47.341660 28432 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:17:47.070648 30742 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:47.296063 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:47.344481 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 475409us
I20260709 12:17:47.401852 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:47.402101 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:47.361379 30738 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:47.350507 28255 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:47.352185 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52385.6us
I20260709 12:17:47.355324 28190 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:47.375669 28441 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:47.375188 30571 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.357378 30759 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:47.409991 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:47.398437 28432 pending_rounds.cc:85] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 3
I20260709 12:17:47.410979 28432 pending_rounds.cc:107] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:17:47.411281 28432 raft_consensus.cc:2889] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:47.408139 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 57756us
I20260709 12:17:47.412983 28255 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:47.391413 30615 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.419284 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:47.414356 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:47.414456 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.443780 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:47.404558 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:47.399495 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 276) took 2269 ms. Trace:
I20260709 12:17:47.358130 30723 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:47.467805 30723 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:47.469815 30564 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:47.472765 30722 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:47.473631 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59007.4us
I20260709 12:17:47.415030 30739 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:47.428233 30742 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:47.410284 30627 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.430037 28190 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:47.482829 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 72924.5us
I20260709 12:17:47.418032 30760 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:17:47.415591 30759 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.409919 30745 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:47.415163 30776 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:47.413975 30747 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:17:47.434326 30755 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:47.414817 30671 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.436133 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:47.399731 30560 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:17:47.403678 30716 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:47.393195 28187 rpcz_store.cc:276] 0709 12:17:38.199883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.200086 (+   203us) service_pool.cc:224] Handling call
0709 12:17:47.333219 (+9133133us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:44.382993 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:44.383807 (+   814us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:44.383876 (+    69us) write_op.cc:435] Acquired schema lock
0709 12:17:44.383915 (+    39us) tablet.cc:662] Decoding operations
0709 12:17:44.399616 (+ 15701us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:44.399713 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:44.399745 (+    32us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:17:44.401228 (+  1483us) tablet.cc:701] Row locks acquired
0709 12:17:44.401278 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:17:44.401474 (+   196us) write_op.cc:270] Start()
0709 12:17:44.401645 (+   171us) write_op.cc:276] Timestamp: P: 1783599464401396 usec, L: 0
0709 12:17:44.401665 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:17:44.409584 (+  7919us) log.cc:844] Serialized 6731 byte log entry
0709 12:17:44.981208 (+571624us) op_driver.cc:464] REPLICATION: finished
0709 12:17:45.288552 (+307344us) write_op.cc:301] APPLY: starting
0709 12:17:45.288797 (+   245us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:45.289402 (+   605us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:45.289664 (+   262us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:45.602876 (+313212us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:45.603349 (+   473us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:17:45.603400 (+    51us) write_op.cc:312] APPLY: finished
0709 12:17:45.603881 (+   481us) log.cc:844] Serialized 119 byte log entry
0709 12:17:45.604115 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:45.604320 (+   205us) write_op.cc:454] Released schema lock
0709 12:17:45.604404 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:45.604956 (+   552us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:45.604988 (+    32us) write_op.cc:454] Released schema lock
0709 12:17:45.669768 (+ 64780us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88432a 00007f85be809f14 00007f85be809ec3 00007f85be809e38 00007f85be809df3 00007f85be8090dc 00007f85be832e16 00007f85be81acbb 00007f85be80539d 00007f85be808d75
Metrics: {"mutex_wait_us":10374,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":242851,"apply.run_cpu_time_us":7235,"apply.run_wall_time_us":351574,"num_ops":17,"prepare.queue_time_us":185960,"prepare.run_cpu_time_us":21021,"prepare.run_wall_time_us":196571,"raft.queue_time_us":259652,"raft.run_cpu_time_us":17109,"raft.run_wall_time_us":779771,"replication_time_us":579309,"spinlock_wait_cycles":91930368,"thread_start_us":557079,"threads_started":4,"wal-append.queue_time_us":190956,"wal-append.run_cpu_time_us":17926,"wal-append.run_wall_time_us":1982423}]]}
I20260709 12:17:47.431243 30622 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.361137 30775 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:47.454250 30738 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.460745 28363 raft_consensus.cc:606] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:47.519704 28363 raft_consensus.cc:614] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:47.442571 30590 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.429842 30734 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:47.485329 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:47.415400 30642 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:47.422504 30752 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:47.417536 30773 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:47.495951 28404 leader_election.cc:400] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6.
I20260709 12:17:47.548211 28404 leader_election.cc:403] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:47.481184 30739 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.486992 30627 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.489070 30564 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:47.511719 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 68038.2us
I20260709 12:17:47.517361 30769 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.553716 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:47.553887 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 149444us
I20260709 12:17:47.555047 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:47.492405 30632 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:47.492591 30759 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.560335 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:47.494313 28190 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 1 to 4, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 4 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:47.515460 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 96258.1us
I20260709 12:17:47.492190 30745 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.564641 30745 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.492077 30485 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.565727 30485 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:47.568336 30485 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:47.545074 30642 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:47.550777 30739 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.552644 30627 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.490564 30732 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:17:47.423382 30614 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.587965 30614 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.500321 30755 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:47.590488 30755 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:47.548923 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63661.3us
I20260709 12:17:47.519492 30595 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.521341 30590 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.558931 28434 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:47.522065 30746 raft_consensus.cc:3055] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:47.563521 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:47.560734 30769 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:47.552860 30571 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:47.550549 30773 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:47.461337 28986 rpcz_store.cc:276] 0709 12:17:45.124688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.154977 (+ 30289us) service_pool.cc:224] Handling call
0709 12:17:45.175928 (+ 20951us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.176430 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:45.176467 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:46.674045 (+1497578us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b2c00095d78. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:17:46.819474 (+145429us) log.cc:844] Serialized 26 byte log entry
0709 12:17:46.906316 (+ 86842us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:46.912653 (+  6337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.941669 (+ 29016us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.207621 (+265952us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:47.376279 (+168658us) raft_consensus.cc:1685] finished
0709 12:17:47.376301 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.394069 (+ 17768us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2458,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":17723,"spinlock_wait_cycles":9742976,"thread_start_us":39995,"threads_started":1,"wal-append.queue_time_us":201990}
W20260709 12:17:47.582473 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 98777.2us
I20260709 12:17:47.574098 30642 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.614204 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:47.554687 30587 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:47.619005 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.575809 30632 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:47.605449 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:47.563513 30685 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:47.608373 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44957.3us
I20260709 12:17:47.594702 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:47.593062 30734 consensus_queue.cc:1175] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 4 status { last_received { term: 4 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 4 index: 3 } }
I20260709 12:17:47.625898 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:47.607671 30590 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.626526 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:47.630352 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:47.613991 30671 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:47.637913 30747 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.642074 28190 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:47.643867 30793 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:17:47.644034 28190 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 2 to 3, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 3 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:47.649087 30769 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:47.607733 30746 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:47.612785 30796 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:47.616377 30571 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.616644 30755 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:47.653275 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:47.723258 28363 raft_consensus.cc:606] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:47.665225 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:47.666137 28190 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:47.663878 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:47.667703 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:47.668259 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:47.628168 30752 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.668406 30799 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:47.669165 28986 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 4 LEARNER]: Advancing to term 5
W20260709 12:17:47.669727 30790 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:17:47.671973 28441 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:47.672284 30732 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:47.576033 30794 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:17:47.676483 30760 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:47.678179 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.679780 30759 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.682626 30587 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6.
I20260709 12:17:47.689983 30671 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:47.690173 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:47.695732 30791 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
I20260709 12:17:47.706252 30632 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.708424 30773 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:47.709620 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:47.867544 28436 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:47.651379 30595 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.725641 28991 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:17:47.726874 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63086.3us
W20260709 12:17:47.732986 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 178025us
I20260709 12:17:47.724664 30571 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:47.742527 28283 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:47.742064 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:47.745384 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 80250.1us
I20260709 12:17:47.816083 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:47.742064 30742 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
W20260709 12:17:47.805456 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 201462us
I20260709 12:17:47.728780 30675 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.732116 30752 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:47.739632 30732 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.739184 28985 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:47.865221 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:47.880533 30732 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.880993 28989 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:47.883911 30642 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.906868 30564 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:47.907336 30564 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:47.907657 30770 consensus_meta.cc:287] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.000s
I20260709 12:17:47.908217 30564 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.726063 30796 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:47.846051 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:47.909974 28984 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:47.739601 30785 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.910589 30785 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:47.909539 30564 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.871996 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:47.912207 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:47.893396 30587 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:17:47.910182 30796 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.874475 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:47.856319 30760 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:47.885973 30675 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.886713 30752 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.888877 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.930387 28283 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:47.889003 30739 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.779151 30746 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:47.890111 28285 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:47.922345 30488 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:47.739356 30749 raft_consensus.cc:2764] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:47.892264 30732 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.892701 30745 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:47.893291 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.894207 30627 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.894779 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:47.974717 28288 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:47.895788 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:47.895952 30685 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:47.897166 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.897449 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.897955 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:47.898312 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.899348 30759 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:47.871745 30595 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.856878 30671 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.905391 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:47.924093 30723 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:17:47.993511 30746 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:47.903596 30722 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
W20260709 12:17:47.924273 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.908134 30770 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:47.925614 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 50190.1us
I20260709 12:17:48.048202 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:47.887957 30590 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:47.923267 30775 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:47.977749 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.978003 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:48.064039 28289 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.981061 28292 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:47.982936 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:47.983870 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 110806us
I20260709 12:17:48.065435 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.066278 28367 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:47.992085 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.911036 30785 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.993548 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.996134 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:47.995848 30685 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:47.997198 30765 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:47.997325 28983 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:47.996979 30760 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:47.997021 30671 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:48.001395 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.007241 30732 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:48.009022 30745 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:47.931946 30799 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:17:48.009756 28291 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:47.988584 30595 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:47.995878 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:48.020814 30615 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 12:17:48.022531 30642 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:47.972939 30488 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.013615 30747 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.026084 28986 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 5 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:17:48.027066 30675 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:48.052591 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.058203 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.135377 28359 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:48.082984 30759 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.136504 28283 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:48.058845 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.079411 30765 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:48.051379 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:48.057888 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:48.067315 28255 connection.cc:256] RPC call timeout handler was delayed by -2.2886055419999138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
W20260709 12:17:48.141499 28255 connection.cc:256] RPC call timeout handler was delayed by -2.2499055419998513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
I20260709 12:17:48.141495 28365 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.067327 28286 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 2.
W20260709 12:17:48.069698 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 198639us
I20260709 12:17:48.072039 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:48.146481 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:48.147068 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:48.073640 28358 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:48.137404 28366 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:48.064996 28285 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:48.148813 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:48.077215 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.145890 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:48.150652 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:48.078217 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:48.079927 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:48.153266 28290 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:48.073403 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:48.155315 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:48.156543 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:48.081903 28362 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:48.082994 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:48.159775 28988 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:48.083657 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.083917 28432 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:48.089069 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:48.097321 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:48.098481 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 254150us
I20260709 12:17:48.112969 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:48.100137 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:48.103542 28288 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:48.099678 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:48.183581 28292 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:48.111491 28287 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:48.078330 30798 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.060705 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.099327 28982 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:48.162377 28954 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:48.133222 30571 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.050344 30794 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:48.162278 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.127008 28986 pending_rounds.cc:85] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 3
W20260709 12:17:48.190893 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 85250.9us
I20260709 12:17:48.119567 30488 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.196537 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:48.200253 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 54444.8us
I20260709 12:17:48.221421 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:48.223507 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:48.224406 30781 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:48.225997 30781 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:48.226550 30781 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:48.227764 28362 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:48.227512 30781 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.231199 28362 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:48.217849 28288 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:48.218582 28432 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:48.233446 28432 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:48.234352 28432 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.235339 28432 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.219020 30810 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:48.238385 30810 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.238786 30810 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.242914 30749 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:48.243511 30749 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:48.243875 30749 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.203501 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:48.112488 30615 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.180346 30632 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.163738 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:48.249724 28440 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:48.261030 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.262688 28362 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:48.270584 30615 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:48.245398 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:48.281598 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66934.7us
I20260709 12:17:48.291462 28440 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:17:48.214352 28986 pending_rounds.cc:107] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:17:48.200167 30716 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:48.151538 30759 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.285367 30614 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:48.304306 28432 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:17:48.191524 30587 consensus_queue.cc:1059] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.107s
I20260709 12:17:48.202039 28984 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:48.342676 30627 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:48.325668 30739 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:48.201609 30798 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.352164 30798 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:48.323561 28986 raft_consensus.cc:2889] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:48.356855 28365 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:48.355270 28284 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:48.320410 30632 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.325826 28432 pending_rounds.cc:85] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 2
I20260709 12:17:48.370659 28432 pending_rounds.cc:107] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:17:48.371100 28432 raft_consensus.cc:2889] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:48.379652 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 183202us
I20260709 12:17:48.390049 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.393491 28365 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.390931 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:48.390427 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 158623us
I20260709 12:17:48.395490 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.399348 28366 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:48.400722 28432 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:48.404122 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:48.396713 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 244574us
I20260709 12:17:48.416370 28362 raft_consensus.cc:2410] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:48.419703 28366 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:17:48.420095 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:48.424497 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:48.428036 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:48.429927 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:48.421571 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:48.372788 30809 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.399240 30615 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.436721 30478 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:17:48.439813 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.431651 30675 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.364991 30642 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:17:48.451745 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.452288 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.452965 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:48.473518 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.473946 28362 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.454675 30478 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.431140 30747 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.458746 30828 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:48.438294 30590 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:48.461633 30806 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 12:17:48.462352 30813 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
W20260709 12:17:48.462878 30802 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
I20260709 12:17:48.466907 28289 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:48.437103 30811 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:48.458907 30801 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 12:17:48.431555 30671 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:17:48.437110 30732 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:48.442787 30745 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:48.472880 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.465134 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:48.474228 28363 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.474939 28358 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:48.438087 30685 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:48.395186 30817 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.181s
I20260709 12:17:48.550576 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:48.459669 30818 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:17:48.445235 30615 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.463065 30488 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.477524 28365 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:48.468786 30770 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.475390 30478 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:48.518388 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 53337.6us
I20260709 12:17:48.521309 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.512460 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:48.550712 28289 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:17:48.551795 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 149102us
I20260709 12:17:48.554749 28367 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:48.552472 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:48.526542 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:48.591320 28358 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:48.624037 28360 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:48.625794 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:48.627840 28288 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.526296 30732 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:48.464534 30809 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.531548 30571 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:48.650418 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 220558us
I20260709 12:17:48.632836 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:48.505812 30828 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.654721 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:48.530942 30590 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:48.627005 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 114641us
W20260709 12:17:48.625982 30752 consensus_meta.cc:287] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.001s
I20260709 12:17:48.629132 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:48.559573 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:48.513262 30826 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.653079 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1549) took 1093 ms. Trace:
I20260709 12:17:48.631193 30622 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.594048 30759 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:48.661712 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:48.667824 28435 rpcz_store.cc:276] 0709 12:17:47.559615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.559737 (+   122us) service_pool.cc:224] Handling call
0709 12:17:47.560869 (+  1132us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:47.561357 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:47.561387 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:47.575096 (+ 13709us) log.cc:844] Serialized 26 byte log entry
0709 12:17:47.932623 (+357527us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:47.932666 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.932733 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:48.652833 (+720100us) raft_consensus.cc:1685] finished
0709 12:17:48.652856 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:48.653009 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3033,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":312925,"threads_started":1,"wal-append.queue_time_us":467863}
I20260709 12:17:48.671639 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:48.630224 30478 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.557960 30587 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:48.592832 30685 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:48.666488 30752 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.688417 30685 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:48.690179 30752 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:48.646335 30734 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:48.665585 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:48.647344 30732 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:48.615497 30615 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.598610 30488 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:48.598263 30642 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
W20260709 12:17:48.684429 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 229455us
W20260709 12:17:48.685504 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 56465.7us
I20260709 12:17:48.686457 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:48.518296 30810 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:48.662853 30820 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.664142 30571 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.667573 30716 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:48.695200 30829 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:48.695822 30590 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.698293 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:48.768271 30820 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:17:48.768802 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 70617.6us
I20260709 12:17:48.769176 30590 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.769665 28438 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:48.770164 30781 raft_consensus.cc:2764] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:48.769832 30590 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.770906 30781 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:48.771845 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:48.772920 30590 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:48.773927 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:48.667672 30826 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:48.775637 30810 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:48.775781 30826 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.776818 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.778102 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.778517 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:48.682687 30800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.001s	sys 0.006s
W20260709 12:17:48.779516 30800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.001s	sys 0.006s
I20260709 12:17:48.780424 30829 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:48.781075 28364 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:48.781481 28360 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:48.781368 30829 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.782256 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:48.783030 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.783774 28360 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:48.783809 30752 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:48.784278 30752 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:48.784658 30752 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.784955 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:48.785605 28360 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:17:48.786001 28360 pending_rounds.cc:85] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 6
I20260709 12:17:48.786209 28360 pending_rounds.cc:107] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:17:48.786445 28360 raft_consensus.cc:2889] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:48.787124 28984 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:48.788401 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.789896 28986 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:48.791071 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:48.760609 28288 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:48.647941 30770 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.798321 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:48.801443 30770 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:48.702926 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:48.762754 28290 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:48.687618 30587 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:48.764982 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 64698.3us
I20260709 12:17:48.803560 30587 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.804706 30587 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.805706 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:48.808192 30800 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.808707 30800 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.809072 30800 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.650254 30745 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:17:48.802841 28290 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:48.815994 30587 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:48.816798 30587 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.819054 30587 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:48.809134 28985 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:17:48.823566 28364 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:48.677011 30824 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 12:17:48.707180 30799 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.864737 28364 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:17:48.869010 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 100320us
I20260709 12:17:48.870383 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:48.870679 30838 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.873379 30838 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.873876 30838 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.871038 30587 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:48.874507 30587 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:48.874908 30587 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.877388 30809 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:48.767385 30840 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.703914 30685 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.878165 30840 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:48.879316 30749 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.003s
I20260709 12:17:48.879777 30749 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.880529 30840 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:48.677800 30622 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.830837 30817 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.886700 28364 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:48.887518 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 90153.9us
I20260709 12:17:48.888396 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:48.890916 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:48.906356 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:48.907310 28365 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:48.919906 28364 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:17:48.885185 30746 consensus_meta.cc:287] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.845s	user 0.012s	sys 0.000s
I20260709 12:17:48.934041 30827 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:17:48.934679 30732 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:48.941139 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 70853.7us
I20260709 12:17:48.894542 30810 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:48.941767 30842 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:17:48.946177 30749 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:48.947074 30723 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:48.946761 30826 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.948727 30734 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:48.960073 30734 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.926687 30840 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:48.964562 30826 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:48.908720 30809 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:48.957381 30642 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:48.966536 30809 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.967459 30826 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:48.968849 30809 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:48.968961 30642 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:48.828792 30752 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.958720 30627 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:48.960171 30765 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:48.943220 30785 consensus_meta.cc:287] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.858s	user 0.007s	sys 0.005s
I20260709 12:17:48.990808 30785 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.991245 30732 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:48.955978 30723 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:48.993239 30785 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:48.992908 30723 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:48.994421 30828 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:48.995162 28361 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:48.995746 30595 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.870s	user 0.008s	sys 0.004s
I20260709 12:17:48.977924 30850 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:17:48.996755 30850 consensus_queue.cc:1243] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:48.996379 30595 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.997906 30627 consensus_queue.cc:1243] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:48.998816 30765 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.000932 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1592) took 1001 ms. Trace:
I20260709 12:17:49.001358 28285 rpcz_store.cc:276] 0709 12:17:47.999012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.193676 (+194664us) service_pool.cc:224] Handling call
0709 12:17:48.194063 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:48.374857 (+180794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:48.374891 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:48.388918 (+ 14027us) log.cc:844] Serialized 52 byte log entry
0709 12:17:48.765895 (+376977us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:48.874231 (+108336us) log.cc:844] Serialized 14 byte log entry
0709 12:17:48.881608 (+  7377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.881697 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:49.000657 (+118960us) raft_consensus.cc:1685] finished
0709 12:17:49.000680 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.000858 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15270,"prepare.queue_time_us":108366,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":101714,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":43346,"thread_start_us":469439,"threads_started":2,"wal-append.queue_time_us":555144}
I20260709 12:17:48.952785 28362 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
W20260709 12:17:48.979507 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 276682us
I20260709 12:17:49.004070 28360 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:48.981211 30564 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.980960 30799 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.005995 30564 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:48.956666 30722 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:17:48.969156 30811 raft_consensus.cc:993] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:17:49.018486 28438 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:48.990264 30839 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.223s	user 0.003s	sys 0.001s
I20260709 12:17:49.027072 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.027788 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1585) took 1103 ms. Trace:
I20260709 12:17:48.936590 30824 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:17:49.028484 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.029759 30799 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.028203 28286 rpcz_store.cc:276] 0709 12:17:47.924338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.165443 (+241105us) service_pool.cc:224] Handling call
0709 12:17:48.204758 (+ 39315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:48.285713 (+ 80955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:48.285768 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:48.293656 (+  7888us) log.cc:844] Serialized 52 byte log entry
0709 12:17:48.557253 (+263597us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:48.632918 (+ 75665us) log.cc:844] Serialized 14 byte log entry
0709 12:17:48.633541 (+   623us) log.cc:844] Serialized 14 byte log entry
0709 12:17:48.647225 (+ 13684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.647447 (+   222us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:48.701409 (+ 53962us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b4800080358. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c37b0aa1 00007f85c37a7b37 00007f85c37a79ca 00007f85c389f26f 00007f85c389ef37 00007f85c38b355c 00007f85c38b3508 00007f85c38b3488 00007f85c38b3443 00007f85c38b26ec 00007f85c27dca96
0709 12:17:48.967305 (+265896us) spinlock_profiling.cc:243] Waited 76.9 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:49.027567 (+ 60262us) raft_consensus.cc:1685] finished
0709 12:17:49.027589 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.027725 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26182,"prepare.queue_time_us":125518,"prepare.run_cpu_time_us":143,"prepare.run_wall_time_us":12823,"raft.queue_time_us":114779,"raft.run_cpu_time_us":3087,"raft.run_wall_time_us":408135,"spinlock_wait_cycles":247834880,"thread_start_us":221428,"threads_started":1,"wal-append.queue_time_us":451456}
I20260709 12:17:49.031975 30799 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:49.023217 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:49.035827 28287 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1576) took 1138 ms (client timeout 1501 ms). Trace:
I20260709 12:17:49.036186 30800 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.037171 28982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 428) took 1381 ms. Trace:
I20260709 12:17:49.037599 28982 rpcz_store.cc:276] 0709 12:17:47.655864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.662691 (+  6827us) service_pool.cc:224] Handling call
0709 12:17:47.709059 (+ 46368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:48.205045 (+495986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.036920 (+831875us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400762d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:49.037107 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837756160}
I20260709 12:17:48.944808 30760 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:49.038470 28290 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.012497 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.013638 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.018766 28436 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:49.025216 30478 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.037118 30716 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.042397 28359 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:49.042559 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.043249 28436 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:49.021693 28435 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:49.044852 30800 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:49.044688 30722 raft_consensus.cc:1081] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:17:49.021525 30564 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.022866 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.023264 28432 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:49.023757 28434 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:49.025038 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.011678 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:49.040190 28287 rpcz_store.cc:269] 0709 12:17:47.897052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.897176 (+   124us) service_pool.cc:224] Handling call
0709 12:17:49.035631 (+1138455us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540035f7d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:49.035754 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303317888}
I20260709 12:17:49.041141 28290 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.038091 30850 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.022394 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.041242 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.050269 28440 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:17:49.041167 28441 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:49.051627 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.052212 28438 raft_consensus.cc:2393] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:49.051398 30850 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.020679 28363 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:49.053660 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.062819 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.034551 30752 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:49.073243 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 61650.6us
I20260709 12:17:49.074445 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.078809 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.080160 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:49.082564 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 158827us
I20260709 12:17:49.081498 28290 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:49.085812 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 259) took 1154 ms. Trace:
I20260709 12:17:49.089888 28438 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:49.086088 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:49.084571 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:49.040269 30785 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:17:49.092372 28255 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:49.088868 28290 rpcz_store.cc:276] 0709 12:17:47.931859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.165509 (+233650us) service_pool.cc:224] Handling call
0709 12:17:48.165859 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:48.703327 (+537468us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400282420. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7fedc0 00007f85be8891c0 00007f85be88a231 00007f85be886d8f 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c
0709 12:17:49.038122 (+334795us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b5400282420. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be80506f 00007f85be87df60 00007f85be87f7de 00007f85be889b49 00007f85be88a416 00007f85be886d8f 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62
0709 12:17:49.085448 (+ 47326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.085715 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618514432}
I20260709 12:17:49.059047 30478 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.050944 30587 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.099869 28286 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:49.102855 28289 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:49.111881 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.122694 28437 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:49.124516 28287 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:49.064654 30671 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:17:49.086182 30809 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.126139 28438 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:49.130601 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:49.131269 28290 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:49.083263 30843 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:49.086524 30800 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.091500 30675 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.091727 30799 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.042575 30716 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.137066 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.095906 30824 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:49.065796 30838 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.093235 30614 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
W20260709 12:17:49.140317 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 48024.3us
I20260709 12:17:49.140486 28434 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:49.138947 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:49.149981 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 65491.6us
I20260709 12:17:49.093688 30632 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.093451 30488 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:49.146313 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.154502 30488 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.127209 30852 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:17:49.139755 30675 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.147370 30800 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.137307 30843 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.150843 30799 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.147150 30820 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:49.158679 28283 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:49.159696 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.160778 28363 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:49.191386 28363 pending_rounds.cc:85] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 1
I20260709 12:17:49.191699 28363 pending_rounds.cc:107] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:17:49.161347 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.162125 30785 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:49.163138 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:49.163918 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.166263 30478 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.169775 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:49.218590 28286 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:49.136217 30752 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.130460 30809 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:49.156880 30632 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.179220 28437 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:49.189435 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.191949 28363 raft_consensus.cc:2889] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:49.219825 28358 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:49.170914 30842 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.244875 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:49.247321 28290 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:49.247788 28285 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:49.248435 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.175088 30675 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.246280 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:49.250130 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 357449us
I20260709 12:17:49.117614 30642 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.168983 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:49.250897 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:49.163187 30760 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.254688 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.175387 30824 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:49.256415 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.258373 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.259654 28290 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59778
I20260709 12:17:49.274634 28290 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:49.275137 28290 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.218920 30595 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.190972 30843 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:49.182516 30817 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:17:49.283803 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:17:49.291778 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.283656 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:49.291849 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.292871 28283 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
W20260709 12:17:49.284313 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 92880.5us
I20260709 12:17:49.265014 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:49.242847 30809 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.289718 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.295019 28292 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.302384 28290 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.279587 30829 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:49.274648 30760 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:49.257108 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 89724.9us
I20260709 12:17:49.236624 30746 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:17:49.278366 30827 raft_consensus.cc:2764] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:17:49.237749 30478 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.287868 30828 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.323844 28290 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.283093 30810 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:17:49.243991 30812 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:17:49.273926 30842 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.286913 30749 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:17:49.339694 28365 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:49.322527 30760 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.337016 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:49.330394 28985 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:17:49.283418 30752 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:49.352375 30855 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:17:49.340941 30820 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:49.351562 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.295125 30739 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:17:49.329540 30828 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:49.386369 30838 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:49.375229 30615 consensus_meta.cc:287] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.622s	user 0.012s	sys 0.000s
I20260709 12:17:49.342908 30850 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.347505 30827 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:17:49.415463 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 118693us
I20260709 12:17:49.374163 30828 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.389215 30837 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:17:49.419082 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:49.418921 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 178456us
I20260709 12:17:49.394325 30587 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:49.388548 30615 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.362871 30785 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 12:17:49.429067 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:49.431352 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 110792us
I20260709 12:17:49.436530 30716 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.438040 30716 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:49.439576 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.440048 30765 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:49.440493 30765 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:49.412292 30866 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:49.420063 30812 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.461822 28367 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:49.425997 30800 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.441520 30765 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.354908 30824 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.486652 30820 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.488677 30478 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.490415 30752 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:49.463997 30812 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.501000 28292 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:49.459913 30866 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.464475 30799 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:49.511966 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 88328.2us
I20260709 12:17:49.482331 30488 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.484479 30632 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.485890 30615 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.521404 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:17:49.529079 30840 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.567s	user 0.000s	sys 0.008s
I20260709 12:17:49.530694 30867 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:49.531646 30675 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.534004 30614 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:49.540752 28362 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:49.545562 28362 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:49.507666 30866 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:49.529202 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:49.551172 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 134818us
I20260709 12:17:49.552143 28257 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:49.560446 28362 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.561551 30675 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:49.566730 30739 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:49.567313 30739 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.567740 30739 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.568753 30842 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.570204 28287 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:49.560359 28986 raft_consensus.cc:2393] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:49.561826 28362 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.546092 30840 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:49.567443 30869 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:49.542461 30867 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:49.590023 28255 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:49.561657 30870 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.549431 30615 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.631927 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:49.544735 30817 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:17:49.647325 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:49.638621 30885 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.639169 30871 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:17:49.646822 30884 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:17:49.646944 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:49.646951 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:49.649714 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:49.591777 30829 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:49.656661 28362 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
W20260709 12:17:49.639565 30858 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.137s	user 0.000s	sys 0.004s
W20260709 12:17:49.662343 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.666360 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:49.669566 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 116104us
I20260709 12:17:49.670615 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:49.674664 30878 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:49.674944 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 101637us
I20260709 12:17:49.626377 30488 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:17:49.654105 30842 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:49.624054 30760 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.659896 30871 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:49.681331 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:49.682999 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.650231 30615 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.689904 30892 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:49.690187 30738 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:49.690717 30738 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:49.679983 30632 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:49.691712 30738 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.692718 28358 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:49.688503 30488 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:49.694835 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:49.695539 28358 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.677372 30878 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.698215 30878 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:49.661440 30885 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.683800 30842 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.703253 30867 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:49.704312 30860 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 12:17:49.704672 30867 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.706625 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.708088 28288 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.706411 30827 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.701444 30885 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.713930 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.714082 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.716339 30749 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.716773 30837 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:49.717267 30587 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:49.718923 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.719033 30760 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.716364 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.720361 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.737888 28435 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:49.727752 30889 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:49.727952 30723 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:49.728879 28440 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:17:49.729097 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 224429us
I20260709 12:17:49.729851 30827 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.684978 30828 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.730957 30622 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:49.731539 30886 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:17:49.732451 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.685180 30820 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:49.740037 28358 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:49.756456 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.757362 28358 raft_consensus.cc:2410] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:49.650419 30870 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.752075 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.758221 28362 raft_consensus.cc:2368] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:49.694473 30871 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.713289 30892 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:49.783329 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44038.1us
I20260709 12:17:49.781903 28288 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:49.784893 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:49.787195 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:49.793361 28284 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:49.737012 30716 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.793635 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1642) took 1004 ms. Trace:
I20260709 12:17:49.794356 28360 rpcz_store.cc:276] 0709 12:17:48.789587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.790052 (+   465us) service_pool.cc:224] Handling call
0709 12:17:49.003836 (+213784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.004510 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.793452 (+788942us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54000a7150. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:49.793568 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490878848}
I20260709 12:17:49.701309 30843 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.797188 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.797890 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:49.798849 28255 leader_election.cc:400] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:49.799188 28255 leader_election.cc:403] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:49.798076 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:49.799955 28440 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:49.795559 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.799289 28435 raft_consensus.cc:2410] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:49.796650 30871 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.809846 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.795504 30760 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:49.810642 28362 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:49.755337 30587 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.816449 28285 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:17:49.756070 30749 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:49.818660 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:49.820876 28285 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:49.809383 30836 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:49.822346 30856 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:17:49.822557 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.823457 30881 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:17:49.823904 28290 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:49.824311 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.825214 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.823251 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.825176 28435 raft_consensus.cc:2410] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:49.825896 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:49.826766 28362 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:49.826419 28440 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:17:49.822984 30853 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:49.830322 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:49.829541 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.798215 30838 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:17:49.807500 30642 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:17:49.824138 30861 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:49.823118 30856 consensus_queue.cc:1243] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:49.834554 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.774739 30829 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.800169 30716 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.835716 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.841576 30891 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:17:49.842849 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:49.843729 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:49.801967 30817 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6.
I20260709 12:17:49.896788 30827 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:49.845288 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.814225 30760 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:49.898867 30809 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.000s
I20260709 12:17:49.899217 30760 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.899382 30809 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.900535 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:49.847139 28439 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:17:49.860072 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.901108 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.901396 30836 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.830732 30853 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:49.903028 28435 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:49.904021 28366 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:49.878216 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:49.832985 30896 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.878824 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:49.905696 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.879240 30843 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:49.895226 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:49.895650 30615 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:49.910292 28362 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:49.792680 30892 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:49.905853 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 256215us
I20260709 12:17:49.908844 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:49.890177 30898 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:17:49.834489 30861 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:49.902150 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:49.915118 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:49.916699 28988 raft_consensus.cc:1813] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:49.919781 28988 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:17:49.915899 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.920385 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.887490 30820 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:49.919469 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.923763 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.759895 30889 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.926283 28435 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:17:49.925190 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:49.927680 28435 pending_rounds.cc:85] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 3
I20260709 12:17:49.926522 28330 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.928303 28435 pending_rounds.cc:107] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:17:49.928675 28435 raft_consensus.cc:2889] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:49.928756 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:49.923506 28989 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.931716 30851 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:49.932060 30895 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:17:49.895283 28986 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:49.901957 30836 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.935642 28285 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:49.936902 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:49.916599 30810 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:17:49.914577 30747 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:17:49.935309 30881 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:49.945940 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:49.946866 28290 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:49.946123 30881 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:49.938452 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.927764 30752 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:17:49.949301 30906 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:49.928272 30820 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:49.927600 30885 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:49.925671 30843 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:49.926069 30615 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:49.950871 28362 raft_consensus.cc:1813] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:49.916129 30896 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.951753 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.933630 30900 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:17:49.832526 30837 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.954804 30828 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:49.955202 30892 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:49.959283 28436 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:49.935760 30840 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:49.962071 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:49.967433 30615 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:49.983608 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.963290 28362 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:17:49.986701 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:49.987335 28362 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:49.989925 30867 raft_consensus.cc:2764] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:17:49.985445 28989 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:17:49.991506 30587 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:49.950609 30675 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:49.992396 30675 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:49.991149 30836 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:49.990609 30749 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:49.993906 30820 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.993500 30675 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:49.994277 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:49.995564 30622 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:49.949782 28983 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:17:49.953553 30809 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:49.952409 30881 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.976120 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:49.977613 30716 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:49.978612 30827 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:49.979169 30880 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:49.949697 30853 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:17:49.980509 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.981904 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1636) took 1202 ms. Trace:
I20260709 12:17:49.990592 30867 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:49.949688 30870 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:49.991086 30851 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:49.945992 30888 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:17:49.995549 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:50.013882 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1632) took 1240 ms. Trace:
I20260709 12:17:49.935304 30739 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.045497 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:50.043987 28367 rpcz_store.cc:276] 0709 12:17:48.779633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.779730 (+    97us) service_pool.cc:224] Handling call
0709 12:17:48.780003 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.462418 (+682415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.981551 (+519133us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54003700d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:49.981821 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1567527808}
I20260709 12:17:50.071002 28289 raft_consensus.cc:1813] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:50.071455 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 75993.1us
I20260709 12:17:50.071714 28289 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 3: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:50.069835 28359 rpcz_store.cc:276] 0709 12:17:48.773073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.773857 (+   784us) service_pool.cc:224] Handling call
0709 12:17:48.774172 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.044887 (+270715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.013544 (+968657us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b540027e554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:50.013804 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616439168}
I20260709 12:17:50.075368 28367 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
W20260709 12:17:50.076694 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 176223us
I20260709 12:17:50.083307 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:50.078593 30889 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:50.077278 30870 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.086539 28291 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:50.086192 30870 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.088476 30870 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:50.081136 30853 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.075242 30622 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:50.082155 28367 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:50.069700 30866 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.095873 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:50.101951 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.024492 30827 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:50.110203 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:50.077466 30747 consensus_queue.cc:1175] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a3de34b6f4cf4606a61944db6e2739c8" responder_term: 5 status { last_received { term: 5 index: 4 } last_committed_idx: 3 last_received_current_leader { term: 5 index: 4 } }
I20260709 12:17:50.117038 30836 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.100260 30906 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:50.130070 28360 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:50.121013 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:50.100703 30749 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:50.122012 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1495) took 1896 ms. Trace:
I20260709 12:17:50.133647 28361 rpcz_store.cc:276] 0709 12:17:48.225934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.226100 (+   166us) service_pool.cc:224] Handling call
0709 12:17:48.226421 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:48.995590 (+769169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.121697 (+1126107us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54001d7c54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:50.121918 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1767474944}
I20260709 12:17:50.132853 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:50.135020 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.138017 28361 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:50.110865 30889 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:50.123420 30488 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:50.145695 28982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 437) took 1021 ms. Trace:
I20260709 12:17:50.146440 28982 rpcz_store.cc:276] 0709 12:17:49.124340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.130143 (+  5803us) service_pool.cc:224] Handling call
0709 12:17:50.120801 (+990658us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:50.145616 (+ 24815us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39428864}
I20260709 12:17:50.147203 28983 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:50.123484 30760 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.156971 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:50.157941 28292 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:50.158245 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:50.128794 30906 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.162192 28435 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:50.164053 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:50.166169 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:50.167038 28289 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:50.158861 30871 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:50.139401 30836 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:17:50.168902 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:50.173261 28289 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:50.182823 28285 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:17:50.172173 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:50.148717 30749 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:17:50.141633 30900 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:17:50.149564 30887 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:50.164130 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:50.190265 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 261579us
W20260709 12:17:50.168450 30907 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:17:50.149524 30889 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.180981 30885 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:50.201543 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:50.207973 28435 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:50.162715 30879 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:50.218189 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 54152.1us
I20260709 12:17:50.219244 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:50.254961 30879 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:50.256431 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.259956 28991 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:50.201061 30887 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:50.159468 30827 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.185688 30871 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:50.257824 30875 log.cc:927] Time spent T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:17:50.266238 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:50.256183 30870 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:50.260186 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:50.296157 28367 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
W20260709 12:17:50.265620 30911 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:17:50.296736 28990 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:50.270735 30889 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.277915 30829 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:50.298514 30870 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.305662 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.307158 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.307823 28363 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:50.309005 30914 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:17:50.269888 30887 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.299315 30900 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:50.311199 30900 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:50.311305 30812 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.312788 30879 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:50.317948 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:50.302750 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:50.323177 28991 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.321908 30885 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.326570 30881 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.313694 30829 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:50.329417 30829 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:50.365515 28255 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.4:39985 timed out after 1.926s (SENT)
I20260709 12:17:50.335680 28991 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
W20260709 12:17:50.360970 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:50.362453 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:50.409137 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 106497us
I20260709 12:17:50.386652 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1519) took 1091 ms. Trace:
I20260709 12:17:50.409763 28360 rpcz_store.cc:276] 0709 12:17:49.295687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.795272 (+499585us) service_pool.cc:224] Handling call
0709 12:17:50.386401 (+591129us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b5400054954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:50.386568 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756009856}
I20260709 12:17:50.411175 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:50.411507 28360 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:50.396230 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1515) took 1235 ms. Trace:
I20260709 12:17:50.412034 28292 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:17:50.412667 28365 rpcz_store.cc:276] 0709 12:17:49.160983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.161106 (+   123us) service_pool.cc:224] Handling call
0709 12:17:50.396005 (+1234899us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b540036c4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:50.396152 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338375936}
I20260709 12:17:50.414423 28365 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:50.415757 28365 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:50.417872 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 155196us
I20260709 12:17:50.418289 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:50.416927 28365 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:50.421310 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:50.423792 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:50.299561 30866 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.420552 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:50.424690 28292 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:50.432155 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:50.433609 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:50.408934 30685 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.390312 30760 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:50.421036 30915 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.127s	user 0.001s	sys 0.002s
I20260709 12:17:50.400243 30870 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:50.444350 28365 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:50.400363 30879 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:17:50.377626 30887 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.332976 30615 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:50.384936 30871 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.437588 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1521) took 1082 ms. Trace:
I20260709 12:17:50.456588 28362 rpcz_store.cc:276] 0709 12:17:49.355203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.987914 (+632711us) service_pool.cc:224] Handling call
0709 12:17:49.988264 (+   350us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:49.988723 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.988751 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:49.989399 (+   648us) log.cc:844] Serialized 52 byte log entry
0709 12:17:50.164963 (+175564us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:50.165006 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.165085 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:50.437303 (+272218us) raft_consensus.cc:1685] finished
0709 12:17:50.437329 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.437504 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3147,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":175132,"threads_started":1,"wal-append.queue_time_us":228866}
I20260709 12:17:50.418242 30914 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:50.459033 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:50.460114 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.436882 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:50.392988 28984 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:50.452355 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:50.435949 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:50.454874 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1565) took 1389 ms. Trace:
I20260709 12:17:50.426082 30810 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:17:50.461843 28436 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.473536 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:50.472555 28434 rpcz_store.cc:276] 0709 12:17:49.065179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.065400 (+   221us) service_pool.cc:224] Handling call
0709 12:17:49.082641 (+ 17241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.149197 (+ 66556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.454583 (+1305386us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b5400348854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:50.454789 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132509440}
I20260709 12:17:50.472362 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 295) took 1234 ms. Trace:
I20260709 12:17:50.475234 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:50.442358 30881 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:50.389657 30900 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:50.481904 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46022us
I20260709 12:17:50.485105 28991 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:17:50.485834 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:50.476435 30893 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.004s	sys 0.013s
I20260709 12:17:50.471753 30886 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.474926 28441 rpcz_store.cc:276] 0709 12:17:49.237816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.238952 (+  1136us) service_pool.cc:224] Handling call
0709 12:17:49.239313 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.239574 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:49.283001 (+ 43427us) log.cc:844] Serialized 14 byte log entry
0709 12:17:49.304140 (+ 21139us) log.cc:844] Serialized 14 byte log entry
0709 12:17:49.440281 (+136141us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b1800232b40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb7682a6 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1
0709 12:17:49.482123 (+ 41842us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b1800232b40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:49.690281 (+208158us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b1800232b40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb7682a6 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb7682a6 00007f85bb768240
0709 12:17:49.690756 (+   475us) spinlock_profiling.cc:243] Waited 5.22 ms on lock 0x7b1800232b40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb7682a6 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219
0709 12:17:49.691673 (+   917us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b1800232b40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85c275b58f 00007f85bb79a571 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd
0709 12:17:50.325348 (+633675us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b1800232b40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287
0709 12:17:50.325717 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 12:17:50.470228 (+144511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:50.470277 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:50.470344 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.470424 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.472282 (+  1858us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":280740,"prepare.queue_time_us":214846,"prepare.run_cpu_time_us":154,"prepare.run_wall_time_us":2947,"raft.queue_time_us":227439,"raft.run_cpu_time_us":541,"raft.run_wall_time_us":31842,"spinlock_wait_cycles":229966720,"thread_start_us":223526,"threads_started":3}
I20260709 12:17:50.470745 30892 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:50.478752 30749 consensus_queue.cc:1243] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:17:50.483572 30716 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:50.477509 30888 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:50.491711 30571 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:50.480713 30922 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.511863 28365 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:50.497462 28441 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:50.491170 30893 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.004s	sys 0.014s
I20260709 12:17:50.496699 30925 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.484370 30879 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.521020 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:50.512560 30888 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:50.513866 28362 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 8. Ops: [6.7-6.7]
I20260709 12:17:50.499045 30826 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.539822 28436 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:50.508960 30716 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:50.523643 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.524140 28283 raft_consensus.cc:1217] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Deduplicated request from leader. Original: 4.3->[5.4-5.4]   Dedup: 5.4->[]
I20260709 12:17:50.544091 28283 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:50.512167 30571 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:50.517059 30734 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.507786 30836 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
W20260709 12:17:50.509542 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 144081us
I20260709 12:17:50.553110 28404 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:50.540383 30888 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.566092 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.567579 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:50.568447 28362 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:50.559011 30749 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:17:50.570215 30749 consensus_queue.cc:1243] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:17:50.559150 30923 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:50.546947 30716 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:50.551510 30734 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:50.577984 30868 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:17:50.578902 30488 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.579340 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.579486 30488 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.580915 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1510) took 1656 ms. Trace:
I20260709 12:17:50.590684 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:50.581357 28364 rpcz_store.cc:276] 0709 12:17:48.924627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.926812 (+  2185us) service_pool.cc:224] Handling call
0709 12:17:48.927207 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:48.942395 (+ 15188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:48.942442 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.166912 (+224470us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b2c0000e678. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:17:49.167373 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:17:49.235846 (+ 68473us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:49.280444 (+ 44598us) log.cc:844] Serialized 14 byte log entry
0709 12:17:49.281041 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.281118 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:50.488089 (+1206971us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b18000d01e0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:50.580670 (+ 92581us) raft_consensus.cc:1685] finished
0709 12:17:50.580691 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.580843 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49899,"prepare.queue_time_us":47724,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":3108,"raft.queue_time_us":36229,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":17127,"spinlock_wait_cycles":28922624,"thread_start_us":68081,"threads_started":1,"wal-append.queue_time_us":111629}
I20260709 12:17:50.599118 28364 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:17:50.602393 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.604424 30861 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.614151 30887 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.615142 30571 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.618172 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1709) took 1564 ms. Trace:
I20260709 12:17:50.618674 28432 rpcz_store.cc:276] 0709 12:17:49.054097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.054229 (+   132us) service_pool.cc:224] Handling call
0709 12:17:49.066807 (+ 12578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.067237 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.067267 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.067805 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:17:50.387572 (+1319767us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b180039d660. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:17:50.476205 (+ 88633us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:50.476243 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.476572 (+   329us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:50.617793 (+141221us) raft_consensus.cc:1685] finished
0709 12:17:50.617816 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.618088 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":8178,"spinlock_wait_cycles":30099712,"thread_start_us":88447,"threads_started":1,"wal-append.queue_time_us":1450379}
I20260709 12:17:50.620707 30829 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.622031 28441 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:50.622896 28364 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:50.626579 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.627143 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.634282 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:50.638310 28358 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:50.636224 30627 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:17:50.640399 30927 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:50.642442 28362 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:50.631995 30861 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.631453 30922 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:50.641263 30627 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:50.660677 30627 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:50.661648 30627 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.663015 30627 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.665366 28286 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:17:50.666119 28364 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:50.660336 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:50.648898 30886 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:50.649669 30928 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:50.649910 30933 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:50.651047 30765 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.651566 30716 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.631683 30752 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:17:50.678573 30752 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:50.679109 30716 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:50.680805 30734 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:50.681454 30888 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.682826 30837 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:50.654459 30922 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:50.683336 30837 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:50.657475 30810 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:50.683940 30888 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.638103 30826 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:17:50.658520 30488 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.686333 30899 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:50.641790 30824 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:50.643671 30749 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:17:50.660228 30829 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:50.668931 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.670153 30928 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:50.673635 30842 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.666049 30887 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.702860 30887 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.676158 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:50.676628 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:50.682955 30933 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:50.621721 30925 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:50.686802 30899 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:50.693804 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:50.696192 30870 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:50.699218 30928 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.696297 30716 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.693701 30922 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.700542 30824 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:50.709260 28362 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:50.672132 30770 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:50.671425 30892 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:50.762077 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 68403.6us
I20260709 12:17:50.763335 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:50.693359 30893 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:50.767006 30871 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:50.768945 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:50.770362 30940 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:17:50.771299 28358 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:50.771835 30899 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.778355 30928 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.781903 30870 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.695240 30685 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:50.783901 30734 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:50.676246 30861 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:17:50.739081 30933 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:50.788184 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:50.790110 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1601) took 1628 ms. Trace:
I20260709 12:17:50.787153 30716 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.788087 30892 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.810806 30590 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:50.788179 30770 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:50.800465 28287 rpcz_store.cc:276] 0709 12:17:49.161339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.161467 (+   128us) service_pool.cc:224] Handling call
0709 12:17:49.569826 (+408359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.570605 (+   779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.789884 (+1219279us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b5400332f50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:50.790042 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938436480}
I20260709 12:17:50.811549 30590 consensus_queue.cc:1243] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:50.806396 30869 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:50.796815 30821 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:50.809199 30933 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.826651 30933 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:50.827894 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:50.832702 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.837779 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:50.838536 28432 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:17:50.835490 30781 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:50.842638 30781 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:50.842598 30770 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.843888 30770 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.818427 30899 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:50.796677 30871 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.845394 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:50.818120 30869 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:50.848701 30869 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.825137 30821 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:50.810315 30685 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.827171 30716 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.852053 28363 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:50.853144 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:50.858052 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:50.862607 30887 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:50.864976 30887 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:50.798861 30893 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.843787 30781 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:50.876065 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.877342 30821 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.879925 30821 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.859261 30734 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:17:50.839529 30824 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.886546 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1530) took 1187 ms. Trace:
I20260709 12:17:50.886904 28361 rpcz_store.cc:276] 0709 12:17:49.699039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.143366 (+444327us) service_pool.cc:224] Handling call
0709 12:17:50.159319 (+ 15953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:50.159984 (+   665us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:50.173105 (+ 13121us) log.cc:844] Serialized 14 byte log entry
0709 12:17:50.886057 (+712952us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x7b180040e520. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:17:50.886227 (+   170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:50.886281 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:50.886321 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.886372 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.886490 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12855,"prepare.queue_time_us":24597,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":2639,"spinlock_wait_cycles":14766336}
I20260709 12:17:50.891250 30685 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.892292 30870 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.893662 30871 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.900832 30587 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:50.901360 30587 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:50.901399 30811 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:17:50.882047 30923 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:50.903734 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:50.882230 30893 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:50.902122 30781 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:50.914940 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:50.920770 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:50.915360 30685 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:50.931602 30922 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.923939 30871 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.934993 30893 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:17:50.938902 28985 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:17:50.924789 30627 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:50.942178 30685 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:50.961339 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:50.950202 30716 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.950454 30856 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.960521 30627 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:50.964344 30685 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.953184 30745 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:17:50.923471 30587 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:50.965680 30945 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:50.984375 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 262) took 1341 ms. Trace:
W20260709 12:17:50.978010 30941 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:17:50.975425 30627 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:50.986153 30587 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:50.984733 28288 rpcz_store.cc:276] 0709 12:17:49.643404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.648452 (+  5048us) service_pool.cc:224] Handling call
0709 12:17:50.984218 (+1335766us) spinlock_profiling.cc:243] Waited 40.9 ms on lock 0x7b54000028d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:50.984319 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94159104}
I20260709 12:17:50.988397 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:50.989526 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:50.996111 28984 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:50.997620 28988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 363) took 1117 ms. Trace:
I20260709 12:17:50.970069 30716 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:50.998099 28988 rpcz_store.cc:276] 0709 12:17:49.879991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.880393 (+   402us) service_pool.cc:224] Handling call
0709 12:17:50.997378 (+1116985us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b5400762d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:50.997527 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4095360}
I20260709 12:17:50.999383 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:50.999573 30896 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:51.000237 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.000952 30716 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.001461 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:50.544588 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:51.002671 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:51.003248 28991 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:51.007874 30842 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.008754 30842 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.003165 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.011026 28361 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 8. Ops: [6.7-6.7]
I20260709 12:17:51.005728 30851 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:51.014866 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1712) took 1315 ms. Trace:
I20260709 12:17:51.000574 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:51.018045 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 907903us
I20260709 12:17:51.019034 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:51.015250 28439 rpcz_store.cc:276] 0709 12:17:49.699602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.712152 (+ 12550us) service_pool.cc:224] Handling call
0709 12:17:49.728636 (+ 16484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.904017 (+175381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.014590 (+1110573us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400415354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:51.014765 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271643904}
I20260709 12:17:51.022650 30838 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:51.023965 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:51.027614 30851 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:51.027606 30627 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.028311 30851 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:51.029253 30851 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.030303 30627 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.032771 28365 pending_rounds.cc:85] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 2
I20260709 12:17:51.033160 28365 pending_rounds.cc:107] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 3.3
W20260709 12:17:51.033396 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.033417 28365 raft_consensus.cc:2889] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:51.034116 28361 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:17:51.034956 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:51.018194 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.036659 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.004168 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:51.037652 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.037930 28991 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:17:51.038442 28358 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:51.038779 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:51.039477 28991 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:51.039947 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.040745 28361 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:51.040614 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:50.968540 30856 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.053407 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:51.054404 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.056991 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:51.057319 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:51.057814 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:51.065876 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.066323 28991 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:51.066865 28358 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:51.068058 28991 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:51.063370 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.060093 28330 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:51.061278 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:51.069013 28362 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:17:51.071209 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:51.073030 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:51.056123 30587 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.085249 30810 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:51.063015 30478 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.088837 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:51.088506 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:51.090250 28988 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.093036 28991 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 5 LEARNER]: Advancing to term 7
I20260709 12:17:51.074194 30923 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:51.095914 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:51.058784 30933 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:17:51.079890 30922 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:51.081962 30627 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:51.078714 30615 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:50.984195 30945 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.097095 30886 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:17:51.105449 30812 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.105904 30749 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:17:51.119678 30933 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:51.120503 28984 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
W20260709 12:17:51.121393 28286 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 263) took 1477 ms (client timeout 1810 ms). Trace:
I20260709 12:17:51.131451 30627 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:51.128058 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:51.128824 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:51.128986 28292 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:51.129235 30739 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:51.137496 28991 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 7 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:51.166056 28287 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:51.139606 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.140417 28361 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:51.141508 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:51.094537 30723 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:51.146327 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:51.100353 30770 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:51.135636 28988 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:17:51.125654 30571 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:51.161351 28286 rpcz_store.cc:269] 0709 12:17:49.643785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.646623 (+  2838us) service_pool.cc:224] Handling call
0709 12:17:51.121205 (+1474582us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003daed4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:51.121322 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2318769536}
I20260709 12:17:51.123258 30836 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:51.099351 30478 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.098918 30861 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:17:51.098872 30924 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:17:51.165570 28288 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:51.167043 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:51.171657 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:51.173383 28360 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:51.201036 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:51.145728 30745 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:17:51.203052 30836 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:51.218041 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:51.219007 30587 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:51.219425 30478 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:51.182156 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.203740 28287 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:51.223724 28288 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:51.181175 30770 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:51.277493 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.205366 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:51.212798 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:51.213531 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.279619 28991 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:51.214452 30821 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:51.214717 30900 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:51.176203 30739 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:51.176391 30765 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:17:51.219427 30587 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:51.228847 28437 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:17:51.230890 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:51.231308 28190 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 114) took 13505 ms (client timeout 15000 ms). Trace:
I20260709 12:17:51.236872 30869 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.238262 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.277400 30770 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.166800 30922 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:51.166994 30627 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.240267 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:51.241571 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1716) took 1540 ms. Trace:
I20260709 12:17:51.302317 28433 rpcz_store.cc:276] 0709 12:17:49.700646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.721534 (+ 20888us) service_pool.cc:224] Handling call
0709 12:17:49.759351 (+ 37817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.787713 (+ 28362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.787750 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.802601 (+ 14851us) log.cc:844] Serialized 26 byte log entry
0709 12:17:49.880943 (+ 78342us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:49.880995 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.241085 (+1360090us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b5400085a54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:51.241136 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:51.241187 (+    51us) raft_consensus.cc:1685] finished
0709 12:17:51.241202 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.241489 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":335,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":5781,"spinlock_wait_cycles":64264576,"thread_start_us":53368,"threads_started":1,"wal-append.queue_time_us":160029}
I20260709 12:17:51.242076 30842 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.242549 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1645) took 1428 ms. Trace:
I20260709 12:17:51.311372 28363 rpcz_store.cc:276] 0709 12:17:49.814428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.308408 (+493980us) service_pool.cc:224] Handling call
0709 12:17:50.308720 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:50.852603 (+543883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.242296 (+389693us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54003bc4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:51.242480 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1249130368}
W20260709 12:17:51.242884 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.243206 30760 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:51.244611 30886 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:51.245774 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet e2ac8850682043e5bf8caf7fc64fae05 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33090
I20260709 12:17:51.247447 30836 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.248167 30892 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.331949 28986 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:51.333101 30892 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.333590 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 289) took 1130 ms. Trace:
I20260709 12:17:51.334132 28986 rpcz_store.cc:276] 0709 12:17:50.203565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.204177 (+   612us) service_pool.cc:224] Handling call
0709 12:17:50.851460 (+647283us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:51.333317 (+481857us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b540071e654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:51.333502 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177383040}
I20260709 12:17:51.336027 28986 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:51.251406 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.200335 30615 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:17:51.254262 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 265) took 1609 ms. Trace:
I20260709 12:17:51.254997 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1645) took 1336 ms. Trace:
I20260709 12:17:51.338034 28285 rpcz_store.cc:276] 0709 12:17:49.918902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.919192 (+   290us) service_pool.cc:224] Handling call
0709 12:17:51.254826 (+1335634us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b54000f2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:51.254933 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24518016}
I20260709 12:17:51.255980 30837 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.257815 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:51.261268 30685 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:51.262553 30948 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:17:51.229513 30871 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:51.161432 30933 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:51.272679 30571 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:51.272856 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.273077 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.273665 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.274531 30945 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.274873 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.275128 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:51.275485 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.275494 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:51.285481 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.286603 28988 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:51.289386 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 59693.4us
I20260709 12:17:51.352849 28436 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:51.292143 28441 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:17:51.356856 28441 pending_rounds.cc:85] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 3
I20260709 12:17:51.357167 28441 pending_rounds.cc:107] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:17:51.357210 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:51.358688 28360 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:51.181766 30856 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:51.282580 30827 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.282397 30781 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:51.282431 30821 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:51.289072 28190 rpcz_store.cc:269] 0709 12:17:37.726267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.726412 (+   145us) service_pool.cc:224] Handling call
0709 12:17:51.231210 (+13504798us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:48.374964 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:48.456293 (+ 81329us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:48.456334 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:17:48.456346 (+    12us) tablet.cc:662] Decoding operations
0709 12:17:48.457247 (+   901us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:48.457316 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:48.457341 (+    25us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:17:48.457886 (+   545us) tablet.cc:701] Row locks acquired
0709 12:17:48.457907 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:17:48.458049 (+   142us) write_op.cc:270] Start()
0709 12:17:48.458162 (+   113us) write_op.cc:276] Timestamp: P: 1783599468457999 usec, L: 0
0709 12:17:48.458184 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:17:48.677581 (+219397us) log.cc:844] Serialized 2814 byte log entry
0709 12:17:49.022432 (+344851us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85bb77d0ac 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b
0709 12:17:49.754489 (+732057us) op_driver.cc:464] REPLICATION: finished
0709 12:17:49.969965 (+215476us) write_op.cc:301] APPLY: starting
0709 12:17:49.970055 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:49.970240 (+   185us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:49.970260 (+    20us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:49.977341 (+  7081us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:49.977583 (+   242us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:17:49.977629 (+    46us) write_op.cc:312] APPLY: finished
0709 12:17:49.977955 (+   326us) log.cc:844] Serialized 59 byte log entry
0709 12:17:49.978160 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:49.978311 (+   151us) write_op.cc:454] Released schema lock
0709 12:17:49.978366 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:50.697764 (+719398us) spinlock_profiling.cc:243] Waited 60.5 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be882e28 00007f85c22bd581 00007f85c22bcd58 00007f85c22c19f1 00007f85c22c19a9 00007f85c22c1939 00007f85c22c1901 00007f85c22c0b2d 0000000000388634 0000000000388479
0709 12:17:50.698002 (+   238us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:50.698034 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":58861,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":215132,"apply.run_cpu_time_us":3291,"apply.run_wall_time_us":8822,"num_ops":7,"prepare.queue_time_us":480070,"prepare.run_cpu_time_us":78963,"prepare.run_wall_time_us":2339036,"raft.queue_time_us":78957,"raft.run_cpu_time_us":24757,"raft.run_wall_time_us":77352,"replication_time_us":1296107,"spinlock_wait_cycles":219659008,"thread_start_us":837311,"threads_started":4,"wal-append.queue_time_us":691549,"wal-append.run_cpu_time_us":14152,"wal-append.run_wall_time_us":1593191}]]}
I20260709 12:17:51.369824 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.299639 30922 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:51.316320 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 76148.9us
I20260709 12:17:51.325310 30760 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:51.325201 30886 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.326244 30587 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.328655 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:51.336650 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.376464 28433 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.337404 28291 rpcz_store.cc:276] 0709 12:17:49.644367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.647047 (+  2680us) service_pool.cc:224] Handling call
0709 12:17:51.254023 (+1606976us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b540006fc54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:51.254181 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007645568}
I20260709 12:17:51.298512 30899 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:51.342720 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 295) took 1076 ms. Trace:
I20260709 12:17:51.333899 30938 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:51.349568 28987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 291) took 1093 ms. Trace:
I20260709 12:17:51.350118 28439 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:51.350915 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1518) took 2182 ms. Trace:
I20260709 12:17:51.350952 28435 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:51.394022 28433 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:51.359416 28441 raft_consensus.cc:2889] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:51.357609 28438 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:51.394057 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:51.395058 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:51.395289 30889 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:51.395725 28433 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.395843 30889 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:51.398478 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.367395 28982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 287) took 1158 ms. Trace:
I20260709 12:17:51.398640 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.395187 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:51.400902 28438 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:51.401762 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.402483 28438 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:51.403235 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.403923 28438 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:17:51.404294 30949 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.349s	user 0.001s	sys 0.001s
I20260709 12:17:51.405014 28433 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:51.406133 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:51.406729 28439 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.407073 28982 rpcz_store.cc:276] 0709 12:17:50.202897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.204115 (+  1218us) service_pool.cc:224] Handling call
0709 12:17:51.275005 (+1070890us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762eee 000000000036eb39 00007f85c37d61e0 00007f85c389f0c4 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:51.361361 (+ 86356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79665536}
I20260709 12:17:51.407783 30836 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.408500 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:51.408967 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:51.406148 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.409957 28438 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:51.411578 28438 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:51.412189 30899 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:51.413021 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:51.413615 28363 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.413180 30899 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.413975 30760 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.422384 28363 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:51.394081 28366 rpcz_store.cc:276] 0709 12:17:49.168497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.217686 (+ 49189us) service_pool.cc:224] Handling call
0709 12:17:49.218070 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:49.906763 (+688693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.350639 (+1443876us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b5400078d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:51.350844 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577039232}
I20260709 12:17:51.426944 28439 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:17:51.430313 28366 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:51.363704 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:51.431522 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.346254 30739 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.347380 30842 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.370736 30869 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.371021 28432 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:51.370994 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:51.456993 30892 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:51.456717 30739 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:51.457134 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 86214.2us
I20260709 12:17:51.378860 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.457595 30892 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:51.458047 28436 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:51.366851 30828 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:17:51.458596 30892 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.459122 30739 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.380182 30923 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 12:17:51.377521 28985 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 360) took 1549 ms (client timeout 1921 ms). Trace:
W20260709 12:17:51.460497 28985 rpcz_store.cc:269] 0709 12:17:49.827649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.829176 (+  1527us) service_pool.cc:224] Handling call
0709 12:17:51.369544 (+1540368us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b54001caf54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:51.375196 (+  5652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046560128}
I20260709 12:17:51.461763 30922 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.462662 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:51.463093 30922 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.465062 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.361584 30925 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.386121 28990 rpcz_store.cc:276] 0709 12:17:50.265897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.300835 (+ 34938us) service_pool.cc:224] Handling call
0709 12:17:51.257602 (+956767us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:17:51.342645 (+ 85043us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41796480}
I20260709 12:17:51.424319 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:51.372596 30821 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:51.429306 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:51.434108 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46456.8us
I20260709 12:17:51.470120 28360 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:51.437233 28362 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:51.373992 28989 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:51.349529 30837 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:51.440907 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46957.3us
I20260709 12:17:51.394982 30938 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:51.443497 30886 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:51.478274 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.446167 30824 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:51.445058 30770 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.449923 30842 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:51.450711 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.373986 30893 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.452325 30942 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:51.453986 30829 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:51.454751 30587 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.398722 28983 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.372476 30781 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.459604 30685 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.475688 28362 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:51.511531 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.476544 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:51.481278 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.422184 28987 rpcz_store.cc:276] 0709 12:17:50.256754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.256985 (+   231us) service_pool.cc:224] Handling call
0709 12:17:50.996288 (+739303us) spinlock_profiling.cc:243] Waited 53.5 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:50.996495 (+   207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:50.996726 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:51.099673 (+102947us) log.cc:844] Serialized 14 byte log entry
0709 12:17:51.349228 (+249555us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:51.349282 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:51.349325 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.349390 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.349506 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":58170,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":6180,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":12815,"spinlock_wait_cycles":123084160,"thread_start_us":342265,"threads_started":2,"wal-append.queue_time_us":295653}
I20260709 12:17:51.444785 30896 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:51.490278 28433 raft_consensus.cc:2410] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:51.503103 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:51.501086 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:51.488245 30938 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:51.512293 28362 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:51.524051 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.525071 28362 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 3.
I20260709 12:17:51.520413 28366 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:51.526996 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.488452 30564 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:17:51.541147 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.509020 30829 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:51.546144 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:51.491499 30944 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:17:51.553918 28362 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:17:51.523288 30910 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:51.572494 28366 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:51.523056 30861 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:51.577500 28362 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:51.508004 30893 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.553102 28984 raft_consensus.cc:2368] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
W20260709 12:17:51.609513 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.519500 30886 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 12:17:51.624172 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 78935.1us
W20260709 12:17:51.621567 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 127274us
I20260709 12:17:51.622336 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:51.626370 28285 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:51.551589 30866 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:51.629808 30861 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.623119 30478 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:17:51.625669 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:51.633365 30961 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 12:17:51.633894 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.634681 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:51.578819 30627 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:51.638434 30817 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:51.638844 30817 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:51.639195 30817 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.640249 30627 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.639879 30821 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.640916 30627 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:51.584784 30899 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:51.626773 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:51.642169 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 142343us
I20260709 12:17:51.552184 30734 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 12:17:51.698586 28362 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:51.643013 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:51.700759 28362 raft_consensus.cc:606] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:51.651932 30965 tablet_replica.cc:333] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
W20260709 12:17:51.706151 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 79468.4us
I20260709 12:17:51.615929 30910 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:51.645088 30853 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:51.709075 30812 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:51.709226 30781 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:51.706984 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:51.708505 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:51.585058 30838 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 12:17:51.623703 30871 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:51.712831 30866 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:51.713629 30971 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:17:51.733036 30853 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:51.733435 30824 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.734090 30853 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.734642 30871 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.692737 30811 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 12:17:51.736001 30811 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:51.736385 30811 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:51.736773 30871 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.713994 30829 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.738256 30829 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.693492 30899 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.711051 30723 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.740746 30723 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.737282 30811 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.715039 30889 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.716310 30893 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.718380 30842 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:51.703575 30739 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:51.720870 30910 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:51.726464 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 83543.3us
W20260709 12:17:51.759457 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.723606 30880 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.771713 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:51.783685 28362 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:51.796190 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.796669 28365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:51.797349 28365 raft_consensus.cc:606] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:51.797345 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:51.797989 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:51.799008 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.797998 28365 raft_consensus.cc:614] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:51.800957 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:51.801589 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.801671 28362 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:17:51.803299 28367 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:51.794072 30880 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.813246 30880 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.816545 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:51.819872 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:51.824568 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.764623 30781 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.777879 30811 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.771616 30760 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:51.844744 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.847540 28440 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:17:51.849193 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:51.849476 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 51575.2us
I20260709 12:17:51.851227 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:51.850188 30880 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:51.815511 30893 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:51.860375 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:51.838996 30886 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.842376 30891 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.862768 28440 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.845220 30944 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.844951 30827 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.855551 30871 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:51.847275 30760 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:51.873482 30975 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:51.873991 28367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:51.881211 30899 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:51.882124 30827 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.854435 30564 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:51.886812 28367 raft_consensus.cc:606] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:51.898983 28367 raft_consensus.cc:614] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:51.883702 30891 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:51.862385 30880 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:51.890081 30723 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.861791 30770 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:51.869185 30886 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.886852 30866 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.895797 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:51.888542 30760 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.848001 30781 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.884928 30871 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:51.917531 28359 raft_consensus.cc:2376] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4. Re-sending same reply.
I20260709 12:17:51.899783 28440 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:17:51.888087 30975 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:51.920853 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 104435us
I20260709 12:17:51.924178 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:51.924392 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:51.912889 30880 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.930924 28257 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:51.915661 30564 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:51.948378 28361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:51.949103 28361 raft_consensus.cc:606] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:51.921655 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 288) took 1718 ms. Trace:
I20260709 12:17:51.936677 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:51.932165 30871 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:51.934813 30944 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:51.931612 30975 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:51.965934 28435 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:51.902863 30817 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:51.953373 30945 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:51.933013 30770 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:17:51.982838 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.981477 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:51.985975 28361 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:51.988524 28441 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:51.990765 28437 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:17:51.991175 28437 pending_rounds.cc:85] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 3
I20260709 12:17:51.991411 28437 pending_rounds.cc:107] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:17:51.991634 28437 raft_consensus.cc:2889] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:51.997195 28441 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:51.997948 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:51.972187 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:51.961086 30824 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:51.966178 30893 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.002965 30893 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.003899 30571 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:52.004648 30571 consensus_queue.cc:1243] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:17:52.005065 30893 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:51.981860 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.006999 28432 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:51.974167 30922 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.975657 30984 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:51.998850 28440 raft_consensus.cc:2410] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:17:52.001031 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 258950us
I20260709 12:17:51.998874 28437 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:52.016865 28437 pending_rounds.cc:85] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 2
I20260709 12:17:52.017148 28437 pending_rounds.cc:107] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:52.001953 30838 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:52.017395 28437 raft_consensus.cc:2889] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:52.008741 28361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4036b67205e41c597049a10a71e9467"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:17:52.000276 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:52.018754 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:52.014089 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.002187 30851 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:51.981695 30945 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:52.017446 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.023202 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.020215 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:52.020941 28361 raft_consensus.cc:606] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:52.024240 28361 raft_consensus.cc:614] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:52.027454 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:52.027575 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.028083 28285 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.028275 28287 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:52.031560 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.032335 28286 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:52.034307 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:52.036433 30982 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:17:52.036768 30967 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:17:52.015324 30870 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:52.011992 30974 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:52.043968 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.045056 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:52.059414 28287 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:52.045497 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.047043 28432 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:17:52.060482 28432 pending_rounds.cc:85] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 6
I20260709 12:17:52.060719 28432 pending_rounds.cc:107] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:17:52.060925 28432 raft_consensus.cc:2889] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:52.009438 30812 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:52.024734 30770 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:52.065135 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:52.016425 30984 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:52.022533 30893 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:52.025254 30587 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:51.960629 28989 rpcz_store.cc:276] 0709 12:17:50.203211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.204416 (+  1205us) service_pool.cc:224] Handling call
0709 12:17:50.790865 (+586449us) spinlock_profiling.cc:243] Waited 52.7 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:50.798331 (+  7466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:51.472054 (+673723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.905521 (+433467us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b540048fb54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:51.920852 (+ 15331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415398400}
W20260709 12:17:52.070353 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44745.1us
I20260709 12:17:52.075076 28285 raft_consensus.cc:2410] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:52.075806 30812 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.076326 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.077987 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:52.078701 30770 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:52.084219 30980 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
I20260709 12:17:52.085621 28983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 286) took 1883 ms. Trace:
I20260709 12:17:52.086522 28284 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:52.026140 30983 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:52.033953 30900 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:52.114048 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.085422 30944 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:52.058069 30978 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.106658 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:52.022295 30922 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.059734 30838 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:52.053885 30870 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:52.061800 30986 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.111483 28283 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:52.133986 30986 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.113606 28288 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:52.134887 28287 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:52.135273 30922 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:52.087467 30984 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.111508 28983 rpcz_store.cc:276] 0709 12:17:50.202587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.204137 (+  1550us) service_pool.cc:224] Handling call
0709 12:17:51.151842 (+947705us) spinlock_profiling.cc:243] Waited 52.7 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:17:51.152093 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:51.515852 (+363759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.085363 (+569511us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54004c36d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:52.085554 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626688128}
I20260709 12:17:52.133880 30870 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.075670 30990 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 12:17:52.137791 30870 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:52.115154 30938 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.563s	user 0.014s	sys 0.000s
I20260709 12:17:52.117992 30993 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.092489 28292 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:17:52.125945 28285 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:52.150216 30993 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.113224 30983 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:52.151239 30983 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:52.114306 30892 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.154299 30892 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:52.156338 30938 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:52.158030 30968 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.315s	user 0.002s	sys 0.000s
I20260709 12:17:52.158854 30938 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:52.163067 28984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 449) took 1086 ms. Trace:
I20260709 12:17:52.163506 28984 rpcz_store.cc:276] 0709 12:17:51.076887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.205044 (+128157us) service_pool.cc:224] Handling call
0709 12:17:52.162816 (+957772us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400661a54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:52.162983 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672942976}
W20260709 12:17:52.165093 30837 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.693s	user 0.008s	sys 0.003s
I20260709 12:17:52.166116 28434 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:17:52.166496 28434 pending_rounds.cc:85] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 3
I20260709 12:17:52.166693 28434 pending_rounds.cc:107] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:17:52.166908 28434 raft_consensus.cc:2889] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:52.125229 30827 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:52.107694 30587 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.170300 30587 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:52.129411 30838 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:52.171561 30838 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.172793 30838 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.172914 30984 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.174678 30478 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:52.175086 30478 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:52.175719 30723 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.176002 30478 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:52.177279 30478 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:52.178660 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.179391 28364 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:52.176223 30723 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.183707 30723 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:52.185132 30821 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:52.129818 30978 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.189916 30978 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:52.191018 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:52.191609 28364 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:52.193198 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.194437 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:52.195519 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.200145 28285 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:52.200543 28366 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:52.201327 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.034729 30564 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:52.210343 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.211037 28359 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
W20260709 12:17:52.211225 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.212525 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.130190 30836 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.217546 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.220386 28285 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
W20260709 12:17:52.218732 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 194736us
I20260709 12:17:52.223033 28364 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:17:52.229624 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.152853 30891 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:52.124366 30900 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.234112 28285 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:52.237812 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.242236 28366 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:52.242955 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.243309 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.244366 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.244940 28366 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:17:52.215759 30781 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:52.245718 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.250919 28366 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:52.224934 30886 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:52.256642 28367 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:52.258734 28285 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:17:52.240748 30981 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 12:17:52.237075 30817 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:52.254853 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.269140 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:52.273320 28360 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:52.237576 30837 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:52.253137 30891 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.222316 30760 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:52.228070 30836 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:52.292256 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.277120 30829 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.003s
I20260709 12:17:52.297590 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:52.300426 30685 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:17:52.301998 30827 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:52.228194 30933 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:52.310766 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.128575 30945 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:52.363090 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65568us
I20260709 12:17:52.268143 30866 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:52.374923 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.427119 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.428129 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:52.298234 30837 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.292508 30891 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:52.519910 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.446705 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.960861 30861 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:52.664290 28332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.007s	sys 0.004s
W20260709 12:17:52.853457 30974 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.750s	user 0.007s	sys 0.000s
I20260709 12:17:52.962267 30974 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:52.691535 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.962965 28292 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:52.963076 30853 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:52.963663 30853 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:52.965927 30853 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:52.966580 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.966974 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.967860 30760 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:52.968758 30984 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:52.968845 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:52.971622 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.971904 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:52.978992 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.972332 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:52.973065 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.979910 28285 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
W20260709 12:17:52.980449 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 690403us
I20260709 12:17:52.981258 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:52.976042 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1599) took 1701 ms. Trace:
I20260709 12:17:52.977090 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.982864 28434 rpcz_store.cc:276] 0709 12:17:51.274213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.274327 (+   114us) service_pool.cc:224] Handling call
0709 12:17:51.274608 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.167335 (+892727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.975759 (+808424us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54000d8c54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:52.975972 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049083648}
I20260709 12:17:52.439073 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.984388 28289 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:52.384506 30922 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:17:52.573549 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 145521us
I20260709 12:17:52.979679 28286 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:52.983090 28288 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:52.982542 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:52.961841 28332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.011s	sys 0.008s
W20260709 12:17:52.986250 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.986250 28285 raft_consensus.cc:2376] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7. Re-sending same reply.
W20260709 12:17:52.986879 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.933084 30880 consensus_meta.cc:287] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.979s	user 0.004s	sys 0.004s
W20260709 12:17:52.854615 30478 consensus_meta.cc:287] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.000s
W20260709 12:17:52.867327 30975 consensus_meta.cc:287] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.892s	user 0.009s	sys 0.000s
W20260709 12:17:52.869598 30871 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.897s	user 0.006s	sys 0.003s
I20260709 12:17:52.987736 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.389496 30827 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:52.989107 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.006007 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.641769 30829 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:52.990131 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:53.008744 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:52.993970 30983 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.842s	user 0.009s	sys 0.002s
W20260709 12:17:52.683460 30899 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.001s	sys 0.010s
W20260709 12:17:52.546170 30811 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.694s	user 0.004s	sys 0.007s
W20260709 12:17:52.994985 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.997725 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.998087 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.998826 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:52.469306 30933 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.001610 28359 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:52.867870 30900 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:52.850361 30836 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:52.585304 30886 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.987586 30922 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.018882 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:53.023217 28439 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:53.021442 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:52.436841 30866 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:53.027171 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.028206 28363 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:53.028331 28360 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
W20260709 12:17:53.021409 30838 consensus_meta.cc:287] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.848s	user 0.013s	sys 0.001s
I20260709 12:17:53.003115 30861 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:53.002249 30975 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.000291 30880 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.016772 30871 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.018139 30827 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.017444 30829 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:53.187299 30992 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:17:53.195539 30922 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:53.208316 30965 raft_consensus.cc:2243] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:53.209445 30965 raft_consensus.cc:2272] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:53.213269 30965 ts_tablet_manager.cc:1916] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:17:53.214499 30997 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:17:53.214881 30986 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.216336 30922 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.218698 30922 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.027745 30983 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.113224 30942 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:53.220314 30942 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:53.084636 30836 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.220913 30993 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.062707 30478 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.197883 30866 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.198026 30880 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.198040 30861 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.221356 30836 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.222612 30983 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:52.981333 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:53.338332 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.338909 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.995704 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:52.991895 28285 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:52.996556 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:53.195796 28436 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:53.170724 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:53.194628 28438 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 3.
W20260709 12:17:53.491575 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.372247 30564 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:53.492648 30990 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.121521 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:53.493111 30564 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.493566 30990 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:53.359122 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.496522 30564 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:53.497143 30990 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:53.365784 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:53.511615 28288 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:53.374365 30590 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.375712 30615 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.514297 30590 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.514397 30615 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.516569 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1688) took 1283 ms. Trace:
I20260709 12:17:53.517138 30615 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:53.205845 30587 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:53.518061 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:53.519064 28363 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.518636 30587 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.222090 30478 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.378070 30738 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:53.119197 30811 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 12:17:53.201269 30886 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.200052 30829 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.222836 30891 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.382690 30837 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.387315 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1625) took 1326 ms. Trace:
I20260709 12:17:53.388046 30871 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.392122 30861 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:53.393728 30836 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:53.358039 28290 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:53.375895 28283 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:53.357229 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 375794us
I20260709 12:17:53.368007 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:53.062769 30900 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.378418 30745 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:53.541638 30745 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.444397 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:53.542078 30900 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:53.544299 28284 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:17:53.280551 30975 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:53.477627 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 79094us
I20260709 12:17:53.547638 28291 raft_consensus.cc:1217] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Deduplicated request from leader. Original: 5.3->[6.4-6.4]   Dedup: 6.4->[]
I20260709 12:17:53.548409 28291 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:17:53.549396 31013 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.310s	user 0.014s	sys 0.028s
I20260709 12:17:53.537046 28287 rpcz_store.cc:276] 0709 12:17:52.060504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.061379 (+   875us) service_pool.cc:224] Handling call
0709 12:17:52.061654 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.135320 (+ 73666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.386571 (+1251251us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b54003dbdd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:53.386988 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167979904}
W20260709 12:17:53.480301 31021 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.206s	user 0.000s	sys 0.008s
I20260709 12:17:53.481348 28437 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:53.487882 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:53.489845 28285 pending_rounds.cc:85] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 2
I20260709 12:17:53.479954 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:53.528676 30738 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.531092 30478 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:53.531783 30587 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:53.479789 31014 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.168s	user 0.001s	sys 0.007s
I20260709 12:17:53.552672 28284 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:53.561092 28287 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:17:53.547699 31001 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.279s	user 0.001s	sys 0.006s
I20260709 12:17:53.517064 30590 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:53.566999 28287 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:53.566999 28284 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:17:53.569060 28284 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:53.569622 30892 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.569798 28287 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:53.554551 28285 pending_rounds.cc:107] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:53.570897 28284 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:53.547559 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:53.545588 30900 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.542234 30745 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:53.557448 28435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1602) took 2198 ms (client timeout 2089 ms). Trace:
W20260709 12:17:53.559046 31016 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.167s	user 0.000s	sys 0.002s
I20260709 12:17:53.559679 30993 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.559844 30738 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:53.560104 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.568518 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:53.574865 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:53.575857 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1629) took 1487 ms. Trace:
I20260709 12:17:53.517006 28359 rpcz_store.cc:276] 0709 12:17:52.233144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.235785 (+  2641us) service_pool.cc:224] Handling call
0709 12:17:52.236043 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:53.029079 (+793036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.516029 (+486950us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b540007a154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:53.516340 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1758728576}
I20260709 12:17:53.573516 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 256) took 2086 ms. Trace:
I20260709 12:17:53.571817 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:53.576531 28435 rpcz_store.cc:269] 0709 12:17:51.276082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.394443 (+118361us) service_pool.cc:224] Handling call
0709 12:17:53.462940 (+2068497us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b54002e3bd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:53.474904 (+ 11964us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1301110016}
I20260709 12:17:53.583366 28283 rpcz_store.cc:276] 0709 12:17:52.088074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.134347 (+ 46273us) service_pool.cc:224] Handling call
0709 12:17:52.134676 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:53.545930 (+1411254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.575496 (+ 29566us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540009e7d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:53.575740 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2777842944}
I20260709 12:17:53.590874 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:53.591781 28364 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:17:53.594014 30993 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.594774 28286 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:53.596647 28360 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:53.594038 30824 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:53.598928 30824 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:53.599627 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.600286 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.612709 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1631) took 1504 ms. Trace:
I20260709 12:17:53.613238 28292 rpcz_store.cc:276] 0709 12:17:52.108581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.153674 (+ 45093us) service_pool.cc:224] Handling call
0709 12:17:52.154000 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.963425 (+809425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.612186 (+648761us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b540009e2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:53.612524 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860037888}
I20260709 12:17:53.614393 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1574) took 1415 ms. Trace:
I20260709 12:17:53.579831 28285 raft_consensus.cc:2889] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:53.614796 28361 rpcz_store.cc:276] 0709 12:17:52.189647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.197236 (+  7589us) service_pool.cc:224] Handling call
0709 12:17:52.205001 (+  7765us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:52.205369 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:52.573352 (+367983us) log.cc:844] Serialized 14 byte log entry
0709 12:17:53.463579 (+890227us) log.cc:844] Serialized 14 byte log entry
0709 12:17:53.474533 (+ 10954us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:53.486725 (+ 12192us) log.cc:844] Serialized 26 byte log entry
0709 12:17:53.486911 (+   186us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:53.486947 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.487101 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:53.603154 (+116053us) raft_consensus.cc:1685] finished
0709 12:17:53.603178 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:53.604655 (+  1477us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9008,"prepare.queue_time_us":179268,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":3643,"raft.queue_time_us":22313,"raft.run_cpu_time_us":700,"raft.run_wall_time_us":64261,"thread_start_us":711803,"threads_started":2,"wal-append.queue_time_us":617154}
I20260709 12:17:53.617115 30892 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:53.618436 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1630) took 1511 ms. Trace:
I20260709 12:17:53.618119 30892 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.619925 28291 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:53.620900 30892 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:53.622355 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.622997 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.626032 31017 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.626566 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:53.626729 31017 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.627736 30886 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.628432 30886 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.628890 30886 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.629221 31017 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.584002 28358 rpcz_store.cc:276] 0709 12:17:51.476314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.477025 (+   711us) service_pool.cc:224] Handling call
0709 12:17:51.502042 (+ 25017us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:51.517479 (+ 15437us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:51.517515 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:51.518064 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:17:52.106310 (+588246us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b1800352a80. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:17:52.429655 (+323345us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:52.429703 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.536507 (+1106804us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b54003705d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:53.536598 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:53.536701 (+   103us) raft_consensus.cc:1685] finished
0709 12:17:53.536719 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:53.562849 (+ 26130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34565,"raft.queue_time_us":557,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11446,"spinlock_wait_cycles":42662528,"thread_start_us":323128,"threads_started":1,"wal-append.queue_time_us":1502225}
I20260709 12:17:53.634354 30993 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.640228 30942 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:53.642028 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.600641 30824 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:53.646374 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.647001 28954 connection.cc:256] RPC call timeout handler was delayed by -1.0350190359999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
I20260709 12:17:53.618844 28290 rpcz_store.cc:276] 0709 12:17:52.106752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.108340 (+  1588us) service_pool.cc:224] Handling call
0709 12:17:53.618121 (+1509781us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540018f454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:53.618316 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2810435328}
I20260709 12:17:53.617185 28289 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:53.668493 28363 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:17:53.659013 30590 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:53.674307 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:53.678726 30590 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:53.679668 28360 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:53.680012 30590 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.681066 28358 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:53.672166 30942 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.684425 30886 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.701408 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:53.706058 28358 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:17:53.707535 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.729467 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.742535 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:53.743480 28983 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 7 LEARNER]: Advancing to term 8
W20260709 12:17:53.733181 31027 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:17:53.758437 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.760540 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:53.761315 28358 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:53.764741 28983 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 8 LEARNER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:17:53.770434 30738 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:53.771236 30738 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:53.775440 31028 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.160s	user 0.004s	sys 0.000s
I20260709 12:17:53.787228 28358 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:17:53.798187 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:53.798774 30615 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:53.799151 28983 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:53.799516 30615 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.801810 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:53.801940 30993 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:53.802317 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:53.803143 28362 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:53.802775 30993 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.803475 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:53.804098 30956 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:53.804787 30956 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.805090 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:53.807089 30956 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.808260 30770 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.808961 30770 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.809228 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:53.810380 30615 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.811157 30770 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.811235 30956 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:53.811800 30956 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.811883 30829 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.813308 30995 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.814033 30995 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.814985 30899 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.815626 30899 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.816301 31024 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:53.816473 30995 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.817080 30886 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:53.817281 30900 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.817771 30886 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.818034 30900 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:53.818543 30781 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.819110 30781 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.819944 30886 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.820367 30900 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:53.820525 30886 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.821013 30900 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:53.821548 30781 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.817983 30899 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:53.821913 30587 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:53.822489 30899 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.822649 30587 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.823338 31023 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:53.823086 30899 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.823833 31023 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.824283 31023 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.825001 30587 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.825726 30899 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.826452 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:53.827713 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.828296 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.828747 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.828827 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:53.829704 28434 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:53.830654 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:53.830911 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:53.831137 28332 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:53.831784 28360 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:53.832651 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:53.833956 30842 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:53.835681 30842 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:53.836071 30842 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.836426 30842 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.836889 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:53.788113 30990 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:53.838181 30478 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:53.838646 30478 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.839037 30478 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.838759 30990 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:53.840168 30827 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.636s	user 0.014s	sys 0.002s
I20260709 12:17:53.840626 30827 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.841058 30990 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.842069 28432 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:53.817013 31024 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.845391 31024 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:53.853229 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 69007.2us
I20260709 12:17:53.854346 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:53.855942 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:53.856767 30811 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.857403 30811 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.857983 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:53.858896 28984 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:53.860308 28437 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:53.805218 30993 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:53.871518 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.781664 30738 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.869938 30615 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:53.821235 30886 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:53.894258 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:53.907817 30956 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:53.901130 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:53.862373 30770 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:53.914887 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.920485 28365 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:53.922813 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:53.933497 28441 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:53.880883 30990 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:53.946506 28435 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:53.947746 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:53.948431 28436 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:53.821722 30900 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:17:54.002035 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.005168 28360 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:54.010341 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:54.010533 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 160394us
I20260709 12:17:53.949915 30956 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:53.843226 28189 catalog_manager.cc:5697] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 5, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 5 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:54.025810 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 131706us
W20260709 12:17:54.028663 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.037169 30745 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.039635 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 1747) took 1031 ms. Trace:
I20260709 12:17:54.040077 28432 rpcz_store.cc:276] 0709 12:17:53.008316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.008444 (+   128us) service_pool.cc:224] Handling call
0709 12:17:54.025479 (+1017035us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54002e4ad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:54.039564 (+ 14085us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913654656}
I20260709 12:17:54.042357 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.043365 28438 raft_consensus.cc:2410] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:54.044552 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.046473 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.047359 28440 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:54.048297 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.048934 28440 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:54.050570 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.050900 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.051684 28440 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:54.054551 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.055204 28438 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:54.058976 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.061229 30900 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:54.063375 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:54.064033 28360 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:54.064452 30886 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:53.935823 30770 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:54.066558 30933 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.033s	user 0.016s	sys 0.001s
I20260709 12:17:54.067593 30781 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.068044 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:54.068120 30781 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.066569 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:54.074820 28364 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:54.076282 30478 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.077208 28189 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 4, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 4 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:54.079262 28189 catalog_manager.cc:5697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:54.083662 28433 raft_consensus.cc:2410] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:54.051585 28435 raft_consensus.cc:2410] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:54.087903 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:54.016702 30956 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.964162 30842 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.101213 28441 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:17:54.107589 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 57082.2us
I20260709 12:17:54.089551 30829 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:54.099869 30956 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.100502 30933 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:54.099191 30942 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.098102 30993 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:54.122542 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.128310 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:54.128633 28984 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 8 LEARNER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:17:54.117095 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:54.133575 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.130239 28434 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:54.138756 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:54.139505 28983 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:54.140112 28288 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:54.139595 28287 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:54.146461 28986 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:17:54.136045 30827 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:54.150779 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:54.151587 28283 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
W20260709 12:17:54.154999 31039 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.346s	user 0.001s	sys 0.005s
I20260709 12:17:54.157976 28435 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:54.162935 30842 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.163900 30842 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.164479 30842 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.178002 28290 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:54.178633 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:54.129487 30995 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:54.137503 28189 catalog_manager.cc:5697] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 4, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 4 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:54.190131 28283 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:54.192570 30995 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.193465 30615 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:54.194432 30734 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.195302 30995 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.195075 30734 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.200552 30781 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:54.202047 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:54.203007 28441 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:54.203905 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.204548 28441 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:54.205417 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:54.211089 28287 raft_consensus.cc:2410] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:54.223119 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.214812 28189 catalog_manager.cc:5697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:54.220219 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:54.229305 28288 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:54.224617 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:54.231688 28290 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:54.191674 30478 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.233110 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:54.173303 30942 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.197676 30734 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.170621 30933 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.228442 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:54.256855 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:54.183064 30993 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.259030 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:54.262154 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:54.262058 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:54.234539 30824 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.255954 31041 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.263010 28287 raft_consensus.cc:2376] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8. Re-sending same reply.
W20260709 12:17:54.293088 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.256264 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:54.302042 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:54.295264 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:54.308004 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:54.310586 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 214601us
I20260709 12:17:54.310765 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:54.312431 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 199562us
W20260709 12:17:54.316082 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.317867 28290 raft_consensus.cc:2410] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:54.318668 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:54.321532 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 519878us
I20260709 12:17:54.255019 31023 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.294953 30615 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.122s
I20260709 12:17:54.324349 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:54.460520 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:54.465792 28984 raft_consensus.cc:2410] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:54.467083 30993 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.468637 30942 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:54.408059 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 71084.8us
I20260709 12:17:54.403607 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:54.398898 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 78700.3us
I20260709 12:17:54.335151 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:54.476747 28436 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
W20260709 12:17:54.479727 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:54.481873 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.370440 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:54.483732 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.307354 30842 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:54.360016 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:54.353147 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.335070 28283 raft_consensus.cc:2376] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7. Re-sending same reply.
I20260709 12:17:54.491408 28367 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:17:54.485030 28361 raft_consensus.cc:2410] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 12:17:54.423411 28288 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:54.324683 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.323863 28986 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:54.365456 28189 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 2 to 5, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 5 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:54.418413 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:54.502557 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:54.503551 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:54.498632 28441 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:17:54.506126 28433 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:54.482481 30770 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:54.508347 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 395085us
I20260709 12:17:54.359481 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:54.336776 30995 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.518323 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:54.519143 31036 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.721s	user 0.009s	sys 0.000s
I20260709 12:17:54.520155 28332 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:54.521170 31052 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:54.521704 31052 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:54.522811 31052 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.488996 30781 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.514562 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:54.499153 30829 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.312227 30824 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.499089 30738 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.485855 30942 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:17:54.507045 31042 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:17:54.513765 30975 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.019s	sys 0.022s
I20260709 12:17:54.469297 31046 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 12:17:54.525918 28364 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.525337 28287 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:54.545595 31059 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:17:54.548210 30933 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:54.610085 28435 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:17:54.404187 31041 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:54.549675 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 227202us
I20260709 12:17:54.554893 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:54.540596 30871 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:54.560361 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.544142 30734 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:54.565472 28987 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:54.604868 28989 raft_consensus.cc:2410] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:17:54.567523 28433 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
W20260709 12:17:54.554234 30975 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.019s	sys 0.023s
I20260709 12:17:54.571753 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:54.584743 30824 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:54.586859 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:54.587433 28984 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:54.588685 31023 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.557585 30738 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.552613 30829 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.590780 28189 catalog_manager.cc:5697] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 2 to 3, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:17:54.568974 30811 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.625504 28986 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:54.595686 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:54.575230 30842 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:54.541725 30781 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:54.603184 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 59764.3us
W20260709 12:17:54.611981 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 57163.4us
I20260709 12:17:54.630940 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:54.614298 28433 pending_rounds.cc:85] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 6
I20260709 12:17:54.630872 28436 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:54.631705 28433 pending_rounds.cc:107] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:17:54.632061 28433 raft_consensus.cc:2889] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:54.560752 30995 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.613482 30734 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:54.636545 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:54.629433 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:54.640336 30829 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:54.640743 30886 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:54.641407 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:54.625821 31054 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.625885 31053 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:54.647308 31041 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.652204 30781 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:54.653162 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 85318us
I20260709 12:17:54.638361 30738 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.629153 30842 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.663233 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:54.669164 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:54.665158 28987 raft_consensus.cc:2376] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7. Re-sending same reply.
I20260709 12:17:54.658048 28364 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:17:54.662348 30734 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.684994 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:54.620142 30871 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.666085 31059 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:54.664634 30886 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:54.685387 30842 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.670723 31053 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:54.679687 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:54.688684 30829 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:54.690395 30811 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:54.693934 30975 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.688954 31041 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.691795 30942 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:54.695577 30886 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:54.694257 31059 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:54.715545 28987 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:17:54.736179 28987 raft_consensus.cc:606] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:54.737895 28987 raft_consensus.cc:614] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:54.727650 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:54.696069 30781 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:17:54.709964 30829 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.703591 31053 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.711699 30811 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:54.734141 31063 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.713016 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:54.790330 31067 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:54.738533 31059 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:54.764190 30956 consensus_meta.cc:287] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.627s	user 0.012s	sys 0.000s
W20260709 12:17:54.789135 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 77647.9us
I20260709 12:17:54.787416 31023 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:17:54.787962 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 102920us
W20260709 12:17:54.785552 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 93815.7us
W20260709 12:17:54.793862 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 94076.4us
I20260709 12:17:54.738976 30942 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:54.797463 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:54.712984 30965 ts_tablet_manager.cc:1929] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:54.811825 30965 log.cc:1199] T e2ac8850682043e5bf8caf7fc64fae05 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/e2ac8850682043e5bf8caf7fc64fae05
I20260709 12:17:54.787001 30829 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.789028 30811 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.802848 31063 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.804977 30995 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.728971 31042 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:17:54.805795 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:54.798378 30956 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.738205 31041 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:54.810113 30942 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:54.811270 31060 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:17:54.838604 30745 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.839671 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:54.842434 30933 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:54.843122 30933 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:54.843731 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.844127 30933 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.827862 30871 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:54.828963 31023 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.851653 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:54.853041 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:54.834502 30956 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.834839 30965 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33090 (request call id 73) took 3589 ms. Trace:
I20260709 12:17:54.854300 30965 rpcz_store.cc:276] 0709 12:17:51.245003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.245279 (+   276us) service_pool.cc:224] Handling call
0709 12:17:51.538731 (+293452us) ts_tablet_manager.cc:1151] Deleting tablet e2ac8850682043e5bf8caf7fc64fae05
0709 12:17:51.567731 (+ 29000us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:17:54.055053 (+2487322us) tablet_metadata.cc:665] Metadata flushed
0709 12:17:54.599776 (+544723us) tablet_metadata.cc:665] Metadata flushed
0709 12:17:54.834709 (+234933us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1402231,"tablet-delete.queue_time_us":175798,"thread_start_us":38866,"threads_started":1}
I20260709 12:17:54.836534 31041 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:54.737265 30975 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.844717 28439 raft_consensus.cc:2410] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:54.823918 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:54.858556 30975 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:54.859145 30975 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:54.859616 30975 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:54.860535 30975 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:54.863688 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.865128 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:54.871507 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.874526 28363 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:17:54.875231 31058 log.cc:927] Time spent T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
I20260709 12:17:54.876405 28367 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:54.878062 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:54.801460 31067 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:54.896202 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 76995.7us
W20260709 12:17:54.902978 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 108832us
I20260709 12:17:54.907418 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:54.909189 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.912496 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:54.921603 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:54.917346 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 103846us
I20260709 12:17:54.922608 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:54.901799 31051 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.309s	user 0.001s	sys 0.002s
I20260709 12:17:54.923835 28439 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:17:54.916611 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:54.926599 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:54.927093 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.926669 28365 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:54.934159 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.934365 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.935205 28436 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:54.904307 31067 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.934492 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:54.935489 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.930871 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:54.943976 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.946244 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.946851 28359 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:54.947110 28437 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:54.948910 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.949152 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:54.950547 28433 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:54.949808 28441 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:17:54.931700 31054 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:54.953987 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 148417us
I20260709 12:17:54.961300 28434 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:54.938162 30781 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.930814 30824 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:54.947152 31041 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:54.973651 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:54.974737 28366 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:54.978065 31063 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:54.979004 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.979027 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:54.980202 28436 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:54.982846 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.949115 31073 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:54.943511 31071 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 12:17:54.995369 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 69131.5us
I20260709 12:17:54.949862 30995 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
W20260709 12:17:55.008782 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 83089.5us
I20260709 12:17:54.979866 31041 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:55.012473 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.018163 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:55.026597 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet e2ac8850682043e5bf8caf7fc64fae05 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:17:55.026391 30734 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:55.027345 28176 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 73830.9us
I20260709 12:17:55.014042 31041 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:55.028074 31042 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:55.028872 30734 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:55.034857 30838 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:55.035646 30838 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:55.042554 31075 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:17:55.059624 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:54.982611 30842 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:55.063270 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 175054us
I20260709 12:17:55.072281 28364 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:55.002216 30811 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.064914 30770 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:55.065659 30838 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:55.083259 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.044724 30942 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.066152 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:55.009312 30781 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:55.068333 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47922.6us
I20260709 12:17:55.068377 30734 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.072697 28363 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
W20260709 12:17:55.077890 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51956.6us
I20260709 12:17:55.080035 28436 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:55.024775 30478 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.002843 30824 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.092000 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:55.091923 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:55.094628 28362 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:55.097616 28436 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
W20260709 12:17:55.098560 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 71134.2us
I20260709 12:17:55.078552 30842 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:55.077538 30770 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:55.095556 30478 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:55.105161 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:55.105584 30811 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:55.106535 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:55.107201 30478 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:55.107689 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.108306 28435 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.103864 30770 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:55.111299 31049 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.514s	user 0.003s	sys 0.001s
W20260709 12:17:55.114289 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.122937 30824 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:55.124451 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.125319 28437 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:55.126221 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.126772 28437 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.127358 30781 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:55.108023 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:55.129300 28359 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.077690 30829 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.013284 31073 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:55.135115 28332 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:55.137621 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:55.145977 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:55.146345 28435 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:55.092394 30734 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.151369 31054 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:55.164335 28437 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:55.162853 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.134197 31076 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:17:55.167846 28360 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:55.121538 30886 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.148846 31023 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:55.172267 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:55.138312 30615 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:17:55.149049 30871 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:55.179165 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:55.188971 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1767) took 1133 ms. Trace:
I20260709 12:17:55.165544 30842 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:55.148586 31053 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.195719 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:55.196033 30734 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:55.197001 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:55.197351 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:55.169224 31059 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.435788 28285 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:55.197675 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:55.390218 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.216048 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:55.216121 28287 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:55.318744 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:55.189487 28440 rpcz_store.cc:276] 0709 12:17:54.043425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.052383 (+  8958us) service_pool.cc:224] Handling call
0709 12:17:54.052815 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:54.053315 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:54.053345 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:54.053937 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:17:54.324051 (+270114us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:54.324096 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.324164 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:55.110502 (+786338us) spinlock_profiling.cc:243] Waited 7.87 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:55.166397 (+ 55895us) raft_consensus.cc:1685] finished
0709 12:17:55.166420 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:55.176344 (+  9924us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4685,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"spinlock_wait_cycles":18094848,"thread_start_us":269650,"threads_started":1,"wal-append.queue_time_us":542354}
I20260709 12:17:55.264221 28359 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
W20260709 12:17:55.444799 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.266841 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:55.216609 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43537.4us
W20260709 12:17:55.323312 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 251521us
I20260709 12:17:55.267493 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:55.440743 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:55.216542 28332 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:55.317625 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:55.453003 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 236560us
W20260709 12:17:55.451951 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.453158 28360 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:55.449637 28291 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:17:55.270967 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:55.444031 28283 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:55.175263 31054 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:55.286949 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 318970us
I20260709 12:17:55.215888 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:55.318157 31048 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:17:55.444902 28255 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.4:39985 timed out after 1.663s (SENT) [suppressed 20 similar messages]
I20260709 12:17:55.459259 28290 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 2.
I20260709 12:17:55.463927 28364 raft_consensus.cc:2410] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:55.465410 28289 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:55.468827 28366 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:55.472131 28361 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:55.472139 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:55.415567 30942 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:55.309089 31085 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:55.192674 31076 consensus_queue.cc:1243] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:55.197017 30829 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:55.477252 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:55.478484 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 340943us
I20260709 12:17:55.474617 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:55.459791 30615 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:55.484771 28332 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:55.266438 31023 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.211s
I20260709 12:17:55.258219 30871 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:55.332050 31071 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.152s
I20260709 12:17:55.327634 30995 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.153s
I20260709 12:17:55.402472 31067 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.494958 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:55.479823 30838 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:55.497845 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.488600 30615 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.492208 31091 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:55.494735 31076 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:55.509341 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.511914 28330 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:55.512439 28363 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:55.509121 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:55.514355 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 187820us
I20260709 12:17:55.499778 31053 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:55.503037 30942 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.537230 30829 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:55.502310 31046 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:55.540169 30871 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:55.506829 30615 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.508306 30886 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:55.508167 31091 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.555886 28288 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:55.560801 28362 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
W20260709 12:17:55.564077 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 80147.5us
W20260709 12:17:55.564519 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 60413.8us
I20260709 12:17:55.504498 31052 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:17:55.541707 30478 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:55.567847 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 56027.9us
I20260709 12:17:55.508539 30993 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:17:55.518138 30956 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:17:55.562469 31091 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:55.576323 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:55.578982 31069 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:55.579509 31069 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:55.580452 31069 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:55.563004 31046 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:55.570323 31052 consensus_queue.cc:1243] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:17:55.596585 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59690.3us
I20260709 12:17:55.601172 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:55.564697 30770 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:55.593220 31076 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:55.591415 31059 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:55.599941 31055 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.006s	user 0.005s	sys 0.000s
I20260709 12:17:55.615477 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:55.617192 31053 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:55.545900 31054 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.605576 31069 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:55.611130 31041 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.000s
I20260709 12:17:55.611547 31099 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:55.607086 31042 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:17:55.622185 30995 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.635227 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:55.666977 31073 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:55.668381 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:55.624987 30838 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:55.670127 30838 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.670852 30838 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:17:55.672039 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.672435 30838 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.674005 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:55.674826 31073 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:55.675204 31073 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.675551 31073 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.679298 31063 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:55.681155 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:55.682220 30734 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:55.682664 30734 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:55.683594 30734 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.684847 30734 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:55.685906 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:55.686424 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.649175 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.687505 31100 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:55.688158 31086 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.423s	user 0.002s	sys 0.000s
I20260709 12:17:55.651289 31067 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:55.651314 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1761) took 1610 ms. Trace:
I20260709 12:17:55.632798 31046 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.658840 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:55.626785 31041 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:55.664134 31087 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.467s	user 0.002s	sys 0.000s
W20260709 12:17:55.705904 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47142.9us
I20260709 12:17:55.677111 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:55.647045 30770 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:55.692054 28330 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:17:55.694036 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 96744.2us
I20260709 12:17:55.704820 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.711613 28439 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.694221 28432 rpcz_store.cc:276] 0709 12:17:54.041272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.090073 (+ 48801us) service_pool.cc:224] Handling call
0709 12:17:54.105031 (+ 14958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:54.110952 (+  5921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:54.110989 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:54.124214 (+ 13225us) log.cc:844] Serialized 26 byte log entry
0709 12:17:54.320794 (+196580us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:54.320836 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.320906 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:55.508137 (+1187231us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:55.650956 (+142819us) raft_consensus.cc:1685] finished
0709 12:17:55.650978 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:55.651226 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4648,"raft.queue_time_us":517,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6883,"spinlock_wait_cycles":12955904,"thread_start_us":161806,"threads_started":1,"wal-append.queue_time_us":435214}
I20260709 12:17:55.714633 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.715410 28440 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.701478 28435 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:55.719718 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:55.720854 30770 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:55.721868 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:55.722312 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:17:55.722461 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 113471us
I20260709 12:17:55.723246 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:55.721431 30770 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:55.715847 31079 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:55.728679 31079 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:55.731616 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:55.717242 30745 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:55.733830 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:55.729966 30886 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.741089 30811 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:55.741640 30811 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:55.742641 30811 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.724007 30942 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:17:55.744804 31079 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:55.737164 30871 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:55.730047 31067 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.735204 30745 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.743109 31041 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:55.748046 30745 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.737219 30995 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:55.750092 30995 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:55.751128 30995 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:55.752629 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:55.754169 30824 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:55.754662 30824 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:55.755061 30824 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.743996 30811 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.735406 31081 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:17:55.728924 30770 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:55.757181 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:55.760588 28986 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:17:55.763756 31103 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:17:55.765091 31067 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:55.761170 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:17:55.776389 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 59860.5us
W20260709 12:17:55.776754 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47491.4us
I20260709 12:17:55.777463 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:55.778308 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.786141 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.789700 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:55.791076 28440 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:55.793233 28432 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.794860 28435 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:55.798916 28366 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:55.803774 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:55.804659 28290 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:17:55.807730 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:55.792725 30942 raft_consensus.cc:2764] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:17:55.808233 28289 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:55.793689 31081 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:55.824673 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:55.825603 28288 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:17:55.797101 30990 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:55.829351 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:55.725972 31076 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.838937 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:55.836035 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:55.839401 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.836217 28363 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:55.806262 31103 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:55.842697 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:55.843758 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66391.3us
I20260709 12:17:55.795433 31071 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:17:55.846608 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:55.847602 28358 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.849810 28366 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:17:55.852241 28987 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:17:55.832914 30990 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.858023 28361 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:17:55.858204 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 60804.2us
I20260709 12:17:55.859997 28289 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:55.865258 28435 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:17:55.832298 28983 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:17:55.871830 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:55.872884 31054 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:17:55.873054 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:55.873544 31054 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:55.874030 31054 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:55.874450 31101 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:55.875123 31101 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:55.876262 31101 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:55.878103 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.884413 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:55.841104 31081 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:55.889133 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.889187 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:55.892444 28366 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:55.894058 28439 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:55.896327 28988 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:55.840847 30615 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:55.892846 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:55.866194 30990 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.865922 28984 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:55.931305 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:55.934616 28982 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 5 LEARNER]: Advancing to term 6
W20260709 12:17:55.935708 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51398us
W20260709 12:17:55.939117 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:55.927158 31089 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.003s	sys 0.009s
I20260709 12:17:55.939853 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:55.930169 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:55.945559 28361 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:55.902652 30886 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:55.944522 31089 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.003s	sys 0.010s
W20260709 12:17:55.960103 28330 leader_election.cc:336] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.053s (SENT)
I20260709 12:17:55.960815 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:55.956280 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:55.909891 31091 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:55.855471 31103 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:55.949404 30838 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:55.950714 31085 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:55.953490 30478 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:55.979959 28330 leader_election.cc:336] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.128s (SENT)
I20260709 12:17:55.959079 28358 raft_consensus.cc:2410] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:55.966596 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:55.980587 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:55.943986 31054 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:56.003361 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:55.948505 31102 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 12:17:56.018535 31114 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:17:55.974345 31099 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:56.019520 31042 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:55.974681 31085 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.020388 31085 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:56.023056 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:56.024768 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:56.025082 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:55.981333 31081 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.032083 30829 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:56.032363 30886 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:56.006395 28439 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.007859 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:56.035291 28990 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:56.008692 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:56.009780 31115 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:17:56.010682 31105 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:17:56.014036 28985 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:56.015329 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:56.056595 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 76691.4us
I20260709 12:17:56.037217 28361 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:17:55.973613 30615 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.001902 30838 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:17:56.081288 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 217674us
I20260709 12:17:56.097170 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:56.033425 30886 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:56.107049 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 82332us
W20260709 12:17:56.108479 28330 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.178s (SENT)
I20260709 12:17:56.109936 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:56.115984 28432 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:56.117228 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:56.048094 31052 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.040382 30871 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:17:56.003743 31023 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:17:56.048391 31099 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:56.106117 30829 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.131289 28439 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
W20260709 12:17:56.131932 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 173702us
I20260709 12:17:56.134366 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:56.129590 28986 raft_consensus.cc:2393] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 4: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:56.109122 30886 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:56.149297 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:56.061242 30745 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:56.150900 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 248102us
W20260709 12:17:56.168753 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 72591.3us
I20260709 12:17:56.169559 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:56.157219 30975 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:17:56.175465 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.178146 31093 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:56.180557 30886 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.163905 31073 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.172745 30975 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:56.161103 30734 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.177177 31097 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:17:56.185829 31123 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:17:56.172421 31099 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:56.219281 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 70547.4us
I20260709 12:17:56.248229 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:17:56.212436 31112 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.206s	user 0.000s	sys 0.002s
W20260709 12:17:56.250804 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 101588us
W20260709 12:17:56.251751 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 84576.2us
W20260709 12:17:56.260855 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 90895.1us
W20260709 12:17:56.250233 31122 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:17:56.168519 30838 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.263375 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:17:56.269884 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.262244 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:56.248534 31091 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.250861 31093 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:56.260516 31128 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:17:56.257644 30886 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:56.259065 31097 consensus_queue.cc:1243] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:17:56.312827 30770 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:17:56.315459 31125 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:17:56.271618 31121 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.258896 31054 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:56.248107 30990 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:56.269142 28982 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 6 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:56.289628 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:56.321354 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:56.362314 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44484.8us
I20260709 12:17:56.323421 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:56.372372 28365 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:17:56.375356 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 86069.5us
I20260709 12:17:56.376065 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:56.337467 30745 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.285301 30975 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.379269 31121 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:17:56.396324 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 134301us
W20260709 12:17:56.399451 30824 consensus_meta.cc:287] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.003s
W20260709 12:17:56.400277 31129 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:17:56.400920 30990 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:56.402531 31140 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:56.407564 31023 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:56.368708 30829 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:17:56.419727 28433 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:17:56.421196 31123 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:17:56.419348 31023 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:56.424305 28982 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:17:56.424908 31023 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:56.425835 30975 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.426185 31023 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.427886 31142 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:56.429104 31113 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.485s	user 0.000s	sys 0.002s
I20260709 12:17:56.416496 30824 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:56.438117 28283 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:56.429014 31139 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:17:56.432879 30824 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:56.448086 31116 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:56.448635 31116 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:56.445281 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:56.449536 31116 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.450943 31116 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.451073 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:56.423029 31073 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.465792 31142 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:56.471684 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51071.1us
I20260709 12:17:56.445623 31097 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:17:56.472196 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:56.668395 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:56.666793 28255 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Timed out: RequestConsensusVote RPC to 127.26.123.4:39985 timed out after 1.598s (SENT)
W20260709 12:17:56.502352 31107 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.543s	user 0.004s	sys 0.002s
I20260709 12:17:56.481354 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:17:56.700330 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 230067us
I20260709 12:17:56.702564 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:56.703890 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:56.705358 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:56.707439 31063 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:56.707912 31131 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.707715 31132 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:56.708947 31063 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.709354 31063 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.709954 31140 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:56.710402 31140 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.710734 31140 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:56.713274 31134 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.563s	user 0.001s	sys 0.001s
I20260709 12:17:56.481886 31142 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:56.722534 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:56.728673 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 238776us
I20260709 12:17:56.698359 30975 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.699941 30829 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:56.730130 28440 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:56.730839 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:56.741055 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:56.731611 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:56.741999 28436 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:56.540366 31085 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:56.741685 28435 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:56.698794 31123 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.729584 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:56.744948 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:56.745956 28437 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:56.746924 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:56.749868 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:56.750301 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:56.751004 28434 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:17:56.751439 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:56.754411 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.754782 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:56.755754 28435 raft_consensus.cc:2410] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:56.756711 28438 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:56.752748 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:56.762768 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:56.762866 28436 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:56.765165 28440 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:56.709501 31143 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.325s
I20260709 12:17:56.769208 28441 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:56.753815 31073 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.739977 30829 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:56.781968 28986 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:17:56.775058 31157 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:17:56.788399 31157 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.742517 30975 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:56.789618 28434 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:17:56.791857 30975 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:56.767611 31153 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:56.793161 31153 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.793848 31152 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 12:17:56.795393 31152 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:56.795806 31152 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:56.798611 28437 raft_consensus.cc:2410] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:56.795835 31153 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:56.803118 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:56.804013 28983 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:56.803755 30829 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.805349 30829 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:56.806391 31085 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.808714 31073 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.811383 31073 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.779861 31071 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:17:56.815855 31100 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:56.816048 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 758664us
W20260709 12:17:56.816723 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 699561us
I20260709 12:17:56.817246 30478 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:56.817842 30478 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:56.818778 30478 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.822054 31052 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:56.822582 31052 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.824378 31052 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.824532 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:56.825914 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:56.827181 30770 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.828841 31053 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:56.831553 30811 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:56.832111 30811 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:56.834331 30811 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:56.834266 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 291) took 1001 ms. Trace:
I20260709 12:17:56.835568 28283 rpcz_store.cc:276] 0709 12:17:55.832570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.832759 (+   189us) service_pool.cc:224] Handling call
0709 12:17:55.843753 (+ 10994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:56.439545 (+595792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.834020 (+394475us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b54000028d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:56.834193 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1356632448}
I20260709 12:17:56.796882 31152 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.841419 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:17:56.816850 28330 leader_election.cc:336] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.207s (SENT)
I20260709 12:17:56.842233 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:56.829388 31053 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.845997 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:56.846720 28288 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:56.847080 28255 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:56.847611 31053 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.847872 31123 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:56.832041 31103 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.849264 31123 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:17:56.850051 30975 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.850641 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:56.851649 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:56.851827 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:56.852485 30824 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:56.852883 30824 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.853267 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:56.853464 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:56.855016 28364 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:56.852039 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:56.850225 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:56.854511 28991 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:17:56.846186 31119 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.882s	user 0.002s	sys 0.010s
I20260709 12:17:56.847815 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:56.834853 30811 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.852936 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:56.862026 28285 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.853977 28286 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:17:56.858513 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:56.858928 28292 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59772
I20260709 12:17:56.863117 28292 raft_consensus.cc:606] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Received request to transfer leadership
I20260709 12:17:56.863512 28292 raft_consensus.cc:614] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:56.859725 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:56.860277 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:56.863238 28367 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:56.866029 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:56.866325 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:56.866842 28291 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:56.867204 28284 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:17:56.869474 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 139983us
I20260709 12:17:56.869773 28363 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
W20260709 12:17:56.870618 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.872570 28365 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:17:56.872958 28365 pending_rounds.cc:85] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 2
I20260709 12:17:56.873174 28365 pending_rounds.cc:107] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:17:56.873504 28365 raft_consensus.cc:2889] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:56.873132 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:56.853308 30824 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.847729 30871 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:56.878582 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:56.879660 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:56.854350 30842 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:56.882027 28283 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.884644 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:56.881711 28288 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:56.886407 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:56.886720 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:56.880182 31152 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:56.881928 31101 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.996s	user 0.002s	sys 0.018s
W20260709 12:17:56.910359 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 93548.1us
I20260709 12:17:56.919647 28359 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:17:56.925185 28330 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:56.920974 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:56.925639 28332 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:56.925660 28367 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:56.873692 30811 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:56.931505 31081 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:56.936199 28288 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:17:56.902805 31053 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:56.942353 28366 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:56.943431 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.876232 31140 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.944934 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:17:56.925657 31101 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.043s	user 0.002s	sys 0.019s
I20260709 12:17:56.894187 30842 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:56.944576 31052 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:17:56.960388 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:56.954453 28434 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:17:56.968395 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:56.969136 28288 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:56.973243 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:56.975077 28287 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:56.976823 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:56.947067 31046 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:56.977648 28439 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:56.954761 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:56.988191 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:56.993242 28288 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
W20260709 12:17:56.978826 31120 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.045s	user 0.004s	sys 0.007s
I20260709 12:17:56.960173 31137 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.947443 31123 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:17:57.031311 28434 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:56.980908 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:57.069603 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 313) took 1236 ms. Trace:
I20260709 12:17:57.070091 28985 rpcz_store.cc:276] 0709 12:17:55.833582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.894315 (+ 60733us) service_pool.cc:224] Handling call
0709 12:17:57.069353 (+1175038us) spinlock_profiling.cc:243] Waited 6.68 ms on lock 0x7b54001caf54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:57.069497 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15373696}
I20260709 12:17:57.071867 31123 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:17:57.072320 31123 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:57.072512 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:57.077121 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:57.083845 28285 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:57.046912 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:57.047202 30829 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:57.051157 31160 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:17:57.049894 31063 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:57.055032 30811 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:57.058799 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.060302 31140 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:56.945667 31081 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:56.960590 30478 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:57.092844 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 104736us
I20260709 12:17:57.131385 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:57.095333 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 105454us
I20260709 12:17:57.098778 28292 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:17:57.132872 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:57.049345 30842 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:57.122105 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:57.138077 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:57.138660 28439 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:57.139421 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:57.140669 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:57.141397 28290 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:57.142886 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:57.117882 31163 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:57.149889 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.193660 28283 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:57.195772 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.197598 28436 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:57.198534 28292 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:57.205219 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:57.210592 28441 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:17:57.216080 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.229645 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:57.372201 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45801.6us
I20260709 12:17:57.338598 28366 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:57.311744 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:17:57.374920 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.361449 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.376382 28432 raft_consensus.cc:2410] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:17:57.356287 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 224994us
I20260709 12:17:57.377995 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:57.378829 30829 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:57.379297 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:57.380103 30829 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:57.380216 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:57.380527 30829 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:57.380941 30829 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:57.382176 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.236024 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.384372 28440 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:17:57.198439 31053 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.386963 28362 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:57.146713 31163 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:57.195187 30478 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:57.371187 31076 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.226841 31079 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:57.146449 31123 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:57.344027 31142 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:57.390642 31069 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.396829 31163 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.407320 31100 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:17:57.430822 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 50649.7us
I20260709 12:17:57.404330 31053 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:57.434892 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:57.440244 28439 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:57.436132 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:57.417503 31079 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:57.403720 31054 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 12:17:57.481390 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.374507 31081 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:57.433959 31161 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.421452 31076 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:57.477377 31079 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:57.550684 31171 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:17:57.432428 31142 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:57.560364 28288 pending_rounds.cc:85] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 3
I20260709 12:17:57.560540 31053 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:57.560890 28288 pending_rounds.cc:107] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:17:57.561249 28288 raft_consensus.cc:2889] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:57.566025 31081 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.513096 31161 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:57.578157 31161 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:57.553265 30770 consensus_meta.cc:287] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.002s
I20260709 12:17:57.434122 31069 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:57.581898 31142 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:57.448627 31100 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:57.583596 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.581530 30770 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.584376 31100 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:57.585080 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.553440 28287 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:17:57.536890 30824 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.649s	user 0.014s	sys 0.000s
I20260709 12:17:57.577965 28291 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:57.593853 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:57.594296 28291 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:17:57.596468 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:57.602840 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:57.604518 28290 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
W20260709 12:17:57.608914 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 195136us
W20260709 12:17:57.598774 31152 consensus_meta.cc:287] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.667s	user 0.006s	sys 0.006s
W20260709 12:17:57.612705 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 176750us
W20260709 12:17:57.598064 31174 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:17:57.614843 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:17:57.598254 31165 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.640973 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:57.643503 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.645759 28439 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:17:57.616354 28291 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:57.648564 28291 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:57.604067 31053 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:57.602622 31097 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:57.607446 31046 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:57.648182 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 46812.1us
I20260709 12:17:57.613626 30838 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.673029 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:57.673436 31053 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.674074 31053 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:57.676018 28991 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:17:57.676473 31053 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:57.682855 31076 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:57.691247 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 300576us
I20260709 12:17:57.601523 30824 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:57.525408 30478 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:57.698390 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 83653.1us
I20260709 12:17:57.652907 31152 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:57.705965 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:57.713344 28332 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:57.726750 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:17:57.739621 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.708503 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:57.744344 31131 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:17:57.744788 31175 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:17:57.746310 31131 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:57.669342 31097 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:57.752394 28437 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:57.660480 31165 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:57.755541 28440 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:57.755475 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.756810 28441 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:57.757337 28436 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:57.715682 30829 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:57.737428 31152 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:57.725687 31124 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:17:57.764046 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:17:57.787416 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.791407 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.792305 28441 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:57.827579 28436 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:57.825340 28990 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:57.861297 28433 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:57.853536 31069 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:57.795822 31143 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.879030 28441 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:57.878908 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:57.880353 28440 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:17:57.766983 28291 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:17:57.882812 28291 pending_rounds.cc:85] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 2
I20260709 12:17:57.883105 28291 pending_rounds.cc:107] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:57.883369 28291 raft_consensus.cc:2889] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:57.860402 31085 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:57.869750 30956 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.827618 31152 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:57.922550 28358 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.002s	sys 0.017s
W20260709 12:17:57.925112 28358 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.002s	sys 0.017s
I20260709 12:17:57.885490 31188 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:57.890250 31085 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:17:57.926970 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.928810 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.947186 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.886338 31069 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:17:57.886034 31151 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:17:57.886999 31143 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:57.924054 31189 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:57.925668 31183 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:17:57.923825 30956 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:57.952178 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.974184 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1671) took 1097 ms. Trace:
I20260709 12:17:57.988657 31076 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:57.988950 31085 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:57.988662 31081 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:17:57.950706 31190 raft_consensus.cc:2764] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
W20260709 12:17:58.006428 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 306258us
I20260709 12:17:58.005234 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:58.008502 28435 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:58.007968 28441 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:58.009442 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.974323 31142 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:57.985512 28289 rpcz_store.cc:276] 0709 12:17:56.876786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.876937 (+   151us) service_pool.cc:224] Handling call
0709 12:17:56.886248 (+  9311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:56.905979 (+ 19731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.906009 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:56.920285 (+ 14276us) log.cc:844] Serialized 26 byte log entry
0709 12:17:57.394812 (+474527us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b1800417a60. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:17:57.589042 (+194230us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:57.590531 (+  1489us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.591187 (+   656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.591660 (+   473us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.965825 (+374165us) raft_consensus.cc:1685] finished
0709 12:17:57.973751 (+  7926us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:57.974087 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138592,"prepare.queue_time_us":3844,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":98419,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":37107,"spinlock_wait_cycles":2493824,"thread_start_us":194033,"threads_started":1,"wal-append.queue_time_us":807866}
I20260709 12:17:57.974622 31189 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:58.024235 31085 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.025133 31124 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:58.026549 31124 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:58.026872 31123 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:17:58.018476 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:58.026937 31124 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:17:58.013995 31163 consensus_meta.cc:287] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.005s
I20260709 12:17:58.029971 31163 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:58.030414 31124 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:58.031862 31124 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:58.032279 31163 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:58.037329 28440 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:58.048230 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.949391 31188 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:58.037568 31172 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.702s	user 0.002s	sys 0.007s
W20260709 12:17:58.056694 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 330007us
I20260709 12:17:58.046361 31189 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:58.050038 28954 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:58.063010 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:58.080163 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 229780us
I20260709 12:17:58.101853 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:58.050868 31142 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:58.107671 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 236664us
I20260709 12:17:58.059286 31097 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:58.137473 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 351133us
I20260709 12:17:58.140587 28441 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:58.143038 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:58.166597 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:58.077275 30829 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:58.181100 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:58.164866 28238 auto_rebalancer.cc:893] Removing replace marker failed: Replica a3de34b6f4cf4606a61944db6e2739c8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:17:58.194700 28238 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 12:17:58.106623 31200 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:58.198827 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:17:58.200114 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 57208.9us
I20260709 12:17:58.204000 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:58.201653 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:58.209388 28436 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:58.210613 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:58.194329 30842 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:17:58.195062 28238 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:17:58.212811 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:17:58.196468 31102 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:17:58.241255 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.165495 31163 consensus_queue.cc:1059] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:17:58.208472 31100 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:17:58.208006 31151 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:58.252584 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 49688.4us
I20260709 12:17:58.212393 31053 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:58.195181 31140 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:17:58.239684 31191 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:58.202325 31052 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:17:58.239671 30842 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:58.264189 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 201272us
W20260709 12:17:58.265318 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 70123.1us
I20260709 12:17:58.292490 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:58.293133 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:17:58.265537 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63971.1us
I20260709 12:17:58.199527 31203 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 12:17:58.307077 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.252270 31102 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:58.306648 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:58.260615 31152 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:58.258811 31053 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:58.290241 31050 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.325166 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.259366 31209 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:58.299744 31200 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:58.256656 31142 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:58.321725 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 299) took 1426 ms. Trace:
I20260709 12:17:58.322348 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.318229 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 298) took 1429 ms. Trace:
I20260709 12:17:58.295296 30842 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:58.335780 28283 rpcz_store.cc:276] 0709 12:17:56.895178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.412664 (+517486us) service_pool.cc:224] Handling call
0709 12:17:57.428032 (+ 15368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.428246 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:57.453941 (+ 25695us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.852662 (+398721us) spinlock_profiling.cc:243] Waited 8.75 ms on lock 0x7b1800394240. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be42723a 00007f85be891a2a
0709 12:17:58.321372 (+468710us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:58.321424 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:58.321470 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.321535 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.321662 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25578,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":3931,"spinlock_wait_cycles":20119808,"thread_start_us":468418,"threads_started":2}
W20260709 12:17:58.330420 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 119924us
I20260709 12:17:58.290956 31100 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:58.292773 31143 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:58.317584 31165 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:58.356208 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:58.355995 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.548030 28366 raft_consensus.cc:2410] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:17:58.324906 31053 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:58.259579 31151 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:58.324551 31050 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:58.561184 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.562107 28366 raft_consensus.cc:2368] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:17:58.336032 28284 rpcz_store.cc:276] 0709 12:17:56.889055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.893789 (+  4734us) service_pool.cc:224] Handling call
0709 12:17:56.919602 (+ 25813us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:56.945699 (+ 26097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.945750 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:56.979583 (+ 33833us) log.cc:844] Serialized 52 byte log entry
0709 12:17:57.145898 (+166315us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.212530 (+ 66632us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.212787 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.993165 (+780378us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b540009ecd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:57.993218 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.035452 (+ 42234us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b18003941e0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb7682a6 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db
0709 12:17:58.036867 (+  1415us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:58.317962 (+281095us) raft_consensus.cc:1685] finished
0709 12:17:58.317989 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.318153 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118167,"prepare.queue_time_us":77793,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":11228,"raft.queue_time_us":11675,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":4466,"spinlock_wait_cycles":359477632,"thread_start_us":98976,"threads_started":1,"wal-append.queue_time_us":340830}
W20260709 12:17:58.435552 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 119590us
I20260709 12:17:58.336800 30956 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:58.484266 31063 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:58.410961 31194 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.346s	user 0.000s	sys 0.001s
I20260709 12:17:58.542327 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.570142 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:58.570677 28364 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:17:58.542737 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:58.543341 28990 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 466) took 1598 ms (client timeout 1910 ms). Trace:
I20260709 12:17:58.321148 31067 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:17:58.319624 31163 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:17:58.321162 31201 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:58.354231 31215 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.156s
I20260709 12:17:58.544133 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.332058 30478 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:58.359395 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:58.548753 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.553611 31209 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.562884 30838 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.575767 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.543572 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:58.601765 28362 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
W20260709 12:17:58.552760 31161 consensus_meta.cc:287] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.974s	user 0.017s	sys 0.004s
I20260709 12:17:58.614269 31161 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:58.543252 31085 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.543403 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:58.583119 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:58.585306 28365 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:17:58.549284 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 256203us
I20260709 12:17:58.549201 31140 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:58.572420 28990 rpcz_store.cc:269] 0709 12:17:56.944503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.949047 (+  4544us) service_pool.cc:224] Handling call
0709 12:17:58.488078 (+1539031us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54006e19d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:58.543266 (+ 55188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1404811520}
I20260709 12:17:58.545001 31218 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:58.845219 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:58.586506 30478 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:58.590533 30942 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.591585 31101 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:58.592545 31165 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:17:58.592857 31079 consensus_meta.cc:287] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.003s	user 0.008s	sys 0.004s
I20260709 12:17:58.594120 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:58.594622 31151 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:58.594887 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.595837 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:58.596597 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.598105 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:58.598728 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.576665 31102 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:58.543933 31152 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:58.568833 31050 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.605141 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:58.600760 30838 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:58.671584 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.854099 30478 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:17:58.662052 31213 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.146s	user 0.006s	sys 0.000s
I20260709 12:17:58.662092 31211 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:58.692816 28435 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:58.664868 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 305568us
I20260709 12:17:58.856820 31203 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:58.662249 31198 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.320s	user 0.000s	sys 0.006s
W20260709 12:17:58.662506 31193 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.421s	user 0.000s	sys 0.010s
W20260709 12:17:58.662173 31192 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.386s	user 0.002s	sys 0.004s
W20260709 12:17:58.662207 31204 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.263s	user 0.004s	sys 0.002s
I20260709 12:17:58.620605 30824 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:58.657127 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.666891 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.662132 31196 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.700416 31195 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.693722 31188 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:58.838573 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:58.793972 31200 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:58.842839 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:58.587261 31201 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:58.850397 28365 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.611626 31209 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:58.918548 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.862599 28366 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:17:58.862789 28360 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:17:58.863380 28359 raft_consensus.cc:2410] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:17:58.862807 28367 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:17:58.922003 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:58.662752 31197 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.373s	user 0.000s	sys 0.007s
W20260709 12:17:58.922904 31221 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:17:58.923233 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.924357 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:58.925321 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.876714 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:58.918080 31196 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:58.926455 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49848.5us
I20260709 12:17:58.926673 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 281) took 1733 ms. Trace:
I20260709 12:17:58.927690 31195 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:58.929912 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:58.930828 28433 raft_consensus.cc:2410] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:58.884379 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1682) took 1809 ms. Trace:
I20260709 12:17:58.934079 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.934324 31161 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:58.934903 28360 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:58.934788 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:58.935209 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.936122 28360 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:58.927089 28358 rpcz_store.cc:276] 0709 12:17:57.193302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.193507 (+   205us) service_pool.cc:224] Handling call
0709 12:17:57.193954 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.216856 (+ 22902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.216899 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:57.325378 (+108479us) log.cc:844] Serialized 26 byte log entry
0709 12:17:57.944322 (+618944us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.944376 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.947672 (+  3296us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.926465 (+978793us) raft_consensus.cc:1685] finished
0709 12:17:58.926490 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.926615 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":674,"raft.queue_time_us":921,"raft.run_cpu_time_us":6866,"raft.run_wall_time_us":112380,"thread_start_us":547587,"threads_started":1,"wal-append.queue_time_us":681378}
W20260709 12:17:58.907258 31210 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.349s	user 0.000s	sys 0.001s
I20260709 12:17:58.867805 31165 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:58.868214 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:58.855611 31205 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:17:58.911311 28434 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:58.857542 30942 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:58.917554 28363 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
W20260709 12:17:58.922135 31220 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 12:17:58.851315 31140 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:58.907397 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:58.857112 31085 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:58.979208 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:58.859992 31101 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:58.933652 28288 rpcz_store.cc:276] 0709 12:17:57.074967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.559658 (+484691us) service_pool.cc:224] Handling call
0709 12:17:57.560117 (+   459us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:57.578544 (+ 18427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.578615 (+    71us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:57.579227 (+   612us) log.cc:844] Serialized 52 byte log entry
0709 12:17:57.894831 (+315604us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:57.908725 (+ 13894us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.169823 (+261098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.169900 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.884140 (+714240us) raft_consensus.cc:1685] finished
0709 12:17:58.884163 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.884311 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13123,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":53,"raft.queue_time_us":88225,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":195,"thread_start_us":562363,"threads_started":2,"wal-append.queue_time_us":427000}
I20260709 12:17:58.948101 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:58.949379 28367 raft_consensus.cc:2410] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:58.950407 31067 consensus_queue.cc:1059] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:58.951169 31046 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:17:58.952616 30842 consensus_meta.cc:287] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.003s
I20260709 12:17:58.953503 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1676) took 2010 ms. Trace:
I20260709 12:17:58.866125 31079 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:58.955579 28365 raft_consensus.cc:2410] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:17:58.942387 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:58.873991 31152 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:58.961587 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:17:58.966041 31223 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:58.966995 31136 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:58.975062 28987 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.975684 28358 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:58.943285 31200 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.039485 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 341) took 1137 ms. Trace:
I20260709 12:17:59.050715 28291 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:59.029505 28989 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:59.023075 31085 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:17:58.960058 31124 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.928s	user 0.015s	sys 0.001s
I20260709 12:17:59.013906 31163 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:59.046198 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:59.034569 28284 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:58.899639 31203 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:17:59.146867 31054 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.210273 31054 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.145272 31131 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.211679 31203 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.027267 30942 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:59.025827 31123 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.998s	user 0.010s	sys 0.003s
I20260709 12:17:59.028167 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 469) took 1362 ms. Trace:
I20260709 12:17:59.030072 31195 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.211048 28440 rpcz_store.cc:276] 0709 12:17:57.901632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.901766 (+   134us) service_pool.cc:224] Handling call
0709 12:17:57.902112 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.037914 (+135802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.039199 (+1001285us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400347454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:59.039408 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310436096}
I20260709 12:17:59.025139 30838 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:59.192974 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 250651us
I20260709 12:17:59.028923 31196 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.035670 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:59.032974 31201 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:58.887636 31211 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.022315 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 331) took 1322 ms. Trace:
I20260709 12:17:59.031854 31140 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.037345 28363 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:17:58.975005 31193 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:17:58.971881 31188 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:59.033684 31081 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:59.038921 31209 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.030162 30829 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:58.975227 31198 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:17:59.151137 28404 leader_election.cc:400] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:17:58.958488 31052 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:59.282202 30838 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.156526 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:17:59.211632 31131 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:17:59.010532 31212 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.601s	user 0.006s	sys 0.015s
I20260709 12:17:59.175843 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:59.177099 31116 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:59.157572 28286 rpcz_store.cc:276] 0709 12:17:56.942912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.434847 (+491935us) service_pool.cc:224] Handling call
0709 12:17:57.435171 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:57.492653 (+ 57482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.493039 (+   386us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:57.495549 (+  2510us) log.cc:844] Serialized 52 byte log entry
0709 12:17:57.841650 (+346101us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b1800170380. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:17:58.258169 (+416519us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:58.303085 (+ 44916us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.579494 (+276409us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.585716 (+  6222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.585814 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.949934 (+364120us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b1800170380. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:58.953244 (+  3310us) raft_consensus.cc:1685] finished
0709 12:17:58.953270 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.953431 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117819,"prepare.queue_time_us":43648,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":96,"raft.queue_time_us":166846,"raft.run_cpu_time_us":606,"raft.run_wall_time_us":64434,"spinlock_wait_cycles":51428480,"thread_start_us":690823,"threads_started":2,"wal-append.queue_time_us":823251}
I20260709 12:17:59.021960 28982 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:17:59.299404 31140 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.310207 31140 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.177848 30842 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.022904 31165 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.190292 28359 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.215078 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:17:59.191529 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.360581 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.312235 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 155773us
I20260709 12:17:59.214210 28292 raft_consensus.cc:2410] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:17:59.250814 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:59.192323 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:59.362310 28440 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:17:59.266610 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:59.266674 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:59.267344 28330 connection.cc:256] RPC call timeout handler was delayed by -1.1412364640000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2631s for a response
W20260709 12:17:59.362728 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 112034us
I20260709 12:17:59.361949 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:59.267428 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:59.365388 28362 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
W20260709 12:17:59.181530 31232 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.160s	user 0.000s	sys 0.003s
I20260709 12:17:59.212781 28985 rpcz_store.cc:276] 0709 12:17:57.665705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.665831 (+   126us) service_pool.cc:224] Handling call
0709 12:17:57.674571 (+  8740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.860381 (+185810us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:57.896231 (+ 35850us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.254716 (+358485us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:58.254771 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:58.254820 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.027897 (+773077us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b5400660154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:59.027959 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.028095 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24112,"prepare.queue_time_us":25432,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":95137,"spinlock_wait_cycles":248844288,"thread_start_us":333705,"threads_started":2,"wal-append.queue_time_us":438737}
I20260709 12:17:59.274906 28437 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:59.188764 31218 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.111s
W20260709 12:17:59.214393 31053 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.003s
I20260709 12:17:59.187762 31225 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:59.259594 31046 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:59.208925 30990 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.179628 31101 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.259802 31163 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:59.259272 31099 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:59.287921 28285 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:17:59.186036 31227 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:59.293198 28404 leader_election.cc:403] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:59.274611 31203 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.154963 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:59.306080 28367 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:17:59.208528 31223 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.306558 30838 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.425444 30838 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.286293 31236 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:59.312441 31140 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:59.356190 31206 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.064s	user 0.002s	sys 0.000s
I20260709 12:17:59.426376 31236 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.356808 28983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 328) took 2501 ms. Trace:
I20260709 12:17:59.428862 31236 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.429278 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 339) took 1719 ms. Trace:
I20260709 12:17:59.429675 28439 rpcz_store.cc:276] 0709 12:17:57.710204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.713865 (+  3661us) service_pool.cc:224] Handling call
0709 12:17:57.714292 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.714562 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.714587 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:57.730307 (+ 15720us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.049250 (+318943us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:58.049296 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.049375 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:59.355554 (+1306179us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:17:59.428956 (+ 73402us) raft_consensus.cc:1685] finished
0709 12:17:59.428982 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.429208 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7653632,"thread_start_us":272711,"threads_started":1,"wal-append.queue_time_us":458041}
I20260709 12:17:59.357585 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:59.358615 28991 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:17:59.276434 28986 rpcz_store.cc:276] 0709 12:17:57.699527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.714733 (+ 15206us) service_pool.cc:224] Handling call
0709 12:17:57.715121 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:58.023803 (+308682us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:58.104473 (+ 80670us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.863454 (+758981us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:58.948690 (+ 85236us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.966654 (+ 17964us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:58.966698 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.966804 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.972656 (+  5852us) raft_consensus.cc:1685] finished
0709 12:17:58.975968 (+  3312us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.022208 (+ 46240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6616,"prepare.queue_time_us":58426,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":123,"raft.queue_time_us":170372,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":10499,"thread_start_us":890086,"threads_started":3,"wal-append.queue_time_us":491500}
I20260709 12:17:59.432323 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:59.433058 28363 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:17:59.434298 28363 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:59.366623 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:17:59.436331 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 69808.6us
I20260709 12:17:59.292816 31081 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.437968 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:17:59.152140 31205 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.438829 31200 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:59.439256 31200 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:59.288883 30829 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:17:59.440500 30478 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.000s
I20260709 12:17:59.440632 30829 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.441031 30478 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.268431 31050 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.374511 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:17:59.375200 31189 consensus_meta.cc:287] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.278s	user 0.013s	sys 0.000s
I20260709 12:17:59.471195 31189 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:17:59.376739 31239 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:17:59.475703 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:59.478083 28290 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:59.307509 31196 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:59.275063 31136 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.386495 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:59.387929 31163 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:59.382826 30811 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:17:59.389063 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 407892us
I20260709 12:17:59.389523 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1644) took 2552 ms. Trace:
I20260709 12:17:59.391281 28360 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 8. Ops: []
I20260709 12:17:59.359575 31116 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.289435 31188 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.483331 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:59.429587 31236 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.369194 31195 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.405006 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:59.406913 28989 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:59.407114 31227 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.299682 31124 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:17:59.398965 28359 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:59.418620 31079 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.420509 31131 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.422926 31211 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.391713 31046 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:17:59.361675 31161 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:59.439709 31200 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.441924 30829 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:59.367892 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1665) took 1927 ms. Trace:
I20260709 12:17:59.439201 31205 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.443603 30842 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.385860 31123 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.305199 31052 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.384903 31053 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.400679 30942 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:59.428018 28983 rpcz_store.cc:276] 0709 12:17:56.855223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.868860 (+ 13637us) service_pool.cc:224] Handling call
0709 12:17:56.905248 (+ 36388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:56.964034 (+ 58786us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:57.310157 (+346123us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.941861 (+631704us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:57.986753 (+ 44892us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.013361 (+ 26608us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:58.055050 (+ 41689us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.079606 (+ 24556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.103207 (+ 23601us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:59.312615 (+1209408us) raft_consensus.cc:1685] finished
0709 12:17:59.356527 (+ 43912us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.356709 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":148767,"prepare.queue_time_us":101490,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":163,"raft.queue_time_us":229945,"raft.run_cpu_time_us":486,"raft.run_wall_time_us":123315,"thread_start_us":657993,"threads_started":2,"wal-append.queue_time_us":672541}
I20260709 12:17:59.439795 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:59.449201 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:59.398694 28289 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:59.383226 31209 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:59.450053 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:59.550359 28367 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:17:59.488395 30811 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:17:59.461051 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.454809 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:59.391999 31228 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:59.383878 28990 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:17:59.481593 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 95179.8us
I20260709 12:17:59.582949 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:59.583595 28434 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:59.584390 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:59.421504 31152 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:58.544906 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:59.391785 31191 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:17:59.483100 28361 rpcz_store.cc:276] 0709 12:17:56.831927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.835328 (+  3401us) service_pool.cc:224] Handling call
0709 12:17:56.835660 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:56.989197 (+153537us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:17:57.381948 (+392751us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b180045db40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85c227f3c7 00007f85be8919a7 00007f85be8a10bf 00007f85be8a1038
0709 12:17:57.605704 (+223756us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.715468 (+109764us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b180045db40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:17:57.973227 (+257759us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:57.987455 (+ 14228us) log.cc:844] Serialized 52 byte log entry
0709 12:17:57.987666 (+   211us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:57.988229 (+   563us) log.cc:844] Serialized 14 byte log entry
0709 12:17:57.988457 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.022348 (+1033891us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54001d8154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:59.022399 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:59.383669 (+361270us) raft_consensus.cc:1685] finished
0709 12:17:59.383692 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.383857 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101026,"prepare.queue_time_us":73280,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":84,"raft.queue_time_us":49014,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":11152,"spinlock_wait_cycles":291501824,"thread_start_us":513714,"threads_started":2,"wal-append.queue_time_us":529017}
I20260709 12:17:59.399765 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:59.591046 28362 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:59.487076 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:59.487608 28366 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:59.490271 28291 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:17:59.448623 31203 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.482755 31140 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.482893 31196 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:17:59.388916 31225 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.402722 31099 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.500717 28432 rpcz_store.cc:276] 0709 12:17:57.440548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.450393 (+  9845us) service_pool.cc:224] Handling call
0709 12:17:57.501320 (+ 50927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.892875 (+391555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.892909 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:57.905400 (+ 12491us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.258484 (+353084us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:58.258528 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.258604 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:59.010158 (+751554us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b2400214568. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6209c4 00007f85bb61eacb 00007f85bb7d9dc9 00007f85bb7d9a55 00007f85bb7d909d 00007f85be40b10e 00007f85be42288c 00007f85be421bb6 00007f85be4217d8 00007f85be42b221
0709 12:17:59.358533 (+348375us) raft_consensus.cc:1685] finished
0709 12:17:59.358558 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.367818 (+  9260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":378696,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":3084,"spinlock_wait_cycles":17964288,"thread_start_us":666608,"threads_started":2,"wal-append.queue_time_us":349783}
I20260709 12:17:59.489336 31189 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.624300 28362 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:17:59.507938 28288 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:59.490191 31190 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:59.575764 28365 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:59.456615 28987 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:17:59.516990 31101 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:59.574095 28439 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:59.500202 31161 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.643118 31161 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.591547 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:17:59.644287 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52830.2us
I20260709 12:17:59.646096 31140 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:59.594082 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 107063us
I20260709 12:17:59.649065 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:17:59.650923 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 193981us
I20260709 12:17:59.586303 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:59.596053 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:59.522666 30942 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:17:59.652271 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66076.3us
I20260709 12:17:59.652693 30942 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.653528 31085 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:59.550041 31079 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:17:59.628732 31076 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:59.629155 28362 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:59.629498 30838 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:17:59.623929 31241 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:17:59.633098 28440 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.493876 31236 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.639420 31191 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:59.655165 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:59.593336 31123 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.582439 31209 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.653901 30942 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.630857 31203 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.598435 31195 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.603379 31196 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.667690 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:59.608189 31152 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:59.655598 31190 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.707855 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:59.656734 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:59.590330 31228 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.661554 31189 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:59.702770 28988 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 12:17:59.710016 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 53364.3us
I20260709 12:17:59.713315 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:59.720458 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:17:59.661331 31151 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:17:59.669592 31218 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:17:59.666057 31079 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.708516 31203 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:59.698344 28440 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:17:59.763530 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:59.667002 31161 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:17:59.656941 31046 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.772390 31050 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.772922 28988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 385) took 1156 ms. Trace:
W20260709 12:17:59.764163 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 141957us
I20260709 12:17:59.711592 31190 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.706332 31152 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:59.715381 31188 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.699167 31052 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.683760 31191 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.714429 31189 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:17:59.713507 31081 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.769761 31123 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:59.781250 28988 rpcz_store.cc:276] 0709 12:17:58.616639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.655289 (+1038650us) service_pool.cc:224] Handling call
0709 12:17:59.772827 (+117538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:59.716243 31211 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.785265 30811 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:17:59.789199 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 134111us
I20260709 12:17:59.779011 31116 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:17:59.794853 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.795939 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:17:59.788805 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 131009us
I20260709 12:17:59.790998 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:59.783931 31152 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:59.806452 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.787132 31189 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.791468 30824 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.791888 31151 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:59.793423 31223 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.791888 31161 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:59.799515 31205 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.833479 31140 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.804049 31079 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.835088 31053 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.837603 31203 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:59.854590 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:17:59.855070 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.861588 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:59.843080 30838 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:59.861663 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:17:59.835387 31102 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:17:59.868269 31102 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:59.868700 31102 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:59.869351 31236 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.844103 31247 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.874468 31247 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.878142 30838 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.878387 31225 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.869572 31102 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.879900 30838 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:59.845132 31211 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:17:59.863551 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 77677.4us
I20260709 12:17:59.856498 31205 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.842900 31161 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:59.844605 31227 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.846616 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:59.841056 31151 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:17:59.905941 31151 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.863292 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:59.855890 31123 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:59.850234 31116 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:59.908094 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 62752.7us
I20260709 12:17:59.902973 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:59.911655 31123 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.907421 31211 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.877496 31247 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.921622 31116 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.923281 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1724) took 1326 ms. Trace:
I20260709 12:17:59.924765 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:59.925161 31236 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:17:59.947227 28284 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:17:59.925501 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1672) took 1072 ms. Trace:
I20260709 12:17:59.926502 31191 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:17:59.946357 28360 rpcz_store.cc:276] 0709 12:17:58.597302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.939985 (+342683us) service_pool.cc:224] Handling call
0709 12:17:58.940947 (+   962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.483310 (+542363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.922924 (+439614us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b54003ba1d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:59.923201 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1035203840}
I20260709 12:17:59.926760 31054 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.904580 31227 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:17:59.910781 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:17:59.927242 31228 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.911680 31248 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:59.927649 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.927974 28436 raft_consensus.cc:2410] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:59.928809 28332 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:17:59.930207 31151 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.931314 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1828) took 1789 ms. Trace:
W20260709 12:17:59.933226 31243 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 12:17:59.920912 31195 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:59.911895 31079 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.941906 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:59.943945 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1717) took 1343 ms. Trace:
I20260709 12:17:59.858372 31140 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.977113 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 289) took 1101 ms. Trace:
W20260709 12:17:59.949894 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 93683.2us
I20260709 12:17:59.980124 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:17:59.947871 28367 rpcz_store.cc:276] 0709 12:17:58.852885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.559838 (+706953us) service_pool.cc:224] Handling call
0709 12:17:59.560289 (+   451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:59.577411 (+ 17122us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:59.686333 (+108922us) log.cc:844] Serialized 14 byte log entry
0709 12:17:59.791132 (+104799us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:59.798999 (+  7867us) log.cc:844] Serialized 52 byte log entry
0709 12:17:59.799819 (+   820us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:59.799860 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.799945 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:59.925246 (+125301us) raft_consensus.cc:1685] finished
0709 12:17:59.925269 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.925438 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1884,"prepare.queue_time_us":206363,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":22355,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":17953,"thread_start_us":156065,"threads_started":2,"wal-append.queue_time_us":173563}
W20260709 12:17:59.981324 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.950114 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:17:59.982393 28360 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:17:59.955132 31163 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.955678 31116 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:17:59.976727 28364 rpcz_store.cc:276] 0709 12:17:58.595089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.604763 (+  9674us) service_pool.cc:224] Handling call
0709 12:17:59.936236 (+1331473us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b5400134a54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:17:59.938336 (+  2100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425681792}
W20260709 12:17:59.986645 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 44885.8us
I20260709 12:17:59.987370 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1678) took 1071 ms. Trace:
I20260709 12:17:59.987766 28364 rpcz_store.cc:276] 0709 12:17:58.916100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.986511 (+1070411us) service_pool.cc:224] Handling call
0709 12:17:59.986858 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.987049 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.987075 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:59.987118 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:59.987149 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.987201 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.987308 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:59.988585 31252 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:59.989022 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:17:59.913148 31205 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.989439 31252 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:59.990203 31252 consensus_queue.cc:1243] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer f526369c0d0743b192be6e205540a059 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:17:59.990686 31252 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:59.991109 31252 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:17:59.991698 31205 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.992005 31252 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:17:59.965057 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:59.993362 31252 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.965364 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:59.994048 28439 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.966981 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:17:59.995678 28441 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:17:59.995963 31190 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:17:59.998031 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:17:59.999269 31140 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.964681 28435 rpcz_store.cc:276] 0709 12:17:58.141943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.142134 (+   191us) service_pool.cc:224] Handling call
0709 12:17:58.142451 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.876253 (+733802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.931035 (+1054782us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54003ca854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:17:59.931241 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1265179008}
I20260709 12:17:59.988878 30829 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
W20260709 12:17:59.975761 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 65081.2us
I20260709 12:17:59.989157 31079 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:00.003574 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.013949 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1677) took 1124 ms. Trace:
I20260709 12:17:59.992559 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1679) took 1037 ms. Trace:
I20260709 12:18:00.014701 28364 rpcz_store.cc:276] 0709 12:17:58.955161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.991702 (+1036541us) service_pool.cc:224] Handling call
0709 12:17:59.992031 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.992210 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.992238 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:59.992280 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:59.992314 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.992363 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.992498 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:00.014498 28360 rpcz_store.cc:276] 0709 12:17:58.889619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.012949 (+1123330us) service_pool.cc:224] Handling call
0709 12:18:00.013326 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.013527 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.013555 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:00.013627 (+    72us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:00.013662 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.013718 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.013873 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:00.006431 31190 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:00.007701 31079 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.960639 30811 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:00.028195 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:00.029630 31191 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:00.025074 31257 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:00.034654 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.037480 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.035421 28438 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:18:00.026268 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:00.049539 28438 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
W20260709 12:18:00.051997 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 71957.1us
W20260709 12:18:00.053161 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49902.1us
W20260709 12:18:00.037724 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.957675 31248 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.038240 28435 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:00.025480 28439 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:00.048794 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:17:59.978163 31209 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:00.063642 28332 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:00.055977 28438 pending_rounds.cc:85] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 3
I20260709 12:18:00.089552 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.090404 28438 pending_rounds.cc:107] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:18:00.090698 28438 raft_consensus.cc:2889] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:00.090826 28439 raft_consensus.cc:2410] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:00.091825 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.093032 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.096350 28439 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 4: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:00.102669 31273 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.106421 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.110129 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:00.120306 28289 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:00.121675 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.112833 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.110528 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.048050 31067 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:00.129400 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.130354 28291 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:00.113902 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.066246 31228 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:00.117028 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:17:59.966899 31236 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:00.122017 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.052944 30811 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.123857 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.125864 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.064445 31248 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:00.127628 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.127867 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.131158 28284 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:00.132180 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.134754 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.156038 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.156646 28290 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:00.068437 31255 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:00.136421 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 54826.6us
W20260709 12:18:00.138036 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 111029us
I20260709 12:18:00.108067 31246 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:00.174940 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.141229 28441 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:00.157382 28289 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:00.156929 28286 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:18:00.181244 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.182690 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:00.164150 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48314.9us
I20260709 12:18:00.168306 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:00.162134 28439 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:00.207966 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.209837 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:00.232659 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 62148us
I20260709 12:18:00.183548 28289 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:00.234704 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:00.232307 31200 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.716s	user 0.013s	sys 0.000s
I20260709 12:18:00.266759 31200 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:00.107861 31076 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
W20260709 12:18:00.243798 28404 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.852s (SENT)
I20260709 12:18:00.267051 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:00.280841 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:00.270186 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:00.270498 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.281934 28286 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:18:00.238081 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:00.271296 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.284341 28289 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:00.272403 28434 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.117164 31209 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:18:00.158000 30811 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.302264 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:00.301519 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 198884us
I20260709 12:18:00.308645 28434 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:00.315418 28291 raft_consensus.cc:2410] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:18:00.336091 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.338531 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.342928 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:00.343721 28286 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.346282 31251 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:00.346943 31251 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:00.347375 31251 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:00.351454 31265 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:18:00.236294 31246 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:00.377687 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.278405 31123 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:00.166468 30990 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.175s
I20260709 12:18:00.152207 31067 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:00.235656 30838 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:00.142596 31273 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:00.366619 31190 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:00.769037 30995 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.639665 28286 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
W20260709 12:18:00.547669 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 377223us
I20260709 12:18:00.918860 30995 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:00.908013 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.920102 28289 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:00.907521 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.920872 31275 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:00.921873 31067 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:00.644440 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 371314us
W20260709 12:18:00.740469 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.924108 30838 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:00.568490 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.740443 31191 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:00.728544 31189 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.888s	user 0.013s	sys 0.001s
I20260709 12:18:00.926010 31277 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:00.926541 31277 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:00.926944 31189 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:00.909499 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.927433 31277 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:00.928706 31277 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.929162 31189 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:00.401150 31152 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.001s
I20260709 12:18:00.930228 31152 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:00.931722 28441 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:00.933686 31152 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:00.534871 31270 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:18:00.839986 31200 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:00.925678 31191 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:00.936626 31273 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:00.783128 31268 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:18:00.716078 31269 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 12:18:00.367072 31246 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.383491 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:00.580431 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.938536 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:01.002403 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:01.002485 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 64045us
W20260709 12:18:01.002871 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.001379 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:01.003240 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:01.006786 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.939150 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:00.939410 31211 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:00.917181 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 673360us
W20260709 12:18:00.916535 28439 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.008s	sys 0.000s
W20260709 12:18:00.971212 28957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.008s	sys 0.011s
W20260709 12:18:01.018018 28957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.009s	sys 0.011s
I20260709 12:18:00.973277 31191 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:00.974402 31267 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:18:01.015349 28404 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 1.875s (SENT) [suppressed 25 similar messages]
I20260709 12:18:00.941668 31163 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:00.969815 31189 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:00.941813 31067 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:18:01.027431 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.000128 31200 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.032912 28291 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:01.033363 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.044631 28290 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:01.018890 31275 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:01.018245 31277 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:01.053648 28404 leader_election.cc:336] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.987s (SENT)
I20260709 12:18:01.054481 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1723) took 2457 ms. Trace:
I20260709 12:18:01.054919 28358 rpcz_store.cc:276] 0709 12:17:58.597018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.938048 (+341030us) service_pool.cc:224] Handling call
0709 12:17:58.938335 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.260428 (+322093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.054202 (+1793774us) spinlock_profiling.cc:243] Waited 37.1 ms on lock 0x7b540036c4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:01.054415 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85404544}
W20260709 12:18:01.055696 28288 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1693) took 2010 ms (client timeout 1818 ms). Trace:
W20260709 12:18:01.056093 28288 rpcz_store.cc:269] 0709 12:17:59.045379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.046118 (+   739us) service_pool.cc:224] Handling call
0709 12:18:01.055504 (+2009386us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54000affd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:01.055616 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330996352}
I20260709 12:18:01.054404 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:01.073778 28991 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 387) took 2438 ms (client timeout 2075 ms). Trace:
I20260709 12:18:01.028288 30990 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:01.020612 31223 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 12:18:01.081869 31116 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.027s	sys 0.062s
W20260709 12:18:01.094256 31116 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.027s	sys 0.062s
I20260709 12:18:01.047572 31276 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:01.051406 31277 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:01.099375 31101 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:01.099879 31277 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.034439 31236 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:01.106796 28291 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
W20260709 12:18:01.090505 28330 leader_election.cc:336] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.687s (SENT)
I20260709 12:18:01.107928 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:01.086184 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.109427 28288 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
W20260709 12:18:01.077489 28991 rpcz_store.cc:269] 0709 12:17:58.619170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.842579 (+223409us) service_pool.cc:224] Handling call
0709 12:18:01.056968 (+2214389us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54004ec354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:01.057048 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":824774400}
I20260709 12:18:01.078212 31191 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:01.121093 28330 leader_election.cc:336] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.691s (SENT)
I20260709 12:18:01.121570 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:01.122223 28330 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.695s (SENT)
I20260709 12:18:01.122332 28991 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 8 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:01.117048 28290 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
W20260709 12:18:01.123684 28330 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.722s (SENT)
I20260709 12:18:01.126052 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:01.126245 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.127120 28291 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:01.128225 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.128878 28291 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.129877 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:01.131574 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1860) took 1095 ms. Trace:
I20260709 12:18:01.132033 28435 rpcz_store.cc:276] 0709 12:18:00.036204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.061603 (+ 25399us) service_pool.cc:224] Handling call
0709 12:18:00.061998 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.062216 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:00.106807 (+ 44591us) log.cc:844] Serialized 14 byte log entry
0709 12:18:01.131193 (+1024386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:01.131256 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:01.131310 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.131383 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.131514 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":42017,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":9149,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":416016,"threads_started":2,"wal-append.queue_time_us":926708}
I20260709 12:18:01.134240 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:01.134860 28435 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:01.135751 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:01.099999 31101 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:01.093794 31285 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:18:01.131227 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:01.143707 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:01.145094 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:01.078608 31275 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:01.139077 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:01.108847 31248 consensus_meta.cc:287] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.871s	user 0.008s	sys 0.003s
W20260709 12:18:01.140998 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 87377us
I20260709 12:18:01.146476 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.149577 28288 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:01.078871 31282 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.043800 31189 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.125872 31052 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:01.048765 31200 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.140254 31123 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:01.159976 28291 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:01.168046 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1722) took 1007 ms. Trace:
I20260709 12:18:01.154302 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:01.089980 30990 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.168582 28291 rpcz_store.cc:276] 0709 12:18:00.160166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.127920 (+967754us) service_pool.cc:224] Handling call
0709 12:18:01.167957 (+ 40037us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:01.174460 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.174571 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.177448 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1729) took 1015 ms. Trace:
I20260709 12:18:01.177670 28290 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:01.177822 28288 rpcz_store.cc:276] 0709 12:18:00.162734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.158663 (+995929us) service_pool.cc:224] Handling call
0709 12:18:01.177379 (+ 18716us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:01.178272 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1726) took 1017 ms. Trace:
I20260709 12:18:01.178614 28290 rpcz_store.cc:276] 0709 12:18:00.161113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.154403 (+993290us) service_pool.cc:224] Handling call
0709 12:18:01.178212 (+ 23809us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31104}
I20260709 12:18:01.179803 28986 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:01.130107 31276 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.168937 31191 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.171272 31123 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.182283 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.183568 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1680) took 1095 ms. Trace:
W20260709 12:18:01.175309 28955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.011s	sys 0.010s
I20260709 12:18:01.190085 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.202271 28439 rpcz_store.cc:276] 0709 12:18:00.087653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.235726 (+148073us) service_pool.cc:224] Handling call
0709 12:18:00.236161 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:00.236421 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.236449 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:00.272040 (+ 35591us) log.cc:844] Serialized 26 byte log entry
0709 12:18:01.018002 (+745962us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:01.018046 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.018122 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.183257 (+165135us) raft_consensus.cc:1685] finished
0709 12:18:01.183282 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.183479 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":648412,"threads_started":1,"wal-append.queue_time_us":707372}
W20260709 12:18:01.203917 31279 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:18:01.206480 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.207088 28290 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:01.203722 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:01.204174 28984 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:18:01.181941 31196 consensus_meta.cc:287] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.466s	user 0.010s	sys 0.005s
W20260709 12:18:01.239530 28330 leader_election.cc:336] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.975s (SENT)
I20260709 12:18:01.158953 31248 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.210280 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1711) took 1052 ms. Trace:
I20260709 12:18:01.227950 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1846) took 1264 ms. Trace:
I20260709 12:18:01.202342 28291 raft_consensus.cc:2376] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8. Re-sending same reply.
I20260709 12:18:01.227190 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.227190 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:01.243997 28290 rpcz_store.cc:276] 0709 12:18:00.157387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.179291 (+1021904us) service_pool.cc:224] Handling call
0709 12:18:01.210198 (+ 30907us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:01.204999 31123 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:01.193363 31251 consensus_meta.cc:287] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.846s	user 0.012s	sys 0.000s
I20260709 12:18:01.245352 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:01.227830 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:01.200102 31283 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:01.182654 31116 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:01.233259 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 102102us
I20260709 12:18:01.248678 28439 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:18:01.185892 30990 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:01.205615 28955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.012s	sys 0.011s
I20260709 12:18:01.243417 28432 rpcz_store.cc:276] 0709 12:17:59.963241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.965453 (+  2212us) service_pool.cc:224] Handling call
0709 12:18:00.011214 (+ 45761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:00.011650 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.011682 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:00.029703 (+ 18021us) log.cc:844] Serialized 26 byte log entry
0709 12:18:00.318007 (+288304us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:00.318050 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.318118 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.227654 (+909536us) raft_consensus.cc:1685] finished
0709 12:18:01.227680 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.227868 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13487,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":198102,"threads_started":1,"wal-append.queue_time_us":943211}
I20260709 12:18:01.245379 31275 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.246994 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.249475 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:01.249521 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.162810 31282 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.205425 31191 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.230608 31189 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:17:59.977600 28362 rpcz_store.cc:276] 0709 12:17:58.875596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.659970 (+784374us) service_pool.cc:224] Handling call
0709 12:17:59.672296 (+ 12326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.672547 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:59.904127 (+231580us) log.cc:844] Serialized 14 byte log entry
0709 12:17:59.976677 (+ 72550us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:59.976758 (+    81us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:59.976815 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.976897 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.977041 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54682,"prepare.queue_time_us":112613,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":32865,"thread_start_us":225395,"threads_started":2}
W20260709 12:18:01.276473 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.278090 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 388) took 2585 ms. Trace:
I20260709 12:18:01.276976 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:01.248919 31200 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:01.275179 30995 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.293591 30838 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:01.295158 30838 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:01.292709 28990 rpcz_store.cc:276] 0709 12:17:58.692814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.850605 (+157791us) service_pool.cc:224] Handling call
0709 12:17:58.851059 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:59.582127 (+731068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.277791 (+1695664us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400660dd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:01.278018 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224685440}
I20260709 12:18:01.296990 31277 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.255743 31248 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.297955 31291 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:01.298449 31291 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.298820 31291 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.299041 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.299721 28289 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:18:01.300040 31191 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:01.258029 31116 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:01.302842 30838 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.300691 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.229228 31276 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.309973 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.268818 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:01.309446 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:01.311262 28439 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:01.312110 28289 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:01.313043 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.310731 28367 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:18:01.292604 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.318434 28290 raft_consensus.cc:2376] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6. Re-sending same reply.
I20260709 12:18:01.290120 28291 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:18:01.276590 31255 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.978s	user 0.010s	sys 0.000s
W20260709 12:18:01.271824 30811 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.906s	user 0.011s	sys 0.000s
I20260709 12:18:01.323436 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.250174 31251 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.246467 28989 raft_consensus.cc:2376] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4. Re-sending same reply.
I20260709 12:18:01.325953 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.327764 31257 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:01.328768 28292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1675) took 4409 ms (client timeout 2631 ms). Trace:
I20260709 12:18:01.331447 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.344156 28289 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:01.345410 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.346488 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.347111 28290 raft_consensus.cc:2376] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7. Re-sending same reply.
W20260709 12:18:01.343743 28292 rpcz_store.cc:269] 0709 12:17:56.919162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.222021 (+302859us) service_pool.cc:224] Handling call
0709 12:18:01.328503 (+4106482us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54003daed4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:01.328654 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4229080832}
I20260709 12:18:01.348480 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.245281 31196 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.342285 28288 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:18:01.359948 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1848) took 1396 ms. Trace:
I20260709 12:18:01.366070 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.368052 28288 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.369846 28439 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:18:01.370235 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:01.372143 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:01.335858 31200 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.337164 31255 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:01.363957 28433 rpcz_store.cc:276] 0709 12:17:59.963818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.965832 (+  2014us) service_pool.cc:224] Handling call
0709 12:18:01.107471 (+1141639us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b58001d4698. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:18:01.107696 (+   225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:01.108110 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:01.108140 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:01.146102 (+ 37962us) log.cc:844] Serialized 26 byte log entry
0709 12:18:01.245858 (+ 99756us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:01.245901 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.245974 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.352261 (+106287us) raft_consensus.cc:1685] finished
0709 12:18:01.352286 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.359860 (+  7574us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2809,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":11939,"spinlock_wait_cycles":4181888,"thread_start_us":67437,"threads_started":1,"wal-append.queue_time_us":88693}
W20260709 12:18:01.375097 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65045.7us
I20260709 12:18:01.253403 31283 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.377661 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:01.301638 31116 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.382303 28283 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:01.383524 28991 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:01.386264 28292 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:01.376437 28439 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:01.336372 31101 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.387022 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1714) took 1228 ms. Trace:
I20260709 12:18:01.347834 30811 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.322237 31246 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:01.360488 31257 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.397809 31282 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:01.401875 31246 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.402889 31251 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.422130 28360 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:01.422673 28288 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
W20260709 12:18:01.428014 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.459769 28433 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:18:01.460341 28433 pending_rounds.cc:85] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 6
I20260709 12:18:01.460588 28433 pending_rounds.cc:107] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:18:01.460850 28433 raft_consensus.cc:2889] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:01.465567 28288 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:01.466490 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1715) took 1308 ms. Trace:
I20260709 12:18:01.466815 28288 rpcz_store.cc:276] 0709 12:18:00.158415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.426641 (+1268226us) service_pool.cc:224] Handling call
0709 12:18:01.465304 (+ 38663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:01.466172 (+   868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.466418 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:01.468675 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 129253us
I20260709 12:18:01.380127 31200 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.467528 31123 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.496371 31131 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:01.497013 31131 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:01.498154 31099 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.498577 31277 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.500500 31101 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:01.503710 31295 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.504326 31295 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.388244 28292 rpcz_store.cc:276] 0709 12:18:00.158170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.375034 (+1216864us) service_pool.cc:224] Handling call
0709 12:18:01.386062 (+ 11028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.386737 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.386959 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:01.529390 28292 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:01.530746 28287 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:01.549628 28292 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:01.536732 31123 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.539106 31257 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:01.539763 31200 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.542183 31131 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.546823 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1813) took 3793 ms. Trace:
W20260709 12:18:01.547727 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 98037.6us
I20260709 12:18:01.515199 31211 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
W20260709 12:18:01.627331 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 157399us
I20260709 12:18:01.625007 28437 rpcz_store.cc:276] 0709 12:17:57.753836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.753952 (+   116us) service_pool.cc:224] Handling call
0709 12:17:57.754296 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.367264 (+1612968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.546512 (+2179248us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400417154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:01.546714 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3496663296}
I20260709 12:18:01.536897 31299 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.536023 31196 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.583317 31217 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.618098 31046 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:01.607620 31295 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.632054 31234 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:01.650143 31234 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.651100 30838 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.651583 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:01.652416 31234 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:01.639533 31299 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.641319 31191 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.655345 31191 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.656615 31299 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.652451 31140 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:01.645857 31217 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.659252 30838 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.633805 31131 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:01.646508 31283 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.647203 30811 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.634220 31189 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:01.648013 31046 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.658128 31219 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:18:01.659986 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.671945 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:01.657547 31191 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:01.661139 31217 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.670704 31099 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:01.671339 28292 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:01.672537 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:01.684654 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.690511 28292 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:01.678706 28364 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:01.679540 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:01.680384 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.672101 31219 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:01.701720 28358 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:01.673569 31189 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.683951 31277 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.681799 31282 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:01.683739 30842 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:01.682052 31246 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.719017 31246 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.705251 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:01.712968 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.688509 31191 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:01.726672 31297 raft_consensus.cc:1064] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:18:01.727535 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.727926 31257 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:01.657300 31161 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.690023 31099 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.728341 31257 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:01.716056 31046 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:01.716629 31211 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.723766 31136 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.726711 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.730973 28358 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:01.739117 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:01.739912 28358 raft_consensus.cc:2368] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:01.729535 31297 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.689597 31291 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.728358 31191 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.743281 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:01.744326 31219 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.749689 31234 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:01.751506 28285 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:01.751811 31219 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.753809 30478 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:01.755731 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.757524 28433 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:01.758427 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:01.733821 30842 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.759184 28437 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:01.760363 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:01.760707 30842 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:01.762176 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.730595 31282 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.763417 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.729269 31257 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.737360 31211 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:01.731271 31277 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:01.740674 28367 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:01.749172 31161 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:01.762053 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1670) took 2909 ms. Trace:
I20260709 12:18:01.813002 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:01.971097 28986 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:01.787990 28291 raft_consensus.cc:2368] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
W20260709 12:18:01.912811 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.970134 28359 rpcz_store.cc:276] 0709 12:17:58.852296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.497656 (+645360us) service_pool.cc:224] Handling call
0709 12:17:59.498070 (+   414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:59.498480 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.498517 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:59.500243 (+  1726us) log.cc:844] Serialized 52 byte log entry
0709 12:18:01.506432 (+2006189us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b180040dce0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:18:01.554724 (+ 48292us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:01.554781 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.554876 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.761692 (+206816us) raft_consensus.cc:1685] finished
0709 12:18:01.761717 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.761967 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1514,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":13121,"spinlock_wait_cycles":54177152,"thread_start_us":48020,"threads_started":1,"wal-append.queue_time_us":2169054}
I20260709 12:18:01.849756 28990 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:01.775708 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:01.974960 28437 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:01.811437 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:01.764529 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.982954 28364 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:01.746687 28358 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:01.761914 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:01.984882 28432 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:18:01.772027 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:01.995632 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 223698us
I20260709 12:18:01.997217 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:01.998783 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:01.999753 28367 raft_consensus.cc:2410] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:02.002345 28292 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:02.006151 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.009491 28364 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:02.011132 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.012133 28364 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:02.012966 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.014964 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.016234 28364 raft_consensus.cc:2368] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
W20260709 12:18:02.016918 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.764602 31282 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:02.018064 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.019398 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.020227 28359 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:02.008579 28956 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.029670 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:01.848488 31291 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:01.810391 31234 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:01.827396 31191 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:02.029844 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.034106 28990 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:02.031565 28986 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:01.851357 31189 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:01.827028 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.020566 31217 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:18:02.037493 28989 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:18:02.037041 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.020802 31136 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:02.039211 31136 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:02.039423 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.039661 31136 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.040038 28359 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:02.045389 31277 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:02.045296 31099 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:02.031023 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.051258 28288 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:02.052474 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.053303 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1721) took 1893 ms. Trace:
I20260709 12:18:02.053472 28983 raft_consensus.cc:2393] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:02.030669 31273 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.054558 31273 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.054952 31273 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.058761 31246 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:02.054241 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1669) took 3202 ms. Trace:
I20260709 12:18:02.051127 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1709) took 1140 ms. Trace:
I20260709 12:18:02.060942 28287 rpcz_store.cc:276] 0709 12:18:00.911068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.362109 (+451041us) service_pool.cc:224] Handling call
0709 12:18:02.050870 (+688761us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540018f454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.051049 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335793920}
I20260709 12:18:02.017555 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.096823 28359 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:02.060997 28288 rpcz_store.cc:276] 0709 12:18:00.159937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.467769 (+1307832us) service_pool.cc:224] Handling call
0709 12:18:02.051049 (+583280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.051770 (+   721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.053054 (+  1284us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b540018f450. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.053217 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1340482304}
I20260709 12:18:02.060526 28363 rpcz_store.cc:276] 0709 12:17:58.851965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.435052 (+583087us) service_pool.cc:224] Handling call
0709 12:17:59.435375 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:59.715931 (+280556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.715963 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:59.716010 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:59.716039 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.053994 (+2337955us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400103454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:02.054041 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:02.054176 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644671232}
I20260709 12:18:02.029161 31257 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.063108 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.103436 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.075711 28358 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:02.102088 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.105594 28991 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
W20260709 12:18:02.107178 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.110486 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.111334 28983 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:02.059659 31102 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:02.112493 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.113132 28359 raft_consensus.cc:2376] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7. Re-sending same reply.
W20260709 12:18:02.119033 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.026270 31283 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:18:02.125454 28364 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:18:02.098081 31231 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.379s	user 0.021s	sys 0.024s
I20260709 12:18:02.060128 31191 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:02.129583 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.131039 28363 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:02.133342 31211 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:02.126430 31231 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.419s	user 0.022s	sys 0.024s
I20260709 12:18:02.035651 31234 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:02.142779 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 331438us
I20260709 12:18:02.144784 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.149217 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:02.150460 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.143338 31136 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.150441 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:02.165855 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.167615 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:02.169723 28404 leader_election.cc:400] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:02.170035 28404 leader_election.cc:403] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:02.173336 28404 leader_election.cc:400] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:18:02.173671 28404 leader_election.cc:403] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:02.175081 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:02.180874 31203 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:02.181188 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:02.181532 31203 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:02.182619 31203 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.173146 31316 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.174062 31191 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.189459 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:02.197672 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:02.264468 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 75090us
I20260709 12:18:02.175666 31188 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
W20260709 12:18:02.268747 31264 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:18:02.185319 31189 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:02.183777 31287 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.281422 31188 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:02.279964 31318 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:18:02.292266 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.294225 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.293211 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.298578 28358 raft_consensus.cc:2376] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8. Re-sending same reply.
I20260709 12:18:02.299185 28364 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:02.299638 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.300179 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.300422 28358 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:02.301365 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.302632 28358 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:02.304013 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.304952 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.189256 31316 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.298517 28359 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:02.303506 31287 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.314015 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 290) took 3433 ms. Trace:
I20260709 12:18:02.314505 28365 rpcz_store.cc:276] 0709 12:17:58.881083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.907626 (+1026543us) service_pool.cc:224] Handling call
0709 12:17:59.924272 (+ 16646us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:59.931066 (+  6794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.931096 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:59.931708 (+   612us) log.cc:844] Serialized 52 byte log entry
0709 12:18:00.069549 (+137841us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:00.103476 (+ 33927us) log.cc:844] Serialized 14 byte log entry
0709 12:18:00.232167 (+128691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.234814 (+  2647us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.101421 (+866607us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b1800201400. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:02.182366 (+1080945us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b1800201400. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:02.313673 (+131307us) raft_consensus.cc:1685] finished
0709 12:18:02.313698 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:02.313931 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6328,"prepare.queue_time_us":34097,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":38392,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":217,"spinlock_wait_cycles":206059136,"thread_start_us":265700,"threads_started":2,"wal-append.queue_time_us":232566}
I20260709 12:18:02.310446 31316 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.309971 31234 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:02.310307 31188 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:02.318584 31273 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.320151 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.325368 28439 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:02.326527 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.327189 28433 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.325645 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 296) took 1017 ms. Trace:
I20260709 12:18:02.327852 28362 rpcz_store.cc:276] 0709 12:18:01.308711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.309795 (+  1084us) service_pool.cc:224] Handling call
0709 12:18:01.330319 (+ 20524us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:01.331670 (+  1351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:01.331699 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:01.343768 (+ 12069us) log.cc:844] Serialized 52 byte log entry
0709 12:18:01.489455 (+145687us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:01.689002 (+199547us) log.cc:844] Serialized 14 byte log entry
0709 12:18:01.690803 (+  1801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.325221 (+634418us) spinlock_profiling.cc:243] Waited 919 us on lock 0x7b54001d8b54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:02.325267 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:02.325323 (+    56us) raft_consensus.cc:1685] finished
0709 12:18:02.325432 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:02.325581 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1055,"prepare.queue_time_us":247208,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":12192,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":13949,"spinlock_wait_cycles":2113408,"thread_start_us":280561,"threads_started":2,"wal-append.queue_time_us":158128}
I20260709 12:18:02.332888 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.333407 28439 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.329874 31273 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:02.330768 31136 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:02.349051 28438 raft_consensus.cc:2410] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:18:02.351351 31163 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.352294 31163 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:02.350971 28438 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 349) took 1350 ms (client timeout 1762 ms). Trace:
W20260709 12:18:02.354298 28438 rpcz_store.cc:269] 0709 12:18:01.000801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.001020 (+   219us) service_pool.cc:224] Handling call
0709 12:18:02.350707 (+1349687us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400085f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.350879 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3093772160}
W20260709 12:18:02.358704 28434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 1849) took 2394 ms (client timeout 1649 ms). Trace:
W20260709 12:18:02.359143 28434 rpcz_store.cc:269] 0709 12:17:59.964105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.964233 (+   128us) service_pool.cc:224] Handling call
0709 12:18:02.358455 (+2394222us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54002e45d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.358585 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629365504}
I20260709 12:18:02.361495 28439 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
W20260709 12:18:02.364038 28440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 1842) took 2909 ms (client timeout 3080 ms). Trace:
W20260709 12:18:02.364498 28440 rpcz_store.cc:269] 0709 12:17:59.454349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.454533 (+   184us) service_pool.cc:224] Handling call
0709 12:18:02.363854 (+2909321us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b540026a2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.363968 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117764352}
I20260709 12:18:02.364832 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.367146 28440 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:02.368268 28439 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:02.370189 28988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 339) took 1261 ms. Trace:
I20260709 12:18:02.376458 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:02.379132 28988 rpcz_store.cc:276] 0709 12:18:01.109059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.110700 (+  1641us) service_pool.cc:224] Handling call
0709 12:18:01.140079 (+ 29379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.204664 (+ 64585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:01.204698 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:01.204743 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:01.204777 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.369959 (+1165182us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b54006e2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:02.370006 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:02.370129 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148036864}
I20260709 12:18:02.381212 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:02.382290 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.382951 28367 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.383569 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.384663 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.385026 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.385211 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.385555 28289 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:18:02.387331 31277 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:02.389676 31163 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:02.387153 31307 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:02.389952 31246 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:18:02.391263 31099 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:02.391629 31307 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.392096 31307 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:02.391220 31304 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.392987 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.393234 31304 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.386559 28287 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:02.385676 31069 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:02.394155 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.395464 28364 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:02.395612 31304 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:02.394824 31069 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:02.396950 31069 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.400394 31304 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:02.393641 28441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1683) took 2305 ms (client timeout 2192 ms). Trace:
I20260709 12:18:02.402132 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:02.402612 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.405830 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.406172 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:02.403748 28433 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:02.408818 28361 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 9. Ops: []
I20260709 12:18:02.408499 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.408460 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:02.409456 28433 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
W20260709 12:18:02.402074 28441 rpcz_store.cc:269] 0709 12:18:00.088409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.237790 (+149381us) service_pool.cc:224] Handling call
0709 12:18:02.393414 (+2155624us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b5400415354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.393560 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1489078144}
I20260709 12:18:02.410658 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:02.353629 31188 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.407490 31304 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.411862 31196 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:02.411854 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.407716 31299 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:02.413115 31299 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.413631 31304 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:02.414588 28366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1673) took 3555 ms (client timeout 1909 ms). Trace:
I20260709 12:18:02.415370 31299 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:02.415850 31304 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:02.416227 28434 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:02.415984 31322 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:02.416831 31234 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:02.417300 31234 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:02.415019 28366 rpcz_store.cc:269] 0709 12:17:58.853176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.934944 (+ 81768us) service_pool.cc:224] Handling call
0709 12:18:02.408383 (+3473439us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54003bb5d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.408534 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1268674304}
I20260709 12:18:02.417354 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.417797 31234 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:02.418995 28436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 346) took 2840 ms (client timeout 2866 ms). Trace:
I20260709 12:18:02.418352 28437 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:02.419350 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:02.420192 28283 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1702) took 2308 ms (client timeout 2152 ms). Trace:
I20260709 12:18:02.420552 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.420717 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.421181 28437 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.421873 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.418542 28434 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
W20260709 12:18:02.420584 28283 rpcz_store.cc:269] 0709 12:18:00.112195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.112312 (+   117us) service_pool.cc:224] Handling call
0709 12:18:02.420011 (+2307699us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54002418d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.420115 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817250048}
I20260709 12:18:02.423612 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.413076 28359 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:02.416229 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:02.425503 28438 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:02.425474 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:02.426678 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.427202 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:02.419574 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.428889 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.384462 31257 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.429973 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:02.429721 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:02.419414 28436 rpcz_store.cc:269] 0709 12:17:59.578494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.578644 (+   150us) service_pool.cc:224] Handling call
0709 12:18:02.418799 (+2840155us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b5400184f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.418937 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634116992}
I20260709 12:18:02.430845 28434 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:02.394728 31246 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:02.432024 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:02.434371 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:02.418460 31315 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.306s	user 0.003s	sys 0.000s
I20260709 12:18:02.416950 31322 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:02.434067 31257 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.435683 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.447991 31257 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.448472 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:02.449613 31229 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:02.451706 28434 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:18:02.453907 31291 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:02.455148 31124 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:02.455612 31124 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.455986 31124 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:02.406471 31161 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:18:02.462329 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:02.463114 28283 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.466529 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.467767 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:02.470446 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:02.470677 28437 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:02.470506 28367 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:02.476104 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:02.434681 31250 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.431035 31304 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:02.435346 31246 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:02.435174 31299 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.460987 31189 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:02.492091 31307 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.495651 31234 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.497448 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1729) took 3636 ms. Trace:
I20260709 12:18:02.497911 28361 rpcz_store.cc:276] 0709 12:17:58.861457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.601433 (+739976us) service_pool.cc:224] Handling call
0709 12:18:02.408632 (+2807199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.409241 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.497210 (+ 87969us) spinlock_profiling.cc:243] Waited 2.81 s on lock 0x7b54003bb5d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.497382 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6455381760}
I20260709 12:18:02.499393 28438 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:02.431216 31163 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:18:02.479897 31102 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:02.483927 31323 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:18:02.474504 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.486312 31131 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.507347 28283 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:02.482149 30995 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.483268 31250 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.482887 31283 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:02.506248 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51894.2us
W20260709 12:18:02.513687 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 75762.2us
I20260709 12:18:02.511543 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:02.489645 31189 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:02.517350 31189 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.517584 31081 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:02.518020 31081 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:02.518395 31081 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.523661 28257 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:02.508055 31102 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:02.526918 28440 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:02.532660 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.533193 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.534382 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.535260 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.535567 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.535065 28367 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:02.536206 28365 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:02.538281 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.538553 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.539067 28365 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:02.539446 28367 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:02.515955 31329 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:02.515368 30995 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.510649 31299 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:18:02.515057 28984 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:02.507519 31273 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:18:02.532013 31102 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.540503 31069 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.531443 31234 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:02.546782 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.569269 28366 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:02.525147 31163 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.548023 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:02.549378 28359 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:02.533380 31246 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.555474 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:02.553498 28291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1720) took 2393 ms (client timeout 3090 ms). Trace:
W20260709 12:18:02.573594 28291 rpcz_store.cc:269] 0709 12:18:00.159695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.345081 (+1185386us) service_pool.cc:224] Handling call
0709 12:18:02.553275 (+1208194us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54003d9fd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:02.553422 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009941248}
I20260709 12:18:02.574997 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.575850 28361 raft_consensus.cc:2368] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8.
I20260709 12:18:02.531693 31295 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:18:02.516489 31250 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.540210 31291 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.546208 30995 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.531219 31136 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:18:02.544145 31329 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:02.543172 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.590178 28361 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:02.590090 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1685) took 3221 ms. Trace:
I20260709 12:18:02.595809 28360 rpcz_store.cc:276] 0709 12:17:59.368549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.046948 (+678399us) service_pool.cc:224] Handling call
0709 12:18:00.047224 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.422676 (+1375452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.589822 (+1167146us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400127854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:02.590024 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3161460608}
I20260709 12:18:02.578438 31124 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.579193 31069 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.601541 31329 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.603016 31329 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.604153 31069 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:02.611539 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.612543 28367 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:02.613858 28366 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:02.615127 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.615903 28360 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:02.618263 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.618906 28364 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:02.619266 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.620144 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.520746 31322 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.621706 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.621958 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.623560 28362 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:02.625770 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:02.630486 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.626123 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:02.631201 28365 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:02.638149 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.639526 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.641454 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.639904 28364 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:02.643988 28360 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:02.588195 31291 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.647058 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:02.642143 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.648752 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.650218 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.651340 28437 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:02.653595 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.593859 31307 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:02.599716 31246 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:02.658308 28366 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:02.577466 31163 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.692836 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.670162 31203 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:02.617383 31081 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.671211 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.677577 28365 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:02.659125 31234 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:02.660663 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:02.687533 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.688819 28435 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:02.663502 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:02.646894 31291 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.682142 28330 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:02.697330 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 166633us
I20260709 12:18:02.594939 31316 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:02.700583 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 197210us
I20260709 12:18:02.670528 31287 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:02.698024 28433 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.660034 31326 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:02.680724 31277 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:02.671283 31191 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:02.703059 28437 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.707056 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.705750 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:02.722993 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.717105 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:02.733062 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 165687us
W20260709 12:18:02.735283 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 109625us
I20260709 12:18:02.746208 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.746874 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:02.751161 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:02.692106 31053 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:18:02.746987 28432 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.758049 28433 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:02.758313 28437 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:02.760251 28983 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:02.762223 28991 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 6 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:18:02.765765 28991 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:02.711994 31278 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:02.701164 31333 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:02.727190 31322 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:02.746954 31069 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.774534 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.793242 28991 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 3.
I20260709 12:18:02.772895 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.773208 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.782315 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:02.772967 31295 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:18:02.785282 31131 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:02.786180 31342 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:02.788558 31316 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:02.789535 31331 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:18:02.774993 31050 raft_consensus.cc:2764] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:18:02.786083 31102 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.770689 30990 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:02.824327 30990 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:02.825136 31320 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.544s	user 0.004s	sys 0.000s
I20260709 12:18:02.825362 30990 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.792744 31322 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.770308 31335 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:02.794437 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:02.732699 31277 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:02.754720 31195 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:02.803258 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 66552.1us
I20260709 12:18:02.778013 31278 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.795670 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:02.809056 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 145612us
I20260709 12:18:02.723537 31326 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.828994 31322 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.834213 31342 consensus_queue.cc:1243] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:02.834719 31335 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:02.836488 31337 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:02.795799 31340 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:02.843425 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.846287 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:02.825034 31231 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:02.855221 28435 raft_consensus.cc:2376] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8. Re-sending same reply.
W20260709 12:18:02.848407 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 84216.3us
W20260709 12:18:02.849357 28404 leader_election.cc:336] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.558s (SENT)
I20260709 12:18:02.856182 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:02.857225 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.574s (SENT)
I20260709 12:18:02.857518 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.857625 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:02.858199 28290 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 2.
I20260709 12:18:02.793913 31257 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:02.866039 31343 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:02.868356 31326 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:02.814878 28984 raft_consensus.cc:2376] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7. Re-sending same reply.
I20260709 12:18:02.845436 31195 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.850751 31278 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:02.859011 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:02.887567 28288 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:02.860790 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.888744 28435 raft_consensus.cc:2410] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:02.868017 30995 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:02.854218 28436 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:02.851806 28985 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:02.876060 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:02.853929 28983 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:02.845748 31277 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.858330 28434 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:02.882323 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:02.884366 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:02.852530 31081 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:02.891610 28432 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:02.896127 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.891040 28988 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:18:02.901990 28954 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.904605 28435 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 3: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:02.860268 31231 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:02.906517 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:02.866055 31163 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:02.885603 31195 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:02.909859 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 202874us
I20260709 12:18:02.910888 28292 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:02.909457 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:02.915161 28289 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.907697 28332 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:02.883347 31234 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:02.913517 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.911479 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.916512 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:02.921449 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:02.885463 31337 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:02.925974 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.926064 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:02.914439 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.919764 28288 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:18:02.928418 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:02.929546 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.929548 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:02.900311 31346 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:02.909022 31287 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:02.940510 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.940997 28285 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:02.930615 28287 raft_consensus.cc:2368] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:02.941802 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.952455 28290 raft_consensus.cc:2376] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8. Re-sending same reply.
I20260709 12:18:02.920519 31099 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:18:02.959954 31076 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:02.907742 31340 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:02.929492 31165 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:02.962666 28988 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1410767867a5470e81621556e44c6560"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:18:02.966579 28988 raft_consensus.cc:606] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:02.926041 31246 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:02.927027 31081 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:02.965433 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:02.979532 28288 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:02.968564 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.930619 31342 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.973598 28285 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:18:02.976156 28286 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:02.950316 31151 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:02.978065 28292 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:02.958690 31287 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.962220 31076 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.001888 28438 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:02.992048 31076 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.009758 28438 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:03.011534 31102 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.017367 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.018177 28286 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
W20260709 12:18:03.021883 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 107531us
W20260709 12:18:03.024257 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.985019 31165 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:03.009130 31250 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.044982 28288 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:02.990725 31287 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:02.992193 31050 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:02.984804 31246 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:03.053064 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.056161 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.057157 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.057346 31250 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:03.058388 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.058967 28288 raft_consensus.cc:2376] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5. Re-sending same reply.
I20260709 12:18:03.057348 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.059825 28286 raft_consensus.cc:2376] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8. Re-sending same reply.
I20260709 12:18:03.013849 31151 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:03.060803 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.060706 31246 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.062144 28286 raft_consensus.cc:2410] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:03.063122 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:03.062260 31351 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:18:03.059782 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:03.055387 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 171108us
I20260709 12:18:03.070521 28289 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:03.080356 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:03.071771 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 149243us
I20260709 12:18:03.082666 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:03.097285 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:03.099951 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:03.101531 28255 leader_election.cc:400] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8.
I20260709 12:18:03.102093 28255 leader_election.cc:403] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:18:03.050549 31355 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:18:03.104677 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.108513 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.109522 28288 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:03.118546 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:02.949118 31335 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.122228 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.126652 28288 raft_consensus.cc:2410] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:03.051535 31297 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:03.125329 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.129387 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.104086 31346 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:02.992877 31342 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.104519 31347 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:03.108609 31336 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:03.135812 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.137955 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:03.138331 31350 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:18:03.138535 31211 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:03.139374 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.123208 31196 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:03.153934 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 55377.6us
I20260709 12:18:03.154860 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:03.032760 31234 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.156795 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.159902 28288 raft_consensus.cc:2376] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9. Re-sending same reply.
I20260709 12:18:03.131040 31257 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:03.157684 28289 raft_consensus.cc:2376] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9. Re-sending same reply.
I20260709 12:18:03.166383 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.168782 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:03.169567 28436 raft_consensus.cc:2376] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8. Re-sending same reply.
I20260709 12:18:03.183743 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.132642 31354 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:03.190629 31354 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:03.191567 31354 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.169680 31165 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.187134 31329 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.196707 31329 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.189405 31099 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:03.200655 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.206590 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.208778 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.149523 30995 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.022851 31337 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.213212 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.220420 28286 raft_consensus.cc:2368] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:18:03.170806 31346 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:03.222325 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:03.196091 31189 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.001s
I20260709 12:18:03.223589 28433 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:03.136880 31297 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.097182 31340 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.137080 31356 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:18:03.207036 31099 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.238714 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:03.239125 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:03.224516 31350 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:18:03.244724 28361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:03.246016 28365 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.242705 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.261271 28361 raft_consensus.cc:606] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:03.261883 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:03.261948 28286 raft_consensus.cc:2368] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11.
I20260709 12:18:03.198939 31151 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.267135 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:03.267761 31364 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:03.268468 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.269381 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:03.225008 31346 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:03.283113 28286 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:03.285262 28365 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:03.288540 31069 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:03.289135 31069 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.240621 31329 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:03.245587 31099 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.289436 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.296275 28291 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:03.293012 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.303746 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.313829 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:03.330948 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 228159us
I20260709 12:18:03.328590 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.236660 31189 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.394695 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:03.262724 31277 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.406425 28437 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:03.413177 28330 leader_election.cc:336] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.543s (SENT)
W20260709 12:18:03.420162 31363 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:18:03.423894 31124 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.400122 31257 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:03.425350 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:03.494866 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 81786.1us
I20260709 12:18:03.485105 28286 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:03.511014 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.515255 31196 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8.
I20260709 12:18:03.521770 31347 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:18:03.394490 30995 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:18:03.497129 31356 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.677433 30995 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:03.678134 30995 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:03.679044 30995 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.680284 30995 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.680964 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:03.679980 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.681893 28284 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:03.682704 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.683220 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:03.683805 28364 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.683970 31333 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:03.684361 31333 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:03.686836 30990 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.687358 30990 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:03.610664 31367 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
W20260709 12:18:03.609766 31360 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 12:18:03.687784 31231 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.402292 31346 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.688846 31231 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.688927 31346 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:03.618551 31373 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 12:18:03.628547 31195 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.699s	user 0.006s	sys 0.003s
I20260709 12:18:03.648312 31189 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:03.651494 28983 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.748s	user 0.010s	sys 0.000s
I20260709 12:18:03.656301 31277 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:03.657603 31291 consensus_meta.cc:287] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.944s	user 0.020s	sys 0.007s
I20260709 12:18:03.555696 28286 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:18:03.745200 28286 pending_rounds.cc:85] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 3
I20260709 12:18:03.745499 28286 pending_rounds.cc:107] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:18:03.685187 31333 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.644843 31069 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.382823 31336 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:18:03.671766 31342 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.671752 31297 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:03.673092 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 151393us
I20260709 12:18:03.673439 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:03.677383 31356 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:03.612237 31076 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.005s
I20260709 12:18:03.678721 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:03.762733 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 84134.1us
I20260709 12:18:03.764114 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:03.764851 28438 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:03.703918 31099 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.712580 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:03.719234 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.728060 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1700) took 1312 ms. Trace:
I20260709 12:18:03.775322 28441 rpcz_store.cc:276] 0709 12:18:02.415920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.421606 (+  5686us) service_pool.cc:224] Handling call
0709 12:18:03.727816 (+1306210us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54002e3bd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:03.727986 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660102016}
I20260709 12:18:03.689524 30990 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.745836 28286 raft_consensus.cc:2889] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:03.664994 28433 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:03.751439 31336 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:03.752446 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:03.747843 28364 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:03.742205 28983 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:03.780156 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.780821 28286 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.743942 31195 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:03.762019 31278 consensus_meta.cc:287] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.861s	user 0.019s	sys 0.009s
I20260709 12:18:03.786136 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:03.767311 31297 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:03.774416 31322 consensus_meta.cc:287] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.918s	user 0.004s	sys 0.010s
I20260709 12:18:03.776876 31342 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:18:03.747864 31291 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.789484 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:03.799690 31278 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.801725 31278 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:03.787863 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:03.774348 31333 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.801483 31291 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.793680 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.806834 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.807574 28292 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:03.808326 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.802085 28291 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:18:03.808426 31287 consensus_meta.cc:287] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.711s	user 0.011s	sys 0.005s
I20260709 12:18:03.808986 28366 raft_consensus.cc:2393] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:03.818924 28286 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 3.
I20260709 12:18:03.819954 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.781915 31336 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:03.802078 28436 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:03.809331 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 344) took 1054 ms. Trace:
I20260709 12:18:03.768913 31076 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.824173 28985 rpcz_store.cc:276] 0709 12:18:02.754993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.755339 (+   346us) service_pool.cc:224] Handling call
0709 12:18:02.774825 (+ 19486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.897049 (+122224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.809067 (+912018us) spinlock_profiling.cc:243] Waited 76.8 ms on lock 0x7b540071e654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:03.809262 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176525568}
I20260709 12:18:03.825019 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.825882 28286 raft_consensus.cc:2368] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:18:03.831535 28288 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:18:03.819810 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.832284 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.836824 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:03.822535 31287 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:03.838317 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 345028us
I20260709 12:18:03.838896 31287 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.839176 31278 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:03.839740 31278 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:03.832726 31136 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:03.819579 31322 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.840612 31278 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.841997 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.842671 31322 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:03.843375 28440 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:03.819391 31257 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:18:03.834321 28292 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
W20260709 12:18:03.839257 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.845057 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.846552 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.846659 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:03.847322 28292 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:03.847306 28441 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.849665 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.826454 31189 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.851445 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:03.852707 28292 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:03.850766 31188 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:03.872043 31188 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:03.872893 31195 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.873176 31188 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.874640 31188 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:03.878705 31069 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:03.886826 31356 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.852360 31277 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:18:03.892205 31076 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:03.865830 31322 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:03.893821 31322 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.896155 31322 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:03.867873 31189 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:03.908843 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 121076us
I20260709 12:18:03.910617 31189 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:03.910758 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:03.870312 31336 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.905260 31322 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:03.915082 31299 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:03.915473 31299 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:03.916292 31299 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.917553 31299 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:03.923002 31099 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:03.912927 31337 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:03.914479 31322 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:03.929626 31337 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:03.931303 31322 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:03.932348 31336 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.938314 31337 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:03.853544 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:03.983420 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 133053us
I20260709 12:18:03.981046 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:03.999986 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:03.994948 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.003302 28292 raft_consensus.cc:2368] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5.
I20260709 12:18:04.004558 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.007001 28292 raft_consensus.cc:2376] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6. Re-sending same reply.
I20260709 12:18:04.010007 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:04.011025 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:04.014854 31050 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:18:03.980950 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.016798 28432 raft_consensus.cc:2410] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:04.017009 28283 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:04.016348 28438 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.019068 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:04.020748 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:04.012323 28435 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:18:04.010972 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:04.022038 28432 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:04.024026 28436 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:04.024704 28436 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:04.025854 28435 pending_rounds.cc:85] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 6
I20260709 12:18:04.026196 28435 pending_rounds.cc:107] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:18:04.026480 28435 raft_consensus.cc:2889] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:04.027558 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.028148 28436 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.028790 28286 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:04.032146 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.033646 31136 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:04.033586 28436 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:04.037513 28290 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:04.042888 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:04.043780 28434 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:04.045513 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:04.046833 31231 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.047758 31231 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:04.049685 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.047000 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:04.062961 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.071069 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.071223 31340 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.073132 31299 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.074452 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:04.076089 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.074149 31299 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:04.077864 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.078608 31333 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.079514 28283 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:04.080353 31299 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:04.080786 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.079612 31333 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:04.086476 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.087472 28290 raft_consensus.cc:2376] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8. Re-sending same reply.
I20260709 12:18:04.088631 31333 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:04.088730 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.092013 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:04.114740 31394 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 12:18:04.088738 28284 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:18:04.116905 28290 raft_consensus.cc:2376] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7. Re-sending same reply.
I20260709 12:18:04.121176 28438 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:04.122956 31188 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.123570 31188 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.127548 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.128612 28290 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
W20260709 12:18:04.128826 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1884304299999258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260709 12:18:04.129346 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1866304299999229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
I20260709 12:18:04.134593 28436 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:18:04.135145 28436 pending_rounds.cc:85] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
I20260709 12:18:04.135244 28434 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:18:04.135416 28436 pending_rounds.cc:107] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:18:04.135848 28436 raft_consensus.cc:2889] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:04.136749 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:04.139048 28985 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:04.141155 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1602229380000608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 12:18:04.141608 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1515304300000935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 12:18:04.148638 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1065856150000855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 12:18:04.149034 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1031935670000621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
W20260709 12:18:04.149329 28404 connection.cc:256] RPC call timeout handler was delayed by -1.0940813150000395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
W20260709 12:18:04.157341 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.163569 28283 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:18:04.172784 28364 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:04.174479 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:04.175959 31380 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:04.176913 31380 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:04.178609 31380 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.182516 28364 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:04.184178 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:04.185086 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 331961us
I20260709 12:18:04.153491 31231 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:04.187397 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:04.190994 31211 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:04.191665 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:04.193475 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:04.194445 28362 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:04.186194 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:04.174744 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:04.198597 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.198738 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:04.199504 28956 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:04.200166 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:04.200348 28360 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.199532 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.201462 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.212566 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.214087 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.214301 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:04.215070 28985 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:04.215425 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.216768 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.217499 31287 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:04.218151 31287 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:04.219499 31287 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.221076 31287 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.221911 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:04.222939 28433 raft_consensus.cc:2410] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:04.223953 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.224174 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.224375 28433 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:04.224772 28286 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:04.225194 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.226229 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:04.226974 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.227867 31340 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:04.228295 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.228839 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1738) took 1043 ms. Trace:
I20260709 12:18:04.229254 28288 rpcz_store.cc:276] 0709 12:18:03.185274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.201678 (+ 16404us) service_pool.cc:224] Handling call
0709 12:18:04.228415 (+1026737us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b540009e2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:04.228628 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1428106112}
I20260709 12:18:04.230087 31335 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:04.230834 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.231493 28288 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:04.232604 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.233381 28288 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:04.234817 28436 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:04.236069 31283 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:04.238427 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.239437 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41314 (request call id 28) took 1016 ms. Trace:
I20260709 12:18:04.239784 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.240628 28291 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:04.244801 31282 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:04.245168 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 1883) took 1472 ms. Trace:
W20260709 12:18:04.245556 28404 connection.cc:256] RPC call timeout handler was delayed by -1.0895813150000322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 12:18:04.227202 28983 raft_consensus.cc:2368] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:04.247268 28292 raft_consensus.cc:2376] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9. Re-sending same reply.
I20260709 12:18:04.256119 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.217473 28290 raft_consensus.cc:2368] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:04.258198 28432 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:18:04.229717 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:04.239836 28361 rpcz_store.cc:276] 0709 12:18:03.234571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.234747 (+   176us) service_pool.cc:224] Handling call
0709 12:18:04.238769 (+1004022us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b540010caa0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be804520 00007f85be881652 00007f85c384f60a 00007f85bcdd57d4 00007f85bcdd5762 00007f85bcdd5691 00007f85bcdd560c 00007f85bcdd48c2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:04.239217 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15931776}
W20260709 12:18:04.263980 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46516.6us
I20260709 12:18:04.268432 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:04.255484 28437 rpcz_store.cc:276] 0709 12:18:02.772691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.772950 (+   259us) service_pool.cc:224] Handling call
0709 12:18:04.244748 (+1471798us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54003cc654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:04.244976 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1406980864}
I20260709 12:18:04.273345 31322 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:04.276861 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47363.1us
I20260709 12:18:04.277876 28291 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:04.278093 31211 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:04.278782 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1769) took 1143 ms. Trace:
I20260709 12:18:04.279228 28291 rpcz_store.cc:276] 0709 12:18:03.135286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.268420 (+1133134us) service_pool.cc:224] Handling call
0709 12:18:04.277548 (+  9128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.278450 (+   902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.278688 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:04.280758 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.285336 28360 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:04.275908 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:04.288672 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:04.295012 31322 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.299436 28439 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:04.301342 28404 leader_election.cc:336] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.681s (SENT)
I20260709 12:18:04.302155 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:04.302882 28404 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.700s (SENT)
W20260709 12:18:04.303619 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.304679 28404 leader_election.cc:336] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.706s (SENT)
I20260709 12:18:04.305168 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:04.278808 31211 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:04.307444 28283 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:04.308161 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1765) took 1179 ms. Trace:
I20260709 12:18:04.303009 28292 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:04.309684 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.309259 28441 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:04.312291 28257 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:04.314107 31131 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:04.314693 31131 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:04.315881 31131 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:04.317819 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:04.318761 28361 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:04.308525 31211 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.322564 31366 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.323328 31366 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:04.326660 28985 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:18:04.327445 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1767) took 1193 ms. Trace:
I20260709 12:18:04.328197 28290 rpcz_store.cc:276] 0709 12:18:03.134258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.265690 (+1131432us) service_pool.cc:224] Handling call
0709 12:18:04.326718 (+ 61028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.326984 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:04.327012 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:04.327086 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:04.327135 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.327189 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:04.327354 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:04.333159 28292 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 4.
I20260709 12:18:04.338321 31371 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:18:04.343380 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:04.344377 28983 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.308506 28283 rpcz_store.cc:276] 0709 12:18:03.128362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.263710 (+1135348us) service_pool.cc:224] Handling call
0709 12:18:04.307122 (+ 43412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:04.307846 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.308080 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:04.345942 31286 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:04.347028 28283 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:04.330585 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.351333 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:04.353042 28292 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:04.314153 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:04.372031 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 90494us
W20260709 12:18:04.364658 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 99426us
I20260709 12:18:04.348560 31322 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:04.372133 31196 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:04.372558 31346 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:04.373106 31286 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:04.362396 28361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:04.362529 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.404016 28291 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:04.383387 31161 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:04.385627 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1707) took 1951 ms. Trace:
I20260709 12:18:04.385980 28290 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:04.389906 28330 leader_election.cc:336] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.669s (SENT)
I20260709 12:18:04.394093 28292 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:04.405797 28439 rpcz_store.cc:276] 0709 12:18:02.434581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.837760 (+403179us) service_pool.cc:224] Handling call
0709 12:18:04.273321 (+1435561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.300024 (+ 26703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.385401 (+ 85377us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54003cc650. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:04.385546 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3239932800}
I20260709 12:18:04.403869 28361 raft_consensus.cc:606] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:04.418412 28361 raft_consensus.cc:614] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:04.417160 31286 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:04.397058 31196 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:04.426453 31196 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.427888 31196 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.429073 31188 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:04.465030 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 160388us
W20260709 12:18:04.466063 28404 leader_election.cc:336] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.723s (SENT)
I20260709 12:18:04.466595 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:04.474174 28283 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:18:04.717939 28283 pending_rounds.cc:85] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 3
I20260709 12:18:04.718936 28983 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:04.719503 28283 pending_rounds.cc:107] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:18:04.719282 28291 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:04.474737 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1728) took 1552 ms. Trace:
I20260709 12:18:04.720181 28283 raft_consensus.cc:2889] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:04.587874 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49992.4us
I20260709 12:18:04.559006 28984 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:18:04.721827 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.542593 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 192200us
W20260709 12:18:04.729007 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.729832 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.672811 31375 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 12:18:04.608518 28289 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1756) took 1556 ms (client timeout 1987 ms). Trace:
I20260709 12:18:04.727465 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:04.662900 31287 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.534624 31161 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.535720 31188 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:04.736080 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 270151us
W20260709 12:18:04.732909 28289 rpcz_store.cc:269] 0709 12:18:03.051609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.130891 (+ 79282us) service_pool.cc:224] Handling call
0709 12:18:04.543803 (+1412912us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b540009e7d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:04.608273 (+ 64470us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15400704}
I20260709 12:18:04.654044 31359 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.662452 28190 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 3 to 6, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 6 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:04.741845 31189 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:04.748111 31333 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:04.750716 28435 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:04.727708 31366 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:04.756549 28286 raft_consensus.cc:2410] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:04.758157 28436 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:18:04.757210 28404 leader_election.cc:336] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.783s (SENT)
I20260709 12:18:04.764747 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:04.764540 31397 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:18:04.772274 31304 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:04.778524 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 1887) took 1939 ms. Trace:
I20260709 12:18:04.770828 31189 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.779004 28440 rpcz_store.cc:276] 0709 12:18:02.839267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.842466 (+  3199us) service_pool.cc:224] Handling call
0709 12:18:02.842760 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.843745 (+1000985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.777683 (+933938us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400085f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:04.778353 (+   670us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300811136}
I20260709 12:18:04.776453 31304 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:04.776041 31188 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:04.780942 31304 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:04.781520 31322 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.766052 31395 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:18:04.726678 31286 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.783085 31304 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:04.799805 31189 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:04.813184 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 247858us
I20260709 12:18:04.721374 28287 rpcz_store.cc:276] 0709 12:18:02.911279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.962350 (+ 51071us) service_pool.cc:224] Handling call
0709 12:18:02.982058 (+ 19708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:03.161077 (+179019us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:03.198307 (+ 37230us) log.cc:844] Serialized 14 byte log entry
0709 12:18:03.239650 (+ 41343us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:03.425063 (+185413us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b18000e1d00. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:04.050227 (+625164us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b2c00032f88. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:18:04.051010 (+   783us) log.cc:844] Serialized 26 byte log entry
0709 12:18:04.051237 (+   227us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:04.133249 (+ 82012us) log.cc:844] Serialized 14 byte log entry
0709 12:18:04.147891 (+ 14642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.431008 (+283117us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400163d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:04.431058 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:04.431104 (+    46us) raft_consensus.cc:1685] finished
0709 12:18:04.431116 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:04.463291 (+ 32175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103780,"prepare.queue_time_us":66374,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":118,"raft.queue_time_us":54831,"raft.run_cpu_time_us":452,"raft.run_wall_time_us":24381,"spinlock_wait_cycles":427877504,"thread_start_us":120654,"threads_started":2,"wal-append.queue_time_us":154602}
I20260709 12:18:04.766891 31287 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.832417 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:04.834760 31404 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:18:04.842418 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.842770 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:04.844599 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:04.847064 28441 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:18:04.854832 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:04.855221 28360 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:04.855332 28367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
W20260709 12:18:04.856518 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.859844 28367 raft_consensus.cc:606] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:04.860180 31395 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.860466 28367 raft_consensus.cc:614] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:04.859845 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:04.862303 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 12:18:04.871613 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 114429us
I20260709 12:18:04.868311 28361 raft_consensus.cc:2376] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11. Re-sending same reply.
I20260709 12:18:04.876693 28359 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:04.876891 28366 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.877378 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:04.882252 31388 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:04.882362 31392 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:04.883069 31388 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:04.883361 31392 consensus_queue.cc:1243] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:04.885154 31388 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:04.901763 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:04.908402 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:04.910359 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:04.914757 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.915249 28366 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:04.917173 31391 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:04.920058 31395 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.917721 31391 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.947926 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:04.949028 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 69396.5us
I20260709 12:18:04.908702 31392 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:04.965199 28286 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:04.967504 28359 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:04.976599 28441 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:04.979053 28290 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:04.982303 31188 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:04.982630 28984 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:04.983530 28255 connection.cc:256] RPC call timeout handler was delayed by -1.3606267440002284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
W20260709 12:18:04.983963 28255 connection.cc:256] RPC call timeout handler was delayed by -1.2951568309999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
I20260709 12:18:04.957757 31391 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.957638 31392 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:05.002017 28287 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:05.007349 28361 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:05.004142 31196 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:05.008870 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.009601 28439 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:05.012116 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.013077 31366 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:05.014423 28983 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:05.014632 31380 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:05.015215 31380 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:05.016235 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.016412 31380 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:05.017500 31354 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:05.018872 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:05.020978 31286 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.021366 31286 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.021680 31286 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.025118 31346 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:05.025650 31346 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:05.026490 31392 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.026808 31346 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.028004 31392 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.009456 31304 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:05.031081 31304 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.005177 31287 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.033444 31304 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.033922 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1766) took 1903 ms. Trace:
I20260709 12:18:05.034596 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.045111 31124 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:05.035247 31151 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:05.035753 31283 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:05.036612 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:05.047153 28189 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 217) took 26871 ms (client timeout 15000 ms). Trace:
I20260709 12:18:05.037801 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:05.038935 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.047726 28366 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:05.038380 28190 catalog_manager.cc:5697] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 5 to 6, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 6 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:05.039994 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.040913 28286 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:05.041930 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:05.042140 28365 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:18:05.042388 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.008791 31196 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:05.043141 31403 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.410s	user 0.004s	sys 0.000s
I20260709 12:18:05.043449 28364 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:18:05.012410 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:05.053956 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 159456us
I20260709 12:18:05.057880 31333 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.058528 31333 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.060751 31333 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.060853 31196 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.062198 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:05.064313 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1735) took 2149 ms. Trace:
I20260709 12:18:05.065524 31161 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.044580 28288 rpcz_store.cc:276] 0709 12:18:03.130126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.264136 (+1134010us) service_pool.cc:224] Handling call
0709 12:18:04.391055 (+126919us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:04.391291 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:04.391603 (+   312us) log.cc:844] Serialized 14 byte log entry
0709 12:18:04.703358 (+311755us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:04.703430 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:04.703475 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.703548 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.033648 (+330100us) spinlock_profiling.cc:243] Waited 92.7 ms on lock 0x7b540005a0d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:05.033806 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26767,"prepare.queue_time_us":5025,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":32214,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":185,"spinlock_wait_cycles":213265792,"thread_start_us":281804,"threads_started":1,"wal-append.queue_time_us":360137}
I20260709 12:18:05.065235 28190 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:05.069816 28360 raft_consensus.cc:2410] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:18:05.073264 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.076346 28436 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:05.076813 28441 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:05.077507 28404 leader_election.cc:336] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.812s (SENT)
I20260709 12:18:05.028033 31346 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.084149 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.084885 28436 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:05.085811 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.089610 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.102269 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:05.047627 28189 rpcz_store.cc:269] 0709 12:17:38.175561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.175770 (+   209us) service_pool.cc:224] Handling call
0709 12:18:05.046917 (+26871147us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:54.966143 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:54.973802 (+  7659us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:54.973854 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:17:54.973887 (+    33us) tablet.cc:662] Decoding operations
0709 12:17:54.974779 (+   892us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:54.974880 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:54.974913 (+    33us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:17:54.975590 (+   677us) tablet.cc:701] Row locks acquired
0709 12:17:54.975641 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:17:54.975820 (+   179us) write_op.cc:270] Start()
0709 12:17:54.975988 (+   168us) write_op.cc:276] Timestamp: P: 1783599474975743 usec, L: 0
0709 12:17:54.976008 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:17:54.976733 (+   725us) log.cc:844] Serialized 2906 byte log entry
0709 12:17:55.194798 (+218065us) op_driver.cc:464] REPLICATION: finished
0709 12:17:55.674930 (+480132us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85bb77d0ac 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85c22bdef0
0709 12:17:55.798510 (+123580us) write_op.cc:301] APPLY: starting
0709 12:17:55.798624 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:55.798849 (+   225us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:55.798895 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:55.957327 (+158432us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:56.007371 (+ 50044us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:17:56.007437 (+    66us) write_op.cc:312] APPLY: finished
0709 12:17:56.143091 (+135654us) log.cc:844] Serialized 59 byte log entry
0709 12:17:56.143376 (+   285us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:56.143535 (+   159us) write_op.cc:454] Released schema lock
0709 12:17:56.143620 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:56.144188 (+   568us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:56.144222 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":55180,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":603372,"apply.run_cpu_time_us":5005,"apply.run_wall_time_us":345649,"num_ops":7,"prepare.queue_time_us":205088,"prepare.run_cpu_time_us":56637,"prepare.run_wall_time_us":152827,"raft.queue_time_us":46008,"raft.run_cpu_time_us":14301,"raft.run_wall_time_us":770378,"replication_time_us":218620,"spinlock_wait_cycles":597883904,"thread_start_us":958363,"threads_started":4,"wal-append.queue_time_us":169153,"wal-append.run_cpu_time_us":14188,"wal-append.run_wall_time_us":1999691}]]}
I20260709 12:18:05.064733 28285 rpcz_store.cc:276] 0709 12:18:02.915105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.985830 (+ 70725us) service_pool.cc:224] Handling call
0709 12:18:02.986201 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:02.989724 (+  3523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:02.989777 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:02.992150 (+  2373us) log.cc:844] Serialized 26 byte log entry
0709 12:18:04.227691 (+1235541us) spinlock_profiling.cc:243] Waited 8.13 ms on lock 0x7b180023c260. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:18:04.402683 (+174992us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:04.475915 (+ 73232us) log.cc:844] Serialized 14 byte log entry
0709 12:18:04.583085 (+107170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.605433 (+ 22348us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:05.042339 (+436906us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:05.063975 (+ 21636us) raft_consensus.cc:1685] finished
0709 12:18:05.063998 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.064148 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73829,"prepare.queue_time_us":54224,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":60,"raft.queue_time_us":44746,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":27793,"spinlock_wait_cycles":78386688,"thread_start_us":56563,"threads_started":1,"wal-append.queue_time_us":1451118}
I20260709 12:18:05.112154 28190 catalog_manager.cc:5697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:05.117143 31412 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.108s	user 0.000s	sys 0.008s
I20260709 12:18:05.114297 28190 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 6, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 6 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:05.077040 31287 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:18:05.121630 28190 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 5 to 7, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 7 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:05.123811 28190 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 2 to 4, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 4 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:05.071368 31161 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:05.133311 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 174911us
W20260709 12:18:05.142177 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.128723 31322 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.134023 28190 catalog_manager.cc:5697] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 3 to 5, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 5 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:05.155305 28441 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:05.137444 31287 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.145015 31359 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:05.166396 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 62375us
W20260709 12:18:05.195384 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 117916us
I20260709 12:18:05.146253 31151 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:18:05.147202 31391 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.162719 31161 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:05.211234 28190 catalog_manager.cc:5697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 3 to 5, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 5 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
W20260709 12:18:05.237195 28404 leader_election.cc:336] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.875s (SENT)
I20260709 12:18:05.245119 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:05.246982 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 63388.4us
I20260709 12:18:05.187103 31287 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:05.253726 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 77687.4us
I20260709 12:18:05.245950 28190 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:05.206840 31333 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:18:05.189590 31359 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.211547 31366 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:05.269659 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:05.273444 28190 catalog_manager.cc:5697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 6, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:05.271857 31161 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:05.292044 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 79820.6us
W20260709 12:18:05.307713 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.308028 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.264544 31277 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
W20260709 12:18:05.337682 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 118411us
W20260709 12:18:05.338820 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 69228.3us
W20260709 12:18:05.340322 28404 connection.cc:256] RPC call timeout handler was delayed by -1.033781314999942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
I20260709 12:18:05.263018 31196 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:18:05.340827 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.273874 31366 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.348027 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:05.360270 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 92275.7us
W20260709 12:18:05.362951 31428 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 12:18:05.364303 31415 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:18:05.366498 31420 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:18:05.367663 31431 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:18:05.368814 31359 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.369652 31436 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.023s
W20260709 12:18:05.370905 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.369707 31257 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:05.372818 31426 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:18:05.374795 31161 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.375444 31161 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.361449 31304 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:05.378261 31304 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.378685 31304 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.380929 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.382262 28363 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:18:05.383184 31366 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:05.372131 31257 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.384455 31322 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.385314 31257 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.387245 31286 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.388571 31287 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:05.390141 28190 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 3 to 4, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 4 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:05.391232 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.392482 28190 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 5 to 8, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 8 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:05.377616 31161 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:05.394701 31391 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.396291 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:05.397639 31286 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.398867 31380 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.398064 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:05.411015 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:05.366626 31422 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:18:05.419714 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.416594 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.422673 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:05.424453 31380 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.426604 28437 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:05.397727 31419 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:05.431512 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:05.414171 31359 raft_consensus.cc:2764] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:18:05.434408 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 96700us
W20260709 12:18:05.440940 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49760.1us
I20260709 12:18:05.445729 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:05.449381 31441 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.450165 31257 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:05.450368 31441 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:05.451164 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:05.451370 31257 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.451552 31441 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.451838 28437 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.452188 31441 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:05.453236 31441 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.454617 31441 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.449282 31304 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.454707 28190 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 1 to 2, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:05.469553 28437 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:05.451828 31257 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:05.479457 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:05.480259 31443 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:05.479980 31257 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.481030 31443 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:05.488926 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.489868 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 58479.3us
I20260709 12:18:05.491492 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.492563 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:05.493273 31429 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
W20260709 12:18:05.496349 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.472062 31392 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.501124 28190 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 3 to 4, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 4 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:05.502828 28190 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:05.504658 28190 catalog_manager.cc:5697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 1 to 4, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 4 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:05.512015 31307 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.512487 31307 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:05.513435 31304 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.516703 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:05.519533 28433 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:05.518638 31336 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.521140 31336 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.522827 31188 raft_consensus.cc:2764] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:18:05.518081 31257 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:05.523079 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 56093.1us
W20260709 12:18:05.519968 31424 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.337s	user 0.002s	sys 0.000s
I20260709 12:18:05.524011 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:05.524267 28364 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:05.494949 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:05.525521 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.531472 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.532040 28291 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:05.532315 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.533886 28359 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:05.511907 31322 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.535035 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.536635 28365 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:05.536362 31307 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.537431 31307 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.537946 28434 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:05.531281 31188 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:05.540455 31219 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:05.542481 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51066.8us
I20260709 12:18:05.544070 30990 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:05.537654 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.548141 28190 catalog_manager.cc:5697] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 3 to 7, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 7 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:05.551939 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.552935 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.561148 28291 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:05.561308 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:05.546921 31219 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:05.575232 31219 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.562027 28437 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:05.570996 28404 leader_election.cc:400] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:18:05.577001 28404 leader_election.cc:403] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:18:05.544486 31443 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.577934 31188 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:18:05.578544 31380 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.579350 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.582038 31188 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.560299 28190 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:05.573925 28362 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:05.575528 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.585224 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.574153 31336 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.586485 31257 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:05.586728 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65012.5us
I20260709 12:18:05.596107 31257 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:05.597107 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:05.587236 31151 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.597450 31257 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.598085 31257 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:05.599133 31257 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.600677 31257 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:05.601687 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 58344.7us
I20260709 12:18:05.602398 31188 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.602655 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.604280 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.598045 31151 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.604280 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.617456 31151 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.619160 31231 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:05.624182 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.625061 31380 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.631098 31440 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:05.633841 28434 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:05.638116 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:05.646543 28433 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:05.648746 31286 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:05.650095 31440 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.650780 31440 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.634884 30990 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.652558 31277 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:05.653162 31277 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:05.654153 31277 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:05.653215 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49073.1us
W20260709 12:18:05.658726 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 61726us
I20260709 12:18:05.659598 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.660189 31277 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.660949 31443 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.663641 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.657658 31136 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:18:05.671384 31440 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:05.672175 31440 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.673187 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.660698 31395 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:05.678049 31395 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.678490 31395 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:05.678750 28359 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:05.680172 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.682837 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:05.684968 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 80818.5us
I20260709 12:18:05.685959 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.687031 31322 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.674113 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:05.690455 28365 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.691465 28404 leader_election.cc:400] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 4 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 4. Current term is 5.
I20260709 12:18:05.658442 31336 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:05.698931 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.644418 31188 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:05.713574 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.716306 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:05.746217 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 73121.1us
I20260709 12:18:05.747217 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.749243 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:05.754078 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45541.4us
I20260709 12:18:05.705785 31307 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:05.773916 31448 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:18:05.780028 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.762662 31388 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.785151 28365 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:05.744268 31286 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:18:05.759892 31443 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.778090 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:05.789943 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.766180 31397 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:05.768812 31188 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.769593 31366 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 12:18:05.802184 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 53012us
I20260709 12:18:05.817960 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.815448 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.824288 31322 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.795877 31307 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.827636 31397 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:05.833918 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.800153 31391 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.861553 31257 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.845548 31388 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:05.874073 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:05.880030 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:05.890955 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 119772us
I20260709 12:18:05.891294 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.872542 31442 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:05.890681 31336 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.892218 31322 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.903780 28330 leader_election.cc:400] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11.
I20260709 12:18:05.907382 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.899652 31333 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:05.912271 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 59293.5us
I20260709 12:18:05.892155 31441 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:18:05.895501 31346 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.896663 31388 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:05.916539 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.923920 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.925249 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.931620 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:05.931972 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.935720 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.923574 31333 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.937139 31257 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.937538 31333 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.940065 31131 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.940647 31333 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:05.940827 31359 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.921079 31304 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:05.941669 31304 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.949549 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.950484 28437 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:05.902366 31395 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.924103 31336 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:05.963927 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.965101 28360 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:05.947245 31307 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:18:05.966372 31333 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.967077 31333 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:05.968031 31454 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:18:05.969393 31333 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.973764 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.975131 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:05.924377 31346 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:05.977351 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:05.986410 28437 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:05.957631 31161 raft_consensus.cc:2764] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:18:05.924185 31439 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:18:05.978566 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.978749 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:05.978987 31436 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:05.979486 31450 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:18:05.980059 31397 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.979878 31388 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.980432 28984 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:05.981453 28983 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:05.981892 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:05.982259 31395 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:05.982877 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.001289 28436 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:05.983090 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:05.983636 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 57156.8us
I20260709 12:18:06.003213 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:05.987803 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:06.004259 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.004530 31131 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:06.004953 31131 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.005276 31161 raft_consensus.cc:2764] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:18:05.991628 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:05.990536 28367 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:06.005715 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.006389 28365 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:06.005321 31131 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.015331 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:06.017256 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:06.008812 31423 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:06.018131 31322 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:06.018630 31161 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:06.018908 31304 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.020110 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.020733 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.021042 31336 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.021389 31436 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:06.021587 31336 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.022733 28437 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:06.023628 31161 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:06.023975 31336 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.024128 31161 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.026707 31322 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.027056 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.027830 28367 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:06.027936 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.028357 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:06.023442 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.009718 31440 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:18:06.030596 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.031847 31346 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.031853 31322 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:06.032939 31322 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:06.033670 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.033975 31322 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.034592 31440 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.992305 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:05.977427 31333 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:05.999276 28988 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 6 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:05.908005 31442 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:05.994927 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.029071 31277 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:18:06.038295 31397 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:06.057844 31397 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:06.058367 31397 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.058547 31442 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.022993 31436 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:06.059332 31436 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.059702 31436 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.062098 31346 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:06.062584 31346 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.062915 31346 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.063177 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.063391 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.064865 31304 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.045677 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.035638 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.047202 28439 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:06.050101 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.051308 28365 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
W20260709 12:18:06.073386 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45111.7us
I20260709 12:18:06.077880 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.074422 31304 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.079169 31304 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.084218 31439 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:06.091621 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 56087.4us
I20260709 12:18:06.085949 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.097842 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.099135 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.078267 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.069299 31395 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.081596 31304 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:06.109076 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.113088 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.092962 31277 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:06.118919 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.097970 31335 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:06.121626 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:06.125121 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:06.126428 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 48264.1us
I20260709 12:18:06.130334 28365 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:06.122016 31188 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:06.131156 28439 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:06.133520 31335 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.125783 31304 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.137532 31304 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.138844 31335 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.115604 31395 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.139891 31304 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.141351 31395 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:06.133875 31151 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:06.144812 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.119601 31277 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.149197 28361 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:06.148032 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.125934 31440 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.131717 31437 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:06.152318 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:06.156566 31333 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:06.158211 31439 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:06.158731 31439 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.153627 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.159127 31439 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.159446 31333 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.159884 31333 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.162070 31346 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.164498 31346 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.149176 31277 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.167682 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:06.169709 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:06.172606 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 80062.2us
I20260709 12:18:06.173307 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.174180 28439 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:18:06.192909 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.170903 31151 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:06.196056 31151 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.196712 31304 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:06.199574 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:06.197500 31304 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.206519 31423 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:06.209333 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.210664 28330 leader_election.cc:400] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
W20260709 12:18:06.212069 31464 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.135s	user 0.002s	sys 0.000s
W20260709 12:18:06.214814 31463 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:18:06.214923 31391 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:06.215898 31391 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:06.216365 31391 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:06.217283 31391 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.213322 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.228497 31440 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.228902 31397 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.231940 31397 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.233603 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:06.238729 31231 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:06.239161 31231 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:06.239521 31231 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:06.240530 31277 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:06.241813 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.241806 28361 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:06.243757 31333 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.244800 31346 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.245357 31346 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.248219 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:06.249092 28432 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:06.190078 31131 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.251410 31346 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.251986 31131 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.253924 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:06.254587 28438 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.265806 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:06.267972 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.259382 31188 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:06.259897 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 86751.5us
I20260709 12:18:06.271123 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:06.272567 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:06.272509 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:06.260075 31322 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:06.273809 31419 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:06.274142 31322 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:06.274567 31322 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:06.274953 31322 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.260757 31161 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.261394 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:06.262050 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.278775 28362 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:18:06.258127 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.279996 31161 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:06.280256 28440 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:18:06.275038 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:06.278822 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.281992 28257 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:06.262537 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.285359 28438 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
W20260709 12:18:06.286666 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 73454.6us
I20260709 12:18:06.291706 28367 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:06.288380 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:06.296146 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:06.297318 28361 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.258673 31333 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.286645 31161 raft_consensus.cc:2764] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:06.305555 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.308001 31257 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:06.308665 31257 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.308236 31304 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.310042 31391 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.311129 31257 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.313449 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.294422 31336 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.314644 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.299413 31441 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:06.296249 31346 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:06.315892 31441 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:06.316273 31346 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.316685 31346 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:06.317159 31354 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:06.319051 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.319963 31161 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.321029 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:06.321506 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.321918 31188 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:06.322894 31336 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:06.323516 31188 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:06.323941 31188 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:06.324332 31188 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:06.293267 31436 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.309167 31277 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.336668 31392 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:06.337203 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:06.316902 31441 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.338022 28360 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.336738 31366 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:18:06.337405 31392 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.340802 31392 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.337508 31423 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:18:06.344359 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.347509 28360 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:06.349355 31161 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:18:06.355104 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47563us
I20260709 12:18:06.360229 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.363642 28440 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:06.363896 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:06.368680 28330 leader_election.cc:400] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:06.369063 28330 leader_election.cc:403] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:18:06.370673 31354 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:06.377935 31131 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.378609 31131 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.257697 31439 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.387742 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:06.410228 28433 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.389400 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.389673 31436 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:06.389427 31188 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.395689 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.396744 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.398209 31397 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:06.400599 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.401983 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:06.418874 28988 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 6 LEARNER]: Advancing to term 8
I20260709 12:18:06.400401 31131 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.378026 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:06.419862 28440 raft_consensus.cc:2410] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:06.403453 31392 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.429783 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.429847 28361 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:06.435214 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.422448 31136 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:06.475279 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 80378us
I20260709 12:18:06.431852 28432 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:06.478731 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 43621.5us
W20260709 12:18:06.460949 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 83024.4us
I20260709 12:18:06.479257 28361 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:06.426829 31354 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:18:06.426011 31346 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.457405 31322 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.490751 31136 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:06.492054 31322 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:06.491842 31136 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.430806 31231 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.494800 31231 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.496658 28439 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:06.490013 28440 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.495242 31188 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:06.502194 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:06.502584 28988 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 LEARNER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:06.503078 28437 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:06.507369 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 77745us
I20260709 12:18:06.455065 31440 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:06.513693 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:06.514952 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:06.518756 28283 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:06.430261 31161 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:06.528606 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:06.501237 31336 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:06.531677 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.532878 28288 raft_consensus.cc:2368] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:18:06.527096 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:06.535414 28290 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:06.522730 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:06.536443 28283 raft_consensus.cc:2368] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:06.535694 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:06.539413 28292 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:06.539625 28288 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:06.540725 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:06.541355 31439 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.541921 28985 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 8 LEARNER]: Advancing to term 9
I20260709 12:18:06.528822 31436 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.542719 31436 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.543567 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:06.544152 28367 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.545109 31436 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.546417 31336 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:06.547338 31336 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.550056 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:06.551371 28285 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:06.554491 28367 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:06.556051 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:06.556031 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:06.566001 28437 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:18:06.569192 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:06.589474 28289 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:06.527705 31441 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:06.542140 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:06.598006 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 85262.9us
I20260709 12:18:06.594839 28985 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 9 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:06.580507 31257 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:06.602128 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.588681 31322 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:06.530097 31231 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:18:06.605989 28287 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:06.593240 31131 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:06.598893 31440 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:06.630120 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:06.630988 28432 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.623136 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:06.644166 28289 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:06.656023 28991 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:06.644695 31441 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.646806 31188 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:06.656625 31304 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:06.688380 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 58458.8us
I20260709 12:18:06.691552 28440 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:06.631423 31131 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.711439 31346 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.649029 31322 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:18:06.718523 28432 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:06.709522 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:06.661801 31391 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:18:06.734118 31483 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.738075 31366 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:06.753896 31366 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:06.753460 31440 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.754796 31366 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.756026 31366 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.756837 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:06.744921 31392 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.694392 31304 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.759927 31304 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:06.760461 31304 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.760944 31304 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.672551 31161 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 12:18:06.774142 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 55758us
W20260709 12:18:06.752528 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.763610 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:06.778520 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:06.779059 28360 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:06.788232 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.683678 31441 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.801390 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:06.774763 31333 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:06.798170 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:06.815812 28432 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:06.834614 28433 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:06.753504 31483 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:06.791411 31419 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:06.826594 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45290.3us
W20260709 12:18:06.816553 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 107092us
I20260709 12:18:07.031324 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:07.032359 31322 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:07.032960 31322 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:07.033950 31322 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.039920 31443 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.040617 31443 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.841452 28360 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:07.048731 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.050541 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.051362 28366 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:07.052420 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.053833 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.054546 28359 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:06.852198 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.058300 28361 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:18:06.859472 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:06.864171 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:07.060214 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 196149us
I20260709 12:18:07.061269 31484 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:06.828081 31482 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:06.830648 31333 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:07.066567 31333 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:07.005295 31441 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:06.849278 31419 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:07.069970 31419 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.868027 31397 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.892660 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:07.074333 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 184713us
I20260709 12:18:07.075971 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:07.076653 28289 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:06.895742 28432 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:06.875738 31257 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:07.078958 31257 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.917459 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:06.903429 31188 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:06.951849 31336 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:06.961724 31487 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:18:07.095079 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:06.964115 31131 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.098023 31131 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:07.098580 31131 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:06.994918 28330 leader_election.cc:400] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6.
I20260709 12:18:07.099432 31131 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:06.963688 31483 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:06.989478 31304 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:07.011523 31136 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:06.990212 31488 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 12:18:07.014288 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.101267 31136 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:07.104691 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 109929us
I20260709 12:18:07.024803 31392 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:07.074097 31397 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:06.921469 31479 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.200s	user 0.001s	sys 0.001s
I20260709 12:18:06.878664 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:18:07.089656 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 172267us
I20260709 12:18:07.113121 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:07.113950 28984 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:07.114464 31287 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:07.090386 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:07.115685 28988 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:07.092103 28433 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:07.092548 31437 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:07.117444 31437 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:07.118381 31437 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:07.097849 31188 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.094018 31336 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:07.004730 31391 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:18:07.099362 28440 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:07.107353 28432 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:07.110411 28289 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:07.109422 31392 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:07.118788 31136 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.096872 31480 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.136518 31336 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:07.125603 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:07.126142 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:07.137657 28434 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:07.137989 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:07.127100 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 95846.3us
I20260709 12:18:07.139375 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:07.127384 28436 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:07.132575 28438 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:07.135577 31392 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.091102 31366 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:07.141217 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:07.136287 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:07.134270 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:07.143178 28361 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:07.145511 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:07.137140 31391 consensus_queue.cc:1243] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:07.154778 28988 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 9 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:07.136884 31480 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.161542 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:07.167213 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:07.168054 28290 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:07.173300 28330 leader_election.cc:400] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:07.121888 31481 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:18:07.155419 31336 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:07.141336 31490 raft_consensus.cc:2764] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:18:07.182266 28359 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:07.158109 31392 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:07.185900 28362 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:07.186894 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.187367 28366 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:07.156764 31443 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.166199 31439 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:07.190725 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.201542 28363 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:07.188452 28359 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:07.160887 31366 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:07.207456 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 45991.5us
I20260709 12:18:07.208468 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:07.194108 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.178056 30990 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:07.214637 28359 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.182358 31481 consensus_queue.cc:1243] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:07.218945 28359 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:07.186000 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:07.215129 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.183110 31336 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.187171 31161 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:07.222929 28361 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
W20260709 12:18:07.227654 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 54504.1us
I20260709 12:18:07.201320 31439 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.230959 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:07.214386 31366 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.227391 31336 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:07.254019 28440 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:07.231446 31490 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:07.226979 31397 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.236464 31439 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:07.256382 31161 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:07.233150 31380 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:18:07.266973 31277 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.267581 31277 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:07.254952 31504 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:18:07.302598 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 73294.1us
I20260709 12:18:07.307534 28359 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:07.308600 28359 pending_rounds.cc:85] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 4
I20260709 12:18:07.276212 31277 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.233350 31477 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:18:07.256753 31440 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:18:07.261022 31495 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:18:07.333242 28436 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:07.308918 28359 pending_rounds.cc:107] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:07.314308 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:07.316156 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 132973us
I20260709 12:18:07.334422 28438 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:07.335331 28359 raft_consensus.cc:2889] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:07.284626 31419 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.346416 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:07.347309 28984 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:07.318692 31161 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.357028 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:07.352339 31439 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.367311 28956 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:07.359234 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:07.362330 28439 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:07.363122 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:07.370468 28437 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:07.370891 28437 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:07.371811 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 140951us
W20260709 12:18:07.373188 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 69368.2us
I20260709 12:18:07.374547 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:07.375090 28359 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:07.450249 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.451020 28359 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:07.455832 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.456137 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.458441 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.456727 28360 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:07.456637 28366 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:07.466180 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:07.467434 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 100236us
I20260709 12:18:07.469102 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:07.468127 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.457360 28437 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.474031 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:07.476255 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.365069 31161 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:07.474424 31502 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:18:07.464847 31480 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.472678 31335 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:07.483953 30990 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:07.454689 31485 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:18:07.484313 31286 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:18:07.485051 31509 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:18:07.453033 31287 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:18:07.511399 28365 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:18:07.515130 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:07.515450 31161 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:07.516099 31513 raft_consensus.cc:2764] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:07.516230 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:07.517114 31513 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:07.517508 31513 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.517895 31513 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:07.477813 31188 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:18:07.447494 31440 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:07.494959 31505 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:18:07.507143 31286 consensus_queue.cc:1243] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:18:07.500599 31506 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:18:07.522414 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:07.492969 31335 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.525493 28437 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:07.544399 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.545094 28361 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:07.498786 31419 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:07.522886 31439 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:07.538002 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 250508us
I20260709 12:18:07.559844 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:07.539979 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:07.543476 28363 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:18:07.519688 31482 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:07.523231 31188 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:18:07.546804 31335 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:07.562678 28438 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:07.592939 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:07.568838 28360 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:07.580075 28433 pending_rounds.cc:85] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 6
I20260709 12:18:07.580456 28366 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:07.608700 28433 pending_rounds.cc:107] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:18:07.609077 28433 raft_consensus.cc:2889] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:07.609985 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 70110.2us
W20260709 12:18:07.611321 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 165494us
I20260709 12:18:07.612829 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:07.618013 31442 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:07.620316 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1791) took 1095 ms. Trace:
I20260709 12:18:07.622104 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:07.626385 31489 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:07.626881 31489 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.627302 31489 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:07.614951 28437 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:07.630127 28437 pending_rounds.cc:85] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 4
I20260709 12:18:07.630386 28437 pending_rounds.cc:107] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:07.630632 28437 raft_consensus.cc:2889] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:07.625514 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 439570us
I20260709 12:18:07.633291 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:07.635567 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 278636us
I20260709 12:18:07.638366 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:07.638682 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:07.639108 28438 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:07.642215 31515 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:07.642712 31484 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:07.643649 31333 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.607334 31490 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:07.612125 31419 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:07.614522 31423 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:07.647099 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:07.620792 28286 rpcz_store.cc:276] 0709 12:18:06.525293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.525414 (+   121us) service_pool.cc:224] Handling call
0709 12:18:06.568845 (+ 43431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:06.624349 (+ 55504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:06.624390 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:06.624994 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:18:07.110172 (+485178us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:07.163715 (+ 53543us) log.cc:844] Serialized 14 byte log entry
0709 12:18:07.619682 (+455967us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b1800363f40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:18:07.619903 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.619979 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:07.620021 (+    42us) raft_consensus.cc:1685] finished
0709 12:18:07.620112 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:07.620244 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19332,"prepare.queue_time_us":44310,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":3435,"raft.queue_time_us":517277,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":199,"spinlock_wait_cycles":46183168,"thread_start_us":462224,"threads_started":2,"wal-append.queue_time_us":553239}
I20260709 12:18:07.688784 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:07.684993 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 52482.2us
I20260709 12:18:07.687878 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:07.689158 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:07.701661 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:07.703202 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:07.704927 28292 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:07.706267 28292 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:07.707470 28292 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:07.634894 31307 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.690021 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:07.644140 31333 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.709204 31307 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.710428 31333 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.711684 31307 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.712625 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:07.713821 28283 raft_consensus.cc:2368] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:07.715451 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:07.717161 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:07.646843 31490 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:07.721056 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 108318us
I20260709 12:18:07.722357 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:07.723227 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:07.723688 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:07.726917 28290 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:07.719440 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:07.725462 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 92571.6us
I20260709 12:18:07.728612 28285 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:07.726310 28291 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:07.653223 31419 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:07.728528 28284 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:07.685709 31485 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:07.739140 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:07.719512 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:07.746050 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:07.730114 31490 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:07.719926 31151 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:18:07.740970 31346 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.743487 31522 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:07.780094 31524 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:18:07.791538 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.766513 31304 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:07.743142 31333 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:18:07.772859 31483 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:07.743207 31443 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
W20260709 12:18:07.825062 31525 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:18:07.797430 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 150423us
W20260709 12:18:07.847522 31336 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.001s
I20260709 12:18:07.777716 31346 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.859488 31336 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:07.819434 31515 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.860680 31515 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.862015 31336 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:07.810118 31513 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.864768 31513 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:07.784992 31419 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.793227 31392 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:18:07.840217 28437 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:07.863054 31515 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.868233 31346 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.855173 31335 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:07.875846 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 150229us
I20260709 12:18:07.890923 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:07.774878 31322 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:07.916069 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:07.876338 31483 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.935331 28988 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:07.937752 31483 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:07.934777 31346 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:07.945188 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 199210us
I20260709 12:18:07.951560 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:07.952329 28436 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:07.952772 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:07.954177 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:07.954814 28364 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:07.939400 31336 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:07.957461 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:07.957967 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.958621 28359 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:07.958114 28362 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:07.936914 31480 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:18:07.966621 28361 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:07.971205 31335 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:07.944919 31481 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:18:07.948503 31490 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:07.972534 31496 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:07.947176 31346 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.852027 31489 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:07.975231 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:07.983626 28362 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:07.988647 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:07.962221 31336 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.202236 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:07.963994 31322 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:08.203359 28359 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:08.203318 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:08.205017 28434 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.215792 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:08.216544 28365 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:08.221194 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:08.221860 28365 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.224510 28432 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:08.226406 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:08.228281 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:08.229048 28367 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:08.140625 31440 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:08.212924 31504 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 12:18:08.246547 28434 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:08.200343 31346 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.249773 28289 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:08.210493 31336 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.262063 28364 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:08.203933 31131 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:08.275175 31131 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.277480 31131 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:08.252331 28984 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:08.288102 31539 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.288694 31539 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.284600 28365 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:08.290412 31131 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:08.291213 31492 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:18:08.291035 31131 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.294636 28288 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:18:08.294840 31131 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:08.258344 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 296311us
I20260709 12:18:08.297394 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:08.213644 31307 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:08.281713 31526 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.309348 28983 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:08.310158 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:08.313055 31541 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:08.313575 31541 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.313928 31538 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.314576 31538 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.315053 31481 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.315585 31481 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:08.200781 31496 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:08.313962 31541 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.318601 31496 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:08.320230 31542 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:08.320914 31542 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.333195 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:08.323884 31483 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.291049 31539 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:08.384001 28989 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:18:08.353513 31530 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:18:08.359522 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:08.362918 31544 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:08.365629 31489 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:08.366861 31490 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:08.374162 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:08.336231 31322 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:18:08.326407 31526 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.326243 31307 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.390327 31307 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:08.286751 31440 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:08.390913 31542 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:08.392596 31440 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:08.393566 31490 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:08.394012 31490 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:08.394196 31538 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:08.395128 31490 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.377215 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:08.337761 31188 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:18:08.397851 31188 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:08.398238 31188 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:08.399191 31188 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.400980 31188 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.382930 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:08.402067 28984 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.406731 28288 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:08.381834 31550 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:08.407842 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:08.409041 28291 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:08.413507 31550 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:08.425271 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:08.428740 28289 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:08.431032 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:08.440553 28364 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:08.443310 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.446573 31526 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:08.448936 31483 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:08.449986 28291 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:08.450591 31336 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:08.452980 31336 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:08.456492 28289 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:08.488993 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49921.3us
I20260709 12:18:08.495857 31380 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:08.496417 31380 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:08.543751 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:08.544554 28290 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:08.546734 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 208241us
I20260709 12:18:08.526893 31532 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.694468 28364 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
W20260709 12:18:08.695116 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 206280us
I20260709 12:18:08.497296 31380 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:08.708984 28360 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1793) took 1227 ms (client timeout 1602 ms). Trace:
W20260709 12:18:08.709671 28360 rpcz_store.cc:269] 0709 12:18:07.468605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.468859 (+   254us) service_pool.cc:224] Handling call
0709 12:18:08.695866 (+1227007us) spinlock_profiling.cc:243] Waited 76.7 ms on lock 0x7b5400125f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:08.695999 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176302976}
I20260709 12:18:08.541599 31336 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.171s
W20260709 12:18:08.742082 28255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.011s	sys 0.008s
I20260709 12:18:08.696736 31538 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:08.755434 28255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.011s	sys 0.010s
I20260709 12:18:08.832005 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:08.713569 31527 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.833577 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:08.737576 31307 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:08.842634 28404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.012s	sys 0.034s
I20260709 12:18:08.738070 31550 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:08.844481 28404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.013s	sys 0.034s
I20260709 12:18:08.738044 31561 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:08.847474 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 547983us
I20260709 12:18:08.791196 31489 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:08.851006 28404 leader_election.cc:400] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:08.837424 31440 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:08.849934 31346 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 12:18:08.861761 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1772) took 2026 ms. Trace:
I20260709 12:18:08.863063 28439 rpcz_store.cc:276] 0709 12:18:06.835549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.836475 (+   926us) service_pool.cc:224] Handling call
0709 12:18:06.858178 (+ 21703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:07.369255 (+511077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.861430 (+1492175us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b5400086454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:08.861662 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1158893824}
I20260709 12:18:08.869503 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:08.707620 31532 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:08.875909 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:08.865993 31354 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:08.850642 31307 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.846624 31538 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.878757 31513 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:08.857643 31489 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.907812 28438 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:08.988831 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:08.874441 31440 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:08.931522 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:08.866510 31346 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:08.990110 31552 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 12:18:09.226083 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 390594us
I20260709 12:18:09.227260 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:09.228765 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:08.853856 31561 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:09.261416 28256 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.008s	sys 0.041s
W20260709 12:18:09.261804 28256 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.009s	sys 0.041s
W20260709 12:18:08.952405 28984 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.550s	user 0.006s	sys 0.006s
I20260709 12:18:08.944032 31439 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:08.912288 31513 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:08.992048 31563 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:08.887778 31354 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:08.883627 31441 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:08.991166 31560 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:18:08.972919 31555 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:08.991737 31557 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:08.963089 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:08.989121 31554 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:18:09.284086 31576 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:09.284651 31576 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.285030 31576 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.842720 31527 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:09.088397 31541 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.745s	user 0.012s	sys 0.001s
W20260709 12:18:09.106458 31423 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.025s	sys 0.084s
I20260709 12:18:09.203828 28439 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:09.266314 31439 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.234622 28285 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:09.235242 31564 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
W20260709 12:18:09.238412 31559 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:18:09.246637 31538 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.248111 31532 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.248836 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:09.305272 28437 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.225423 31565 consensus_queue.cc:1059] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 12:18:09.249869 31558 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:18:09.250921 31489 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.251310 31346 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.251854 31307 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.256265 31484 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:09.258554 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:09.264473 31440 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.270706 31513 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:09.314251 28437 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:09.315358 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.315992 28437 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:09.316820 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:09.233405 28290 consensus_meta.cc:287] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.688s	user 0.005s	sys 0.004s
I20260709 12:18:09.317925 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.318014 28290 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:09.271549 28984 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
W20260709 12:18:09.301414 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 106004us
I20260709 12:18:09.298945 31555 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:09.300257 31423 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.999s	user 0.025s	sys 0.084s
I20260709 12:18:09.323674 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:09.323870 31527 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.313560 31484 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.325029 31484 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.307720 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:09.298260 31541 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.314249 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.332423 28441 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:18:09.301419 31188 consensus_meta.cc:287] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.859s	user 0.008s	sys 0.004s
I20260709 12:18:09.339788 31532 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:09.340811 31532 consensus_queue.cc:1243] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:09.341341 31532 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.341859 31532 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.343297 31439 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.343909 31532 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.344481 31583 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.345086 31583 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.347332 28366 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:09.347754 31322 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:09.348287 31322 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:09.349094 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 90653us
W20260709 12:18:09.351614 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:09.352136 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.354835 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:09.355962 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.356595 28360 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:09.357882 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.358510 28366 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:09.361791 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:09.365576 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:09.366156 28365 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.366801 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:09.367683 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:09.370158 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1800) took 1143 ms. Trace:
I20260709 12:18:09.344373 31527 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.370867 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.319046 31354 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.362473 31307 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:09.357395 31532 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.347324 31583 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.371955 28363 rpcz_store.cc:276] 0709 12:18:08.220831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.234966 (+ 14135us) service_pool.cc:224] Handling call
0709 12:18:08.235419 (+   453us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:08.309539 (+ 74120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:08.309569 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:08.423070 (+113501us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b2c0000e678. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:18:08.423694 (+   624us) log.cc:844] Serialized 52 byte log entry
0709 12:18:08.534184 (+110490us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:08.888928 (+354744us) log.cc:844] Serialized 14 byte log entry
0709 12:18:08.889195 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.889273 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:09.348876 (+459603us) raft_consensus.cc:1685] finished
0709 12:18:09.363837 (+ 14961us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:09.364000 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84122,"prepare.queue_time_us":567428,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":1681,"raft.queue_time_us":59312,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":13434,"spinlock_wait_cycles":39344000,"thread_start_us":297385,"threads_started":2,"wal-append.queue_time_us":263224}
I20260709 12:18:09.380475 31346 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.392732 31354 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:09.392565 31346 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.383275 31580 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.398214 31580 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.399991 31346 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.400522 31580 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.401113 31580 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.401626 31580 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.411381 31541 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:09.415781 31548 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 1.035s	user 0.002s	sys 0.003s
I20260709 12:18:09.368659 31579 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.416664 31579 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.418701 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1801) took 1458 ms. Trace:
I20260709 12:18:09.419113 28359 rpcz_store.cc:276] 0709 12:18:07.960322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.960896 (+   574us) service_pool.cc:224] Handling call
0709 12:18:08.203160 (+242264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:08.203855 (+   695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.418486 (+1214631us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54002f3ad0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:09.418633 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556451712}
I20260709 12:18:09.420136 31561 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.430546 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.431327 28437 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:09.419152 31579 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.380043 31439 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.384317 31582 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.444137 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:09.453653 28365 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:09.391397 31307 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:09.455684 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.457445 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:09.458317 28437 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:09.348300 31555 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.429526 31561 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.384538 31527 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.391803 31532 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.461871 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.438939 31583 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.478754 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:09.479869 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:09.465631 28437 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:09.485275 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.486056 31532 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.486351 31581 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.486946 31581 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.489108 31581 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.490640 31513 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:09.489624 31581 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.493649 31576 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.500450 28290 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:09.500905 31527 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.501510 31576 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.502565 31580 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.503203 31588 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.467679 31561 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.503744 31581 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.503819 31588 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.504451 31581 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.506247 31557 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:09.507118 31581 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.475760 31583 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.475003 31585 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.457459 31322 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.535676 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1821) took 1110 ms. Trace:
I20260709 12:18:09.536128 28291 rpcz_store.cc:276] 0709 12:18:08.424815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.425001 (+   186us) service_pool.cc:224] Handling call
0709 12:18:09.535453 (+1110452us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54003da4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:09.535605 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54289792}
I20260709 12:18:09.459853 31582 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.538080 31585 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.539165 31582 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.540642 31585 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.541499 31307 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.509630 28437 raft_consensus.cc:2410] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:18:09.543574 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 86270.4us
I20260709 12:18:09.543588 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.544510 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:09.545038 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.545627 28438 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:09.548066 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:09.512882 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:09.461565 31439 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.494578 31346 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:09.483382 31441 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.519913 31583 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.521270 31588 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.493039 31513 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.523798 31419 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.525347 31423 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.525529 31484 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.525882 31538 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:09.571964 31538 raft_consensus.cc:2764] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:09.531167 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.573123 28433 raft_consensus.cc:2410] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:18:09.573561 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2056) took 1115 ms. Trace:
I20260709 12:18:09.558337 31582 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:09.505236 31561 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.545589 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:09.554879 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.561486 31257 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:09.575583 31257 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.576004 31257 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.576506 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.569242 31423 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.578895 31480 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.579437 31480 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.580127 31423 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.561022 31441 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.581990 31346 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:09.554656 31477 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:18:09.558920 31585 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.553332 31576 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:09.590134 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44590.1us
I20260709 12:18:09.590864 31441 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.558128 31581 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.592020 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:09.573892 28433 rpcz_store.cc:276] 0709 12:18:08.458399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.530898 (+1072499us) service_pool.cc:224] Handling call
0709 12:18:09.573497 (+ 42599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:09.596870 28284 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:09.599345 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:09.601231 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 53251.1us
I20260709 12:18:09.601647 28433 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:09.607522 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 94706.9us
I20260709 12:18:09.626814 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:09.627362 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:09.632931 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.635123 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:09.636296 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:09.629031 28255 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.4:39985 timed out after 1.504s (SENT) [suppressed 63 similar messages]
I20260709 12:18:09.639163 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.640365 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.597859 31439 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.627727 31532 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:09.623906 31441 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.623935 31322 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:09.624148 31583 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.591040 31585 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.594834 31582 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:09.645188 31322 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:09.645694 31583 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.594192 31513 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:18:09.634411 31580 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:09.652092 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.634807 31565 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:09.597998 31576 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.655360 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.597803 31581 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.654132 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:09.590183 31346 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.629160 31555 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:18:09.647172 31532 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:09.661676 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.660615 31580 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:09.670969 31580 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.671824 31257 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.672340 31587 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:18:09.673976 31587 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.674474 31587 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.676380 31589 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:09.676847 31589 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.677248 31589 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.681818 31557 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:09.682425 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.682241 31322 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.653434 31188 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:09.685422 31484 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.684939 31188 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.687269 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2054) took 1247 ms. Trace:
I20260709 12:18:09.687680 28434 rpcz_store.cc:276] 0709 12:18:08.439430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.439937 (+   507us) service_pool.cc:224] Handling call
0709 12:18:09.686618 (+1246681us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b58001d4698. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762f8c 00007f85bb762ef6 000000000036eb39 00007f85c37d61e0 00007f85c389f0c4 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:18:09.686759 (+   141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.686941 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:09.686968 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:09.687009 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:09.687042 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.687097 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:09.687207 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11413888}
I20260709 12:18:09.688591 31581 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.690774 31583 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.656100 31565 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:09.694866 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 56445.6us
I20260709 12:18:09.658121 31480 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.696667 31257 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.657822 31582 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:09.701756 31188 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.691653 31532 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.702422 31565 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.653928 31441 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.713354 31594 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.715183 31588 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.716267 31593 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:18:09.718014 31594 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.725299 31441 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.707825 31585 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.735215 31590 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.735831 31590 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.736692 31594 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.736487 31576 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:09.743541 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.744509 28360 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:09.746428 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:09.747161 28360 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:09.747356 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.748844 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.752895 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.754804 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.737924 31593 consensus_queue.cc:1243] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:18:09.767949 31513 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.768566 31513 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.734897 31587 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:09.789728 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 160714us
W20260709 12:18:09.790026 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 154686us
I20260709 12:18:09.728257 31588 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.746802 31585 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:09.857579 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66215.8us
I20260709 12:18:09.863230 31602 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.865209 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:09.756610 31582 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:09.879468 28288 pending_rounds.cc:85] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 4
I20260709 12:18:09.864277 31603 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.931346 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:09.947662 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 293602us
I20260709 12:18:09.951413 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.952327 28433 raft_consensus.cc:2410] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:09.953156 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.954078 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.954797 28434 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:18:09.955828 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.956485 28434 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
I20260709 12:18:09.857724 31587 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.962009 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:09.957432 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:09.963346 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.966127 28433 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:18:09.966648 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.967096 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.967334 28360 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:09.965492 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:09.969864 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:09.970983 28439 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:18:09.971378 28439 pending_rounds.cc:85] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 4
I20260709 12:18:09.971589 28439 pending_rounds.cc:107] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:18:09.971843 28439 raft_consensus.cc:2889] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:09.973031 28358 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:09.972440 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:09.935803 31307 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:18:09.930881 31304 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.974637 28433 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:18:09.983667 28367 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:09.961097 31257 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:09.975070 31583 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.950519 31419 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.951876 28288 pending_rounds.cc:107] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 5.5
I20260709 12:18:09.989790 28441 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:09.992265 28291 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:09.994549 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:09.996706 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:09.999130 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:09.999487 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.000347 31607 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:10.000862 31607 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:10.001255 31607 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:10.002544 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.004674 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.006770 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:10.008824 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:10.006778 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.009604 28289 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:18:10.011868 28287 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:10.015043 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:09.991029 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.017803 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.020555 28288 raft_consensus.cc:2917] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Aborting config change with OpId 5.5: Aborted: Op aborted by new leader
I20260709 12:18:09.936128 31585 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.021049 28288 raft_consensus.cc:2889] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:10.018712 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.026839 31586 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.027613 31590 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:10.027504 31586 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.028430 28290 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:09.985949 31346 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.030102 31346 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.032424 31346 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:10.018651 28358 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
W20260709 12:18:10.032768 31600 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:18:10.034637 31569 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.980386 31484 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.985041 31555 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.041115 28358 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:09.982499 31561 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:09.985275 31539 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.046330 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.047102 28287 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 4.
I20260709 12:18:10.039034 31586 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:09.983381 31513 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:09.987600 31257 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.043062 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.049024 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:10.049926 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.051570 31580 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.051980 31598 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:09.975818 31587 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:09.945050 31602 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.055349 31520 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:10.055783 31596 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:18:10.056424 31589 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.060024 28287 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
W20260709 12:18:10.064932 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.066963 31539 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.046871 31484 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.070145 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.151091 28290 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
W20260709 12:18:10.052649 31605 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:10.073364 31257 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.987728 31304 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.021586 31419 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.042224 31590 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.021066 31583 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.108774 31581 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.153774 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.134528 31527 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:10.164337 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:09.969182 31603 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.058121 31555 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.109428 31188 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:10.106635 31513 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.056795 31607 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.180689 31617 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:10.163815 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:10.154305 31520 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.108831 31580 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.073007 31561 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.182216 31587 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.202914 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:10.190737 28361 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:10.187263 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.362622 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.363484 28362 raft_consensus.cc:2410] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:10.364778 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.366988 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.367468 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:10.368690 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.368907 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:10.369664 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.369973 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:10.370445 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.359701 28290 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:18:10.371081 28987 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:10.371594 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.371982 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.372828 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.373224 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.374197 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.374786 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.370394 28989 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:10.375996 28987 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:18:10.376748 31521 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.376653 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:10.377960 28986 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:10.372022 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.381007 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.382812 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:10.382836 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:10.386037 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:10.391815 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:10.180480 28991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 498) took 2218 ms. Trace:
I20260709 12:18:10.160956 31593 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:10.150178 31598 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.399804 31598 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.377329 31521 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.180583 28983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 501) took 1863 ms. Trace:
W20260709 12:18:10.222121 31564 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:18:10.416960 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:10.396541 31565 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.001s
I20260709 12:18:10.180436 31616 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:10.357582 31602 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.282119 31582 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.184983 31419 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.302913 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.418197 28287 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:18:10.358335 31188 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.395406 28991 rpcz_store.cc:276] 0709 12:18:07.950488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.962313 (+ 11825us) service_pool.cc:224] Handling call
0709 12:18:07.965644 (+  3331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:07.972897 (+  7253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:07.972928 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:07.973593 (+   665us) log.cc:844] Serialized 52 byte log entry
0709 12:18:08.214550 (+240957us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:08.779770 (+565220us) log.cc:844] Serialized 14 byte log entry
0709 12:18:08.837615 (+ 57845us) log.cc:844] Serialized 14 byte log entry
0709 12:18:09.415432 (+577817us) spinlock_profiling.cc:243] Waited 6.67 ms on lock 0x7b240034eb48. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6209c4 00007f85bb61eacb 00007f85bb7d9dc9 00007f85bb7d9a55 00007f85bb7d909d 00007f85be40b10e 00007f85be42288c 00007f85be421bb6 00007f85be4217d8 00007f85be42b221
0709 12:18:09.936146 (+520714us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b18002f6fe0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:18:10.160550 (+224404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.168755 (+  8205us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:10.168933 (+   178us) raft_consensus.cc:1685] finished
0709 12:18:10.168947 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.169082 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75858,"prepare.queue_time_us":180033,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":2342753,"raft.run_cpu_time_us":492,"raft.run_wall_time_us":11119,"spinlock_wait_cycles":19082880,"thread_start_us":934291,"threads_started":3,"wal-append.queue_time_us":406669}
I20260709 12:18:10.432086 31602 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:10.432677 31626 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:10.433130 31582 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.421628 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.427114 31629 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:10.434348 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:10.435011 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.436061 28362 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:10.442066 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.395128 31617 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.187461 31590 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.395467 31513 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.444620 28290 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:10.444532 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:10.452941 28991 raft_consensus.cc:2376] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9. Re-sending same reply.
I20260709 12:18:10.446545 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.428190 31557 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:10.447469 31613 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:10.448589 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.438246 31419 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.434100 31188 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.450076 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:10.442881 28362 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:10.447132 31629 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.447003 31617 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.454110 28287 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:10.433972 31322 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:10.414661 31581 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.419198 31527 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.469182 28434 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:10.448518 31618 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:10.471825 28362 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:10.457932 31557 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:10.480832 31607 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:10.483659 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.425156 31565 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.487452 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.417944 31593 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.489879 28287 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:10.470844 31629 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.427875 31616 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.478511 31617 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:10.501763 31617 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:10.502277 31617 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:10.433358 31626 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.475059 31322 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.468528 31419 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.482206 31557 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:10.456394 31613 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:10.483098 31618 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.484186 31257 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.514279 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.502902 28283 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:10.540431 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 154491us
I20260709 12:18:10.532064 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1803) took 2045 ms. Trace:
I20260709 12:18:10.541728 28285 rpcz_store.cc:276] 0709 12:18:08.486442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.486586 (+   144us) service_pool.cc:224] Handling call
0709 12:18:08.488618 (+  2032us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.301573 (+812955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.531767 (+1230194us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54003dc2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:10.531987 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715156864}
I20260709 12:18:10.492586 31539 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:10.529974 31628 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:10.513449 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.536080 31603 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:10.592391 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:10.544613 31322 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.564642 28361 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:10.565074 28291 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:10.571980 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.538345 31634 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:10.583809 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.552120 31580 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:10.632905 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.414512 28983 rpcz_store.cc:276] 0709 12:18:08.317462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.339756 (+ 22294us) service_pool.cc:224] Handling call
0709 12:18:08.357004 (+ 17248us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:08.357463 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:08.357502 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:08.358243 (+   741us) log.cc:844] Serialized 52 byte log entry
0709 12:18:08.781213 (+422970us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:08.847521 (+ 66308us) log.cc:844] Serialized 14 byte log entry
0709 12:18:09.997455 (+1149934us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.109089 (+111634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.153442 (+ 44353us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:10.160181 (+  6739us) raft_consensus.cc:1685] finished
0709 12:18:10.162231 (+  2050us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.178655 (+ 16424us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1177533,"prepare.queue_time_us":42012,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":1754,"raft.queue_time_us":1289922,"raft.run_cpu_time_us":459,"raft.run_wall_time_us":40723,"thread_start_us":225569,"threads_started":3,"wal-append.queue_time_us":472708}
I20260709 12:18:10.593094 31555 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:10.557559 31527 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.502967 31583 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.634285 28287 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
W20260709 12:18:10.680080 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 318821us
I20260709 12:18:10.797833 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:10.680100 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.798970 28362 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 4.
W20260709 12:18:10.695639 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 155828us
I20260709 12:18:10.691105 28435 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:18:10.706923 31620 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:18:10.647919 31419 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.651144 31513 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:10.632234 31628 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:10.759778 31257 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:10.584190 31561 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:10.813508 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:10.803741 31565 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:10.816556 28365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.004s	sys 0.007s
I20260709 12:18:10.626632 31520 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:10.664160 31532 consensus_meta.cc:287] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.947s	user 0.014s	sys 0.000s
W20260709 12:18:10.693281 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 301564us
I20260709 12:18:10.752280 31626 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:10.683845 31616 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.804034 31613 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.808081 31607 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.813323 31604 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:18:10.820320 28365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.004s	sys 0.007s
I20260709 12:18:10.820425 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.822484 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.818687 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:10.827832 28432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2032) took 3229 ms (client timeout 1668 ms). Trace:
I20260709 12:18:10.814725 31628 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:10.721907 31593 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:10.849943 28432 rpcz_store.cc:269] 0709 12:18:07.591647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.592610 (+   963us) service_pool.cc:224] Handling call
0709 12:18:10.821093 (+3228483us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54003cad54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:10.821262 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427484416}
I20260709 12:18:10.817198 31634 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:10.823383 31513 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:10.854933 31481 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.856199 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:10.856887 31637 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.856312 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.868217 28289 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:10.869396 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.872697 28289 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:18:10.834296 31607 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.939080 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.939844 28289 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:10.852828 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.943500 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:10.942495 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.832057 31532 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.953251 28362 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:10.955180 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.956359 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.956737 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.957435 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:10.940819 31607 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.957409 28291 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:10.958709 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.959106 28291 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 4.
I20260709 12:18:10.866575 31634 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:10.870352 31587 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:10.867978 31637 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:10.963306 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:10.952876 31633 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:18:10.970381 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.972862 31642 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:10.862069 31618 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:10.975920 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.979297 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:10.973632 31642 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.979494 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.964856 31587 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:10.967064 31637 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.967383 31419 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:10.968290 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.983062 28291 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:10.846554 31604 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:10.993103 31642 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:10.996151 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:10.996933 28289 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:10.985044 31587 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:11.002940 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:11.003770 28362 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:10.989044 31489 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:11.013895 31618 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.014528 31618 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:11.024730 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1814) took 1073 ms. Trace:
I20260709 12:18:11.025174 28365 rpcz_store.cc:276] 0709 12:18:09.951525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.951667 (+   142us) service_pool.cc:224] Handling call
0709 12:18:09.976055 (+ 24388us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:10.166406 (+190351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.166459 (+    53us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:10.167284 (+   825us) log.cc:844] Serialized 78 byte log entry
0709 12:18:10.831746 (+664462us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:10.831792 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.831865 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:11.024473 (+192608us) raft_consensus.cc:1685] finished
0709 12:18:11.024495 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.024664 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27618,"raft.queue_time_us":71766,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":52552,"thread_start_us":782328,"threads_started":2,"wal-append.queue_time_us":768356}
I20260709 12:18:11.027407 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:11.028054 28365 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:11.029914 31618 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:10.995592 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:11.024587 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:10.991940 31419 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:11.011539 31489 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:11.052822 28365 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:11.054785 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:11.061094 31621 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.417s	user 0.002s	sys 0.001s
I20260709 12:18:11.060537 31489 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:11.062436 31489 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:11.066164 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:11.070907 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:11.075678 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 90032.5us
I20260709 12:18:11.075955 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:11.076747 28365 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
W20260709 12:18:11.056600 31645 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:18:11.088268 28432 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:11.086663 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:11.092399 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:11.098336 31188 consensus_meta.cc:287] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.625s	user 0.007s	sys 0.007s
I20260709 12:18:11.099009 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:11.100312 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:11.100975 28440 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:11.101995 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:11.098852 31188 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:11.103101 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:11.103770 28432 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:11.104629 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:11.104794 31188 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:11.105189 28432 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:11.089912 31532 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:11.109261 31322 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:11.110582 31557 consensus_meta.cc:287] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.559s	user 0.013s	sys 0.000s
I20260709 12:18:11.103152 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:11.113583 28440 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:18:11.112542 28435 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:11.114919 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:11.116710 28440 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:11.119699 28432 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:11.121099 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:11.121850 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1817) took 1149 ms. Trace:
I20260709 12:18:11.122278 28360 rpcz_store.cc:276] 0709 12:18:09.972705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.973804 (+  1099us) service_pool.cc:224] Handling call
0709 12:18:09.974138 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.168516 (+194378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.168549 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:10.345047 (+176498us) log.cc:844] Serialized 26 byte log entry
0709 12:18:10.811628 (+466581us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:10.811667 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.811770 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:11.121546 (+309776us) raft_consensus.cc:1685] finished
0709 12:18:11.121575 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.121782 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10043,"raft.queue_time_us":69206,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":63638,"thread_start_us":584642,"threads_started":2,"wal-append.queue_time_us":452036}
I20260709 12:18:11.124166 28432 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:11.124101 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:11.127646 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:11.128317 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:11.128384 28362 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:11.129034 28435 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:11.132572 28432 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:11.111097 31557 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:11.137158 28362 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:11.139576 28362 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:11.140610 28440 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
W20260709 12:18:11.141084 28437 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1811) took 1665 ms (client timeout 1503 ms). Trace:
W20260709 12:18:11.141530 28437 rpcz_store.cc:269] 0709 12:18:09.475157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.484991 (+  9834us) service_pool.cc:224] Handling call
0709 12:18:11.140823 (+1655832us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b54000d8754. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:11.140987 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53915904}
I20260709 12:18:11.152190 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:11.100884 31607 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:11.153187 28367 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:11.155212 31617 consensus_meta.cc:287] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.640s	user 0.013s	sys 0.000s
W20260709 12:18:11.742780 28367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1816) took 1555 ms (client timeout 1717 ms). Trace:
I20260709 12:18:11.742750 31617 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:11.158207 31629 consensus_meta.cc:287] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.644s	user 0.007s	sys 0.007s
I20260709 12:18:11.133090 31419 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:12.301918 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 372) took 1917 ms. Trace:
I20260709 12:18:12.302429 28990 rpcz_store.cc:276] 0709 12:18:10.384110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.384261 (+   151us) service_pool.cc:224] Handling call
0709 12:18:10.384624 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.406842 (+ 22218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.406881 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:10.406928 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:10.406961 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.301588 (+1894627us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b54002ab554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:12.301641 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.301842 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50528000}
I20260709 12:18:12.304169 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:12.305176 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:12.306130 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.308161 28364 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:12.309478 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.312266 28991 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:12.313519 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:12.314877 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:11.131791 31322 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:11.206437 31648 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:18:11.259639 31637 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.319341 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1806) took 2966 ms. Trace:
I20260709 12:18:12.319722 28284 rpcz_store.cc:276] 0709 12:18:09.353368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.353508 (+   140us) service_pool.cc:224] Handling call
0709 12:18:09.378121 (+ 24613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.597329 (+219208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.319076 (+2721747us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54003ab454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:12.319271 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502597376}
I20260709 12:18:12.321106 31637 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:12.323590 31557 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:11.273707 31607 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:11.348568 28332 connection.cc:256] RPC call timeout handler was delayed by -2.2853904109999803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
W20260709 12:18:12.325203 28332 connection.cc:256] RPC call timeout handler was delayed by -1.3808809310000925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 12:18:12.326535 31354 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:12.327106 31354 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.328531 31476 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.329348 31354 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:11.355607 31634 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.331379 31634 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:11.345657 31628 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:11.355611 31587 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.334201 31628 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:12.334602 31587 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:12.335340 28291 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:12.336853 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:11.560453 31597 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.338091 31597 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.340457 31597 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:12.341099 28285 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:11.564996 31604 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:11.568681 31588 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:11.621953 31581 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:11.627038 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1828) took 1631 ms. Trace:
I20260709 12:18:12.343058 28286 rpcz_store.cc:276] 0709 12:18:09.996053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.996177 (+   124us) service_pool.cc:224] Handling call
0709 12:18:09.996544 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.029399 (+ 32855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.029438 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:10.029480 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:10.029510 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.626719 (+1597209us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b54003ab954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:11.626769 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.626961 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75064064}
I20260709 12:18:12.344907 28286 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:12.345420 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1825) took 1495 ms. Trace:
I20260709 12:18:11.655819 31593 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:12.346035 31593 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.348130 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 400) took 2024 ms. Trace:
I20260709 12:18:12.348393 31593 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:11.661775 31582 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:11.743337 28367 rpcz_store.cc:269] 0709 12:18:10.187400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.406578 (+219178us) service_pool.cc:224] Handling call
0709 12:18:11.151977 (+745399us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b5800113798. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:18:11.742705 (+590728us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6837632}
I20260709 12:18:11.745935 31629 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:11.745232 31617 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:11.800649 31644 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.831s	user 0.003s	sys 0.015s
I20260709 12:18:12.350142 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:11.805866 31590 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:11.799815 31636 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:11.851436 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1841) took 1026 ms. Trace:
I20260709 12:18:11.832516 31521 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:11.906890 31481 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.972126 31585 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:11.972851 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:12.126178 31513 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.133427 31392 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:12.136068 28987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 374) took 1751 ms (client timeout 1525 ms). Trace:
I20260709 12:18:12.140961 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1831) took 2139 ms. Trace:
I20260709 12:18:11.971580 31589 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:12.141808 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:12.353586 31513 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.351639 31629 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:12.354868 28982 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 375) took 1969 ms (client timeout 2221 ms). Trace:
I20260709 12:18:12.352182 31636 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.144754 31304 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:12.153329 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 448) took 2116 ms. Trace:
I20260709 12:18:12.356209 28441 rpcz_store.cc:276] 0709 12:18:10.036905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.038639 (+  1734us) service_pool.cc:224] Handling call
0709 12:18:10.039553 (+   914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.058732 (+ 19179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.058766 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:10.058814 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:10.058847 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.153009 (+2094162us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b5400184554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:12.153058 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.153215 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43523328}
I20260709 12:18:12.357245 31513 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:12.179955 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.156147 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1814) took 3747 ms. Trace:
I20260709 12:18:12.180586 31527 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:12.357836 31636 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:12.159292 28985 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.003s	user 0.063s	sys 0.075s
I20260709 12:18:12.358057 28435 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
W20260709 12:18:12.358754 28985 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.203s	user 0.063s	sys 0.076s
I20260709 12:18:12.359397 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.201712 31652 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.210615 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:12.221472 31484 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:12.211319 31653 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.003s
W20260709 12:18:12.353657 28988 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 373) took 1969 ms (client timeout 2242 ms). Trace:
I20260709 12:18:12.360109 31652 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.231834 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:12.232470 31477 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:18:12.360878 28988 rpcz_store.cc:269] 0709 12:18:10.384621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.972546 (+1587925us) service_pool.cc:224] Handling call
0709 12:18:12.353593 (+381047us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:12.232185 31489 consensus_meta.cc:287] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.169s	user 0.008s	sys 0.015s
I20260709 12:18:12.361599 31527 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.245604 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:12.363252 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 117745us
I20260709 12:18:12.363824 31527 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:12.247145 31423 raft_consensus.cc:1064] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:18:12.365082 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.253084 31654 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.271623 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1808) took 2804 ms. Trace:
W20260709 12:18:12.273972 28434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1826) took 1907 ms (client timeout 2429 ms). Trace:
I20260709 12:18:12.276388 31440 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.276808 31598 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:12.276706 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.366027 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:12.286657 28221 debug-util.cc:398] Leaking SignalData structure 0x7b08000361e0 after lost signal to thread 27119
I20260709 12:18:12.366922 31440 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.366779 31423 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:12.368054 28221 debug-util.cc:398] Leaking SignalData structure 0x7b08000364e0 after lost signal to thread 28224
I20260709 12:18:12.369098 31440 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:12.288753 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1842) took 1463 ms. Trace:
W20260709 12:18:12.259672 28361 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1817) took 2071 ms (client timeout 2238 ms). Trace:
I20260709 12:18:12.295284 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 397) took 1972 ms. Trace:
I20260709 12:18:12.352103 28292 rpcz_store.cc:276] 0709 12:18:10.825387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.850534 (+1025147us) service_pool.cc:224] Handling call
0709 12:18:11.850907 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:11.851099 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.851123 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:11.851166 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:11.851199 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.851251 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.851365 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.366029 28439 rpcz_store.cc:276] 0709 12:18:09.467326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.482760 (+ 15434us) service_pool.cc:224] Handling call
0709 12:18:09.483119 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.972262 (+489143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.271197 (+2298935us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b54002e2cd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:12.271528 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102386944}
I20260709 12:18:12.299063 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:12.369884 28287 rpcz_store.cc:276] 0709 12:18:10.825711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.287851 (+1462140us) service_pool.cc:224] Handling call
0709 12:18:12.288209 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.288410 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:12.288439 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:12.288483 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:12.288511 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.288568 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.288688 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.366469 31598 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.372167 28292 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:12.370697 28434 rpcz_store.cc:269] 0709 12:18:10.366356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.441476 (+ 75120us) service_pool.cc:224] Handling call
0709 12:18:12.273685 (+1832209us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b540026bbd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:12.273882 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4010368}
I20260709 12:18:12.329134 31476 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.351742 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1823) took 1991 ms. Trace:
I20260709 12:18:12.353895 31392 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:12.374925 31392 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.353034 31481 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.373812 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:12.371292 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.366048 31654 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.378240 31481 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:12.379056 28358 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:12.366127 31568 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:12.376359 31598 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:12.377408 28434 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:12.371193 28289 rpcz_store.cc:276] 0709 12:18:10.323042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.998114 (+675072us) service_pool.cc:224] Handling call
0709 12:18:12.246832 (+1248718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.247052 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:12.247082 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:12.247131 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:12.247163 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.247223 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.295082 (+ 47859us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54003322d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:12.295212 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2824502016}
I20260709 12:18:12.352061 31307 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:12.380930 31307 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:12.366605 28989 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 370) took 1983 ms (client timeout 2040 ms). Trace:
I20260709 12:18:12.381913 31484 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:12.382071 28989 rpcz_store.cc:269] 0709 12:18:10.383067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.231502 (+1848435us) service_pool.cc:224] Handling call
0709 12:18:12.366538 (+135036us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.382859 31568 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.384433 28287 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:12.358003 28288 rpcz_store.cc:276] 0709 12:18:08.408648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.457475 (+ 48827us) service_pool.cc:224] Handling call
0709 12:18:08.457891 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:10.034821 (+1576930us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:10.034856 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:10.035376 (+   520us) log.cc:844] Serialized 52 byte log entry
0709 12:18:10.168266 (+132890us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:10.233252 (+ 64986us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.233710 (+   458us) log.cc:844] Serialized 14 byte log entry
0709 12:18:10.233891 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.007486 (+773595us) spinlock_profiling.cc:243] Waited 8.08 ms on lock 0x7b240016d8e8. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6209c4 00007f85bb61eacb 00007f85bb7d9dc9 00007f85bb7d9a55 00007f85bb7d909d 00007f85be40b10e 00007f85be42288c 00007f85be421bb6 00007f85be4217d8 00007f85be42b221
0709 12:18:12.155758 (+1148272us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540009e7d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:12.155818 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:12.155877 (+    59us) raft_consensus.cc:1685] finished
0709 12:18:12.155892 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.156068 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":486718,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":102,"raft.queue_time_us":13269,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":7617,"spinlock_wait_cycles":3287304832,"thread_start_us":197007,"threads_started":2,"wal-append.queue_time_us":494911,"wal-append.run_cpu_time_us":8105,"wal-append.run_wall_time_us":1673091}
W20260709 12:18:12.354046 28987 rpcz_store.cc:269] 0709 12:18:10.385108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.135818 (+1750710us) service_pool.cc:211] Skipping call since client already timed out
0709 12:18:12.135969 (+   151us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:18:12.181308 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:12.390236 28440 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:12.358258 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 404) took 1870 ms. Trace:
I20260709 12:18:12.391176 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.354274 28283 rpcz_store.cc:276] 0709 12:18:10.001057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.001177 (+   120us) service_pool.cc:224] Handling call
0709 12:18:10.001524 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.541298 (+539774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.140482 (+1599184us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54003f9654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:12.140707 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152622464}
I20260709 12:18:12.382575 31484 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:12.355720 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:12.393723 28986 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 371) took 2010 ms (client timeout 2238 ms). Trace:
I20260709 12:18:12.376262 31392 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.394376 31484 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:12.394850 31652 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:12.168129 31490 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.396155 31476 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:12.396090 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 185560us
I20260709 12:18:12.373934 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.397305 28439 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:12.171716 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.397949 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:12.181089 31545 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.375574 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.398726 31545 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.399199 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:12.399777 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.400584 28433 raft_consensus.cc:2393] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:12.400971 31545 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:12.194483 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:12.246076 28290 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1819) took 2072 ms (client timeout 1697 ms). Trace:
W20260709 12:18:12.401943 28290 rpcz_store.cc:269] 0709 12:18:10.173265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.380613 (+207348us) service_pool.cc:224] Handling call
0709 12:18:12.245793 (+1865180us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b54003322d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:12.245962 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144168192}
I20260709 12:18:12.285669 31655 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:12.403268 31655 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.404086 31522 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:12.381151 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.401656 28432 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.398267 28437 raft_consensus.cc:2410] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:12.374321 28436 rpcz_store.cc:276] 0709 12:18:10.360524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.454050 (+ 93526us) service_pool.cc:224] Handling call
0709 12:18:10.454455 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:10.454891 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:10.454922 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:10.455472 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 12:18:10.846474 (+391002us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:10.846527 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.846625 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:11.800054 (+953429us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:12.351441 (+551387us) raft_consensus.cc:1685] finished
0709 12:18:12.351462 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.351669 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9416,"raft.queue_time_us":886,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":4224,"spinlock_wait_cycles":5792000,"thread_start_us":375591,"threads_started":1,"wal-append.queue_time_us":479989}
I20260709 12:18:12.406738 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:12.407435 28436 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:12.408074 31629 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:12.408668 31629 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:12.409004 28437 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:12.409350 28436 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:12.353000 31617 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.005s
W20260709 12:18:12.369519 28221 debug-util.cc:398] Leaking SignalData structure 0x7b0800339300 after lost signal to thread 28316
I20260709 12:18:12.409965 31581 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.410419 31581 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:12.391496 28285 rpcz_store.cc:276] 0709 12:18:10.487865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.118252 (+630387us) service_pool.cc:224] Handling call
0709 12:18:11.125686 (+  7434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.341519 (+1215833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.358032 (+ 16513us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003fa054. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:12.358199 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2794733440}
I20260709 12:18:12.387607 28367 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:12.396425 31653 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:12.369927 28361 rpcz_store.cc:269] 0709 12:18:10.187738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.420610 (+232872us) service_pool.cc:224] Handling call
0709 12:18:12.259390 (+1838780us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54001d8654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:12.259579 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284898560}
I20260709 12:18:12.405725 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.359508 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 377) took 1972 ms. Trace:
I20260709 12:18:12.345705 28286 rpcz_store.cc:276] 0709 12:18:10.850106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.344208 (+1494102us) service_pool.cc:224] Handling call
0709 12:18:12.344560 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.345204 (+   644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.345365 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.413254 31653 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:12.414587 28434 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
W20260709 12:18:12.355370 28982 rpcz_store.cc:269] 0709 12:18:10.385578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.138813 (+1753235us) service_pool.cc:224] Handling call
0709 12:18:12.354805 (+215992us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.393042 31307 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.321621 28284 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:12.410173 31617 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:12.418220 31617 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:12.414880 31653 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:12.394127 28986 rpcz_store.cc:269] 0709 12:18:10.383652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.355440 (+1971788us) service_pool.cc:224] Handling call
0709 12:18:12.393656 (+ 38216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.419108 31617 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.398339 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.420544 31617 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.420814 28439 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:12.410848 31581 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.421618 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:12.331399 31366 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:18:12.417793 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1824) took 1568 ms. Trace:
I20260709 12:18:12.425464 28364 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:12.357852 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.427199 28441 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:18:12.424840 28284 rpcz_store.cc:276] 0709 12:18:10.849801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.320892 (+1471091us) service_pool.cc:224] Handling call
0709 12:18:12.321249 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.417501 (+ 96252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.417695 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.428396 28441 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:12.429833 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:12.386904 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:12.413878 28985 rpcz_store.cc:276] 0709 12:18:10.386719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.083043 (+696324us) service_pool.cc:224] Handling call
0709 12:18:11.083956 (+   913us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:11.087229 (+  3273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.087260 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:11.127100 (+ 39840us) log.cc:844] Serialized 582 byte log entry
0709 12:18:12.359159 (+1232059us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:12.359195 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.359274 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:12.359311 (+    37us) raft_consensus.cc:1685] finished
0709 12:18:12.359333 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.359452 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3157,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":4060,"thread_start_us":1203485,"threads_started":1,"wal-append.queue_time_us":202542,"wal-append.run_cpu_time_us":2033,"wal-append.run_wall_time_us":1002062}
I20260709 12:18:12.412451 28291 rpcz_store.cc:276] 0709 12:18:10.324061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.000299 (+676238us) service_pool.cc:224] Handling call
0709 12:18:11.000582 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.335747 (+1335165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.347884 (+ 12137us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001aed54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:12.348064 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3069255552}
I20260709 12:18:12.391981 28440 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:12.435649 31568 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:12.438083 28221 debug-util.cc:398] Leaking SignalData structure 0x7b0800325540 after lost signal to thread 28391
W20260709 12:18:12.439465 28221 debug-util.cc:398] Leaking SignalData structure 0x7b080033f3c0 after lost signal to thread 28465
W20260709 12:18:12.440964 28221 debug-util.cc:398] Leaking SignalData structure 0x7b0800042a20 after lost signal to thread 29015
W20260709 12:18:12.441280 28221 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9d80 after lost signal to thread 29016
I20260709 12:18:12.443065 31655 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:12.376500 31380 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:12.446619 31612 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.447250 31612 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.449529 31612 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:12.379505 31654 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:12.450848 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 85848.2us
I20260709 12:18:12.450671 31440 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.451506 31536 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.401412 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.452355 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.452745 28433 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:12.453143 31604 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.452368 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.454347 28330 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:12.455166 31604 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.455583 31604 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.456099 31515 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:12.455539 31628 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.456878 31604 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.457197 31628 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.459434 31628 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:12.459955 28255 connection.cc:256] RPC call timeout handler was delayed by -1.5294919020000179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 12:18:12.460311 28255 connection.cc:256] RPC call timeout handler was delayed by -1.488091901999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 12:18:12.460657 28255 connection.cc:256] RPC call timeout handler was delayed by -1.4241919020000751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
W20260709 12:18:12.460964 28255 connection.cc:256] RPC call timeout handler was delayed by -1.4007919020000372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 12:18:12.461236 28255 connection.cc:256] RPC call timeout handler was delayed by -1.394491902000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 12:18:12.461402 31515 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:12.461865 31515 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:12.422979 28439 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:12.462951 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:12.464318 31496 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:12.464958 31521 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:12.465370 31496 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:12.465678 31520 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:12.466094 31520 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:12.466576 31569 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:12.466967 31569 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:12.467074 31520 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.467896 31569 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.468578 31520 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:12.462738 31515 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.472344 31586 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.472815 31586 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.473192 31586 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:12.474143 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 52651.3us
I20260709 12:18:12.409595 31629 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.475108 31654 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:12.475512 31654 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:12.476322 28255 connection.cc:256] RPC call timeout handler was delayed by -1.3881919020000169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 12:18:12.476761 28255 connection.cc:256] RPC call timeout handler was delayed by -1.3881919020000169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 12:18:12.477085 28255 connection.cc:256] RPC call timeout handler was delayed by -1.3318620700001702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 12:18:12.477406 28255 connection.cc:256] RPC call timeout handler was delayed by -1.3120620700001382s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
I20260709 12:18:12.477831 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.412249 28437 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:12.477712 28255 connection.cc:256] RPC call timeout handler was delayed by -1.1838919019999139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
I20260709 12:18:12.480901 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:12.481312 28255 connection.cc:256] RPC call timeout handler was delayed by -1.1037919020000118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
W20260709 12:18:12.480813 28221 debug-util.cc:398] Leaking SignalData structure 0x7b08002baba0 after lost signal to thread 31559
I20260709 12:18:12.481594 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.482683 28433 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
W20260709 12:18:12.481792 28255 connection.cc:256] RPC call timeout handler was delayed by -1.0825620699999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
W20260709 12:18:12.488726 28255 connection.cc:256] RPC call timeout handler was delayed by -1.0744620699999814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260709 12:18:12.489169 28255 connection.cc:256] RPC call timeout handler was delayed by -1.0704919019999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
I20260709 12:18:12.481638 28366 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.484135 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:12.489523 28255 connection.cc:256] RPC call timeout handler was delayed by -1.0704919019999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 12:18:12.491379 28255 connection.cc:256] RPC call timeout handler was delayed by -1.0589646320001975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 12:18:12.491737 28255 connection.cc:256] RPC call timeout handler was delayed by -1.0560919019999346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
W20260709 12:18:12.491961 28255 connection.cc:256] RPC call timeout handler was delayed by -1.0474339230001988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 12:18:12.492316 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.493129 28284 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:12.493929 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.494673 28440 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:12.496441 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:12.497061 28359 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.483338 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:12.497555 28436 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.499059 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.499982 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.501753 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:12.478083 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:12.475853 31654 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.503227 28439 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.507774 28432 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:12.508535 28434 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:12.455842 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:12.509546 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 181422us
I20260709 12:18:12.513867 28286 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.514657 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.516911 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:12.517544 28985 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.518960 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.520195 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:12.521346 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.522385 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:12.522593 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 66821.2us
I20260709 12:18:12.523977 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:12.525334 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.515353 28364 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:12.527024 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:12.504798 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:12.529686 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.532685 28292 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:12.533546 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.535959 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:12.527184 31366 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.560896 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:12.500891 31652 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:18:12.532760 31484 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:12.541018 31655 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.541985 28985 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:12.522265 31536 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:12.574506 28288 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.546661 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:12.547183 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:12.548143 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.575625 28986 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.577078 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.577879 28436 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:12.552994 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:12.554303 28439 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:12.555395 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:12.580080 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:12.556035 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:12.556303 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1826) took 1705 ms. Trace:
I20260709 12:18:12.581135 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:12.581179 28292 rpcz_store.cc:276] 0709 12:18:10.850410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.371692 (+1521282us) service_pool.cc:224] Handling call
0709 12:18:12.372040 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.554925 (+182885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.556227 (+  1302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:12.582041 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.582119 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:12.536137 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.557336 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.558414 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:12.558888 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:12.582526 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.583582 28283 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 5: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:12.577701 28285 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:12.583091 28367 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:12.583688 28358 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.583948 28292 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:12.582764 28287 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:12.584559 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.585896 28290 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:12.559407 31628 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.564070 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
W20260709 12:18:12.568569 28404 connection.cc:256] RPC call timeout handler was delayed by -1.3881643289998919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
I20260709 12:18:12.571038 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:12.595597 28363 raft_consensus.cc:606] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:12.596025 28363 raft_consensus.cc:614] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:12.571419 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:12.595602 28404 connection.cc:256] RPC call timeout handler was delayed by -1.3215643290000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
I20260709 12:18:12.574262 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.527652 31440 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:12.575546 28284 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.578176 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.597154 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:12.578858 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:12.587751 31636 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.587827 31484 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.589962 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.587949 31604 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.565665 31366 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.622486 31366 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.624302 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:12.625456 31573 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:12.627239 31604 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:12.596988 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1541643289999683s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 12:18:12.636857 28404 connection.cc:256] RPC call timeout handler was delayed by -1.0160461830000713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
I20260709 12:18:12.601804 28291 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:12.552263 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:18:12.638337 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 86173.2us
I20260709 12:18:12.638559 28286 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:12.603976 28986 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:12.611249 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:12.611776 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:12.640478 28284 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:12.612730 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:12.614795 28359 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:12.640718 28285 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:12.606786 28366 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
W20260709 12:18:12.616022 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 94757.7us
I20260709 12:18:12.610731 31589 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:12.607308 31304 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:18:12.618408 31515 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.669696 28438 raft_consensus.cc:2410] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:12.670543 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.618242 31634 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:12.619328 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:12.621317 31286 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:12.578488 31655 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.644443 31477 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.677168 31586 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.678160 31655 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:12.676352 31286 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:12.679543 31286 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.680658 28990 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:18:12.681102 31286 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.675650 31490 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.681769 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 1809) took 3207 ms. Trace:
I20260709 12:18:12.675081 31594 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:12.682191 28438 rpcz_store.cc:276] 0709 12:18:09.474612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.576220 (+101608us) service_pool.cc:224] Handling call
0709 12:18:12.681529 (+3105309us) spinlock_profiling.cc:243] Waited 3.09 s on lock 0x7b54003ca854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:12.681680 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7111762816}
I20260709 12:18:12.681947 31490 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.560148 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.686347 31490 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:12.682245 31304 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.607136 31440 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:12.688325 31440 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.689093 31366 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.644131 31589 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.690124 31589 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.630611 28956 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.646458 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:12.696036 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49673.2us
I20260709 12:18:12.647629 31636 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:12.651758 28187 catalog_manager.cc:5697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 5 to 7, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 7 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:12.655732 31581 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.658389 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.662097 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.700941 28367 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:12.701519 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:12.667023 31307 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.668042 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.709355 28362 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:12.644789 31569 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.672405 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:12.607429 31131 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:12.670164 31580 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.674013 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:12.674707 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55453.7us
I20260709 12:18:12.715114 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:12.668694 31484 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:12.628150 28365 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:12.682698 31594 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:12.664793 31653 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.703394 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:12.723523 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:12.724840 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:12.636101 28288 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:12.672489 28358 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:12.710618 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:12.698943 28187 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:18:12.706284 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 75759.3us
I20260709 12:18:12.679457 31586 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:12.703102 31304 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:12.721909 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 58613.2us
I20260709 12:18:12.644250 31489 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.713404 31652 raft_consensus.cc:2764] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:12.711670 31490 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:12.711850 31286 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.743317 28330 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:12.742820 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 68925.4us
I20260709 12:18:12.732563 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:18:12.791791 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 48567.6us
I20260709 12:18:12.786518 28432 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:12.751894 28438 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:12.744882 31304 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.721170 31594 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.718587 31131 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.788916 28187 catalog_manager.cc:5697] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 4 to 6, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:12.721185 31441 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.804642 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:12.793191 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:12.745201 31513 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.806095 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.806543 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:12.795701 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 63229.7us
I20260709 12:18:12.713565 31336 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 12:18:12.742146 31582 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.798084 28365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
W20260709 12:18:12.798910 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 88408.2us
I20260709 12:18:12.838694 28365 raft_consensus.cc:606] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:12.839015 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:12.730489 31655 raft_consensus.cc:2764] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:18:12.800959 28190 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 223) took 18396 ms (client timeout 15000 ms). Trace:
I20260709 12:18:12.721536 31580 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.802409 31659 consensus_queue.cc:1059] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:12.803217 28187 catalog_manager.cc:5697] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 4 to 7, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 7 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:12.807792 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.785889 31423 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.814585 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.818046 31541 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:12.819082 31538 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:12.819664 31551 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:12.804263 31598 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.789186 31585 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.814175 31441 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.825922 28360 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 5.
I20260709 12:18:12.835566 31476 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.839116 28365 raft_consensus.cc:614] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:12.742802 31550 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:18:12.730746 31593 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:18:12.797619 31304 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.851504 31541 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:12.859002 31551 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.853511 31366 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:12.857580 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49871.8us
I20260709 12:18:12.905412 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:12.906845 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:12.804833 31131 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.804556 31590 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:12.821652 31307 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:12.746399 31527 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:18:12.877480 28187 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:12.878444 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.814628 31513 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.841689 31582 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.850756 31336 consensus_queue.cc:1243] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:18:12.841532 28190 rpcz_store.cc:269] 0709 12:17:54.404552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.404892 (+   340us) service_pool.cc:224] Handling call
0709 12:18:12.800871 (+18395979us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:05.941857 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:05.942641 (+   784us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:05.942695 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:18:05.942714 (+    19us) tablet.cc:662] Decoding operations
0709 12:18:06.046094 (+103380us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:06.046208 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:06.046238 (+    30us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:18:06.149883 (+103645us) tablet.cc:701] Row locks acquired
0709 12:18:06.149930 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:18:06.150124 (+   194us) write_op.cc:270] Start()
0709 12:18:06.150298 (+   174us) write_op.cc:276] Timestamp: P: 1783599486150052 usec, L: 0
0709 12:18:06.150317 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:18:06.151149 (+   832us) log.cc:844] Serialized 7497 byte log entry
0709 12:18:06.533513 (+382364us) op_driver.cc:464] REPLICATION: finished
0709 12:18:06.674606 (+141093us) write_op.cc:301] APPLY: starting
0709 12:18:06.694251 (+ 19645us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:06.710896 (+ 16645us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:06.711042 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:06.735026 (+ 23984us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:06.735564 (+   538us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 12:18:06.735612 (+    48us) write_op.cc:312] APPLY: finished
0709 12:18:06.736096 (+   484us) log.cc:844] Serialized 132 byte log entry
0709 12:18:06.736411 (+   315us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:06.736608 (+   197us) write_op.cc:454] Released schema lock
0709 12:18:06.736684 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:06.799759 (+ 63075us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:06.804618 (+  4859us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":610577,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":111557,"apply.run_cpu_time_us":5910,"apply.run_wall_time_us":109754,"num_ops":19,"prepare.queue_time_us":251888,"prepare.run_cpu_time_us":29713,"prepare.run_wall_time_us":380906,"raft.queue_time_us":177342,"raft.run_cpu_time_us":12190,"raft.run_wall_time_us":57160,"replication_time_us":383004,"thread_start_us":448746,"threads_started":4,"wal-append.queue_time_us":191806,"wal-append.run_cpu_time_us":15784,"wal-append.run_wall_time_us":1397653}]]}
I20260709 12:18:12.794932 31586 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:12.877274 31598 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.883733 31441 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.933523 31440 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.858240 31655 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:12.884087 31634 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.980890 31634 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:12.981415 31634 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:12.906626 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 67701us
I20260709 12:18:12.982446 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:12.982627 31580 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:12.883627 31423 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:12.984495 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:12.908128 31522 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 12:18:12.913439 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.883874 31476 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:12.906291 31541 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:12.906059 31551 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.919660 31582 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.881343 31639 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.018307 31639 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.919514 31513 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.024415 28187 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 4 to 5, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 5 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:13.027596 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.027846 31582 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.986220 31423 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:13.028825 31423 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.030664 31582 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:13.036710 28432 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:13.039039 31562 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:12.883494 31585 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.916100 31652 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:13.042145 31652 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:13.042634 31652 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.043586 31585 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:13.046780 31639 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:12.916074 31629 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.048841 31629 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:13.049463 31629 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.924142 31597 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:18:12.963116 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1827) took 2426 ms. Trace:
I20260709 12:18:13.056459 28360 rpcz_store.cc:276] 0709 12:18:10.536858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.123828 (+586970us) service_pool.cc:224] Handling call
0709 12:18:12.962901 (+1839073us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54003babd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:12.963037 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3908358912}
I20260709 12:18:12.936199 31598 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.968097 31628 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:12.968931 31573 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:12.971422 31441 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.974045 31586 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:12.975463 31663 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:18:12.908849 31568 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:18:13.065063 31568 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:13.065413 31568 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:12.979597 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.911176 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:13.066510 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 155404us
I20260709 12:18:12.989910 31550 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:13.067374 31550 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:12.973569 31665 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.224s	user 0.002s	sys 0.000s
I20260709 12:18:13.067771 31550 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.988444 28365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:12.991515 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.067781 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:13.061939 31628 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:13.070340 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:12.992142 31667 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:13.071002 31538 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:13.070631 31628 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:12.997848 31489 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:13.071439 31667 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.071880 31667 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:13.072232 31667 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.913187 31590 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:13.073038 31590 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.999523 31476 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.008422 31653 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:13.010376 31588 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:13.075253 31588 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:13.011294 31522 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:13.075943 31522 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:13.076217 31588 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.076923 31522 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.883143 31654 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.104516 31654 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:13.105388 28435 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:13.108366 31550 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.110443 31496 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:13.111420 31496 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:13.111747 31496 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:13.112414 31550 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.112915 31496 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:13.059937 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 223835us
W20260709 12:18:13.061381 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 246816us
I20260709 12:18:13.114115 31496 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.062322 31573 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:13.114972 31597 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:12.984988 31581 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:13.077524 31588 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.116030 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:13.078713 31586 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.116101 31597 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:18:13.118772 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:13.119158 31380 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:13.080216 31477 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:13.120570 31477 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:13.082129 31441 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.082230 31580 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:13.115932 31573 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.123541 31597 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:13.083478 31634 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.088804 31513 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.089988 31655 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:13.090493 31582 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:13.125095 31582 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:13.125447 31582 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:13.126391 31582 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.127580 31582 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.128517 31671 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:13.130084 31671 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:13.130456 31671 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:13.131431 31671 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.132843 31671 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.134747 31536 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:13.071415 31538 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:13.136421 31538 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.136798 31536 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:13.137006 28367 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:13.137324 31536 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:13.138343 31536 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.138130 28187 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 4 to 7, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 7 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:13.139297 31634 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.139678 31536 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.136960 31513 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:13.139912 28187 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 4 to 9, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 9 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:13.123960 31597 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:13.146359 31522 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.089406 31629 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:12.986838 31589 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.151993 31589 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.155923 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:13.144959 31597 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.156745 28363 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:13.157382 31597 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.119424 28986 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.118805 31639 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:13.121496 31477 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.166324 31639 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:12.904042 31484 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:13.167019 31477 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.123946 31573 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.167243 31639 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.167670 31484 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.168557 31639 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.082499 31653 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.091426 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:13.134407 31568 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.096783 31628 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.171535 31653 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.172055 31653 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:13.069629 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 78225.9us
I20260709 12:18:13.174384 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.175921 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.176748 31441 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:13.176883 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.177240 31441 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:13.177650 31441 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.178498 31538 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:13.179563 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.179981 31568 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.186601 31669 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:13.187201 31669 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:13.187538 31636 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:13.188138 31636 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:13.189946 31489 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:13.190531 31489 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:13.191430 31489 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.188151 31669 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.189060 31636 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:13.198074 31515 consensus_meta.cc:287] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.006s
I20260709 12:18:13.198737 28359 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:13.205018 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:13.205819 28986 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:13.157847 31131 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.161661 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:13.161949 31440 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:13.163570 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:13.235389 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 144059us
I20260709 12:18:13.097141 31667 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.212275 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:13.241192 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:13.222437 28284 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:13.242728 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.243037 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.243944 28284 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:13.224176 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.244627 28292 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:13.225178 28283 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:13.058775 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 342367us
I20260709 12:18:13.210415 28363 raft_consensus.cc:2376] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9. Re-sending same reply.
I20260709 12:18:13.239164 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.246593 28432 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:13.239933 28290 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:18:13.247622 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.208793 28187 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 3 to 6, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:13.245546 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.249336 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.240057 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.249693 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.241190 28986 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:13.251710 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.250464 28292 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.254783 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.254866 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.255668 28436 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.255741 31336 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:13.256507 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.257140 28441 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.257428 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.258142 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.259891 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.260692 31336 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:13.261088 31336 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:13.261003 28433 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.261619 31336 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.260710 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.263365 28437 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.272822 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.276460 28364 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:13.278033 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.278682 28283 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.250074 28289 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:13.280954 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.281395 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:13.282102 28364 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:13.282195 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.283835 28291 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:13.159058 31629 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:13.290539 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:13.201853 31594 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.318018 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.318845 28289 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:13.319094 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.320108 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.293916 28436 raft_consensus.cc:2410] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:13.320832 28284 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:13.321252 28285 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:13.297277 28437 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:13.322438 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:13.322513 28287 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:13.315953 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.290339 31307 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.283056 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.323699 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:13.330485 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.331053 28437 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.302582 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:13.321455 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:13.324470 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 74346.8us
I20260709 12:18:13.318070 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.324541 28187 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 2 to 3, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:13.325192 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.332525 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:13.332259 28436 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:13.333922 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.334291 28290 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:13.335008 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:13.335224 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.332803 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.336057 28332 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:13.333935 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.337978 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:13.339155 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.341217 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:13.340049 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66287.1us
I20260709 12:18:13.342564 28286 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.345505 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.347399 28289 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:13.348453 28291 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:13.248273 31513 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.350644 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.355149 28432 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:13.349606 31659 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.381141 28187 catalog_manager.cc:5697] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 2 to 3, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:13.279776 31550 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.290472 31589 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:13.318372 31582 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:13.316641 31654 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.364140 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:13.423647 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:13.429078 28438 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:13.365978 28360 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:13.429383 28432 raft_consensus.cc:2410] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:13.364907 28441 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:13.431915 28432 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:13.372874 28292 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:13.273803 31336 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.374828 28287 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:13.375605 28437 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:13.375759 28285 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:13.324139 31587 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.322952 31527 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:13.097561 31551 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.381829 28433 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
W20260709 12:18:13.362517 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 79523.1us
W20260709 12:18:13.394584 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.341655 31545 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:13.331511 31440 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.278724 31581 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:18:13.277679 31655 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 12:18:13.347259 31515 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:13.240284 31653 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:13.405781 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.347872 31520 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:13.426276 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:13.278048 31304 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.370808 28283 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:13.364332 31553 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:13.364387 31673 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.363294 31286 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.380306 31541 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.369191 31380 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:13.293532 31580 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 12:18:13.316879 31423 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.434870 28437 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:13.435802 28438 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:13.446550 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.326418 31585 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:18:13.348178 31441 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.379921 31659 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:13.325520 31575 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 12:18:13.422487 31589 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:13.480273 28441 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 4. Current term is 6. Ops: []
I20260709 12:18:13.433244 28982 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 9 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:13.428712 31654 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:13.425880 31582 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:13.443197 31629 raft_consensus.cc:2764] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:13.376192 31670 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.127s
I20260709 12:18:13.444187 31527 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.443922 31587 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:13.423190 31598 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.363360 31674 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 12:18:13.517920 28361 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:13.374573 31513 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.536949 31659 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:13.524403 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.495515 31515 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.536473 31588 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:13.454020 31550 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:18:13.501415 31520 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:13.513491 31617 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:13.498274 31545 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:13.521889 31286 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:13.504191 31440 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.603056 31441 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.603820 31440 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.555742 28367 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:13.604521 31423 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.606043 31598 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.578066 31515 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.609807 31286 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:13.555537 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 179364us
I20260709 12:18:13.530908 31553 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:13.523378 28432 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:13.513698 31476 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.546446 31589 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:13.630844 28362 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.567361 31304 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.554916 31131 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:13.571803 31652 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.529s	user 0.005s	sys 0.004s
I20260709 12:18:13.644286 31652 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.646636 31652 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:13.557081 31582 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.573203 28990 raft_consensus.cc:2410] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:13.575330 28284 raft_consensus.cc:2410] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:18:13.529155 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 82695.1us
W20260709 12:18:13.576807 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.635608 31545 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:13.577777 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.651846 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:13.555366 31654 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.652789 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:13.653581 31585 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.655167 31585 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:13.580302 28365 raft_consensus.cc:606] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.582316 28436 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:13.557049 31527 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.655678 31585 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:13.656692 31585 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.560279 31587 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.655699 28365 raft_consensus.cc:614] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:13.671718 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:13.587807 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 244746us
I20260709 12:18:13.593217 31520 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:13.601821 31336 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:13.600840 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 231995us
I20260709 12:18:13.624696 31553 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.625820 31612 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:13.627461 31476 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:13.634519 31513 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:13.597954 31588 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:13.635710 31541 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:13.637071 31594 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:13.638664 31671 consensus_meta.cc:287] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.000s
I20260709 12:18:13.672835 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.700229 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:13.683496 31652 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:13.708328 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:13.709182 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:13.709579 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:13.710227 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:13.709201 28366 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 5.
I20260709 12:18:13.712060 28360 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:13.683943 28358 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:18:13.684082 28433 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:13.665343 31590 consensus_meta.cc:287] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.000s
I20260709 12:18:13.685891 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:13.686204 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 315160us
W20260709 12:18:13.692092 31597 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.000s
I20260709 12:18:13.612993 31667 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:13.698824 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.694911 31536 consensus_meta.cc:287] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.555s	user 0.003s	sys 0.008s
I20260709 12:18:13.676378 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.723699 31688 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:13.723886 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.724231 31688 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:13.724646 31688 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.726100 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:13.729658 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:13.723750 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:13.731333 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.732581 31671 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:13.733003 31671 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:13.733359 31671 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.735327 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:13.744261 31590 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.750555 28441 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:13.750634 31513 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:13.751869 31588 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.752724 31513 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:13.753692 31513 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.757390 31667 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.758003 31667 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.758330 31653 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:13.751789 31652 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:13.760259 31652 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.766795 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.768121 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:13.732221 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55956.8us
I20260709 12:18:13.770243 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:13.703013 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 204940us
I20260709 12:18:13.702644 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.704734 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:13.772843 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 70291.6us
I20260709 12:18:13.712179 28362 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.681442 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:13.716243 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.717921 28292 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:13.774559 28432 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.721896 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:13.722247 28363 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:13.776036 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.776526 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:13.777341 28365 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:13.774010 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.782883 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.783926 28437 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.784900 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.782040 28366 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:13.790885 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:13.791514 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:13.798992 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:13.799691 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:13.800419 31581 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.800845 31581 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:13.801206 31581 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.789600 31617 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.802155 31617 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:13.802702 31617 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.803601 31687 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:13.804497 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:13.804652 31694 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.781327 31551 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:13.805207 31694 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.806152 31667 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.806720 31590 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:13.807117 31693 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:13.807549 31693 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:13.808460 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.809800 31690 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:13.810060 31695 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:13.776823 28332 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:13.812744 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:13.813423 28292 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:13.813565 28990 raft_consensus.cc:2410] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:13.814599 28437 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:13.814749 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:13.816624 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:13.819067 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.808701 31304 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.823446 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:13.824498 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.825490 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.826248 28283 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.828444 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.832223 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:13.832857 28363 raft_consensus.cc:606] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.833201 28363 raft_consensus.cc:614] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:13.835417 28283 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:13.835628 28986 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:13.822124 31538 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:13.829970 31484 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:13.836961 31551 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:13.837159 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.838582 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.825647 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.838869 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:13.838892 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 62155.6us
I20260709 12:18:13.851620 28291 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:13.853065 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.831764 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.839421 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:13.854303 31521 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:13.822537 31131 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:13.858530 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:13.807480 31694 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:13.840718 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:13.843561 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 44654.4us
I20260709 12:18:13.843564 28330 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.845181 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.845389 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.846383 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.731926 31536 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:18:13.870973 31695 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:13.871426 31695 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:13.872279 31695 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.873201 31551 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:13.789637 31653 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.874313 31696 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:13.875303 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:13.876245 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:13.876637 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44951.3us
I20260709 12:18:13.789991 31307 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:13.829700 31598 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.853701 28291 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:13.849824 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.811733 31690 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.859598 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.861020 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:13.849330 31476 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:13.900591 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:13.901351 28359 raft_consensus.cc:606] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:13.901851 28359 raft_consensus.cc:614] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:13.868712 28292 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:13.869907 28288 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:13.858434 31423 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:18:13.872632 28332 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.790048 31597 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:13.849663 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.917579 28283 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.874747 31696 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:13.843561 31550 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 12:18:13.847020 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 54028.5us
I20260709 12:18:13.935436 28292 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:13.936856 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:13.877846 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.809505 31589 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.802358 31659 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.883776 28287 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:13.835806 28432 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:13.896657 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.843223 31590 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:13.898957 28289 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:13.842919 31636 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.873538 31695 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.877697 31484 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:13.861697 31131 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:13.873338 31671 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.836721 31639 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:18:13.859597 31489 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.916725 31569 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.830431 31587 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.904824 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.854236 28437 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:18:13.923718 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:13.808506 31693 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.802091 31582 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.848039 31440 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:18:13.884470 31673 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:13.848201 31670 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:18:13.883272 31551 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:13.944409 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 66719.6us
I20260709 12:18:13.980777 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.884119 31653 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:13.950878 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:13.981726 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 104257us
I20260709 12:18:13.900256 31690 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:13.954252 28283 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:13.863340 31286 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.958874 28290 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:13.946596 31597 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:13.984002 31597 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:13.984894 31286 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:13.966162 28956 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.949064 31545 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:13.990123 28434 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:13.973209 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 49611.1us
I20260709 12:18:13.916705 31515 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.948890 31590 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:13.988281 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:13.991278 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:13.952888 31628 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:18:14.008672 28440 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:14.015326 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.043818 28283 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:13.967559 31669 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:14.048630 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 57410.1us
W20260709 12:18:13.974365 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 101822us
I20260709 12:18:13.989699 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.980136 31304 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:14.052945 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:14.059715 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 70088.8us
I20260709 12:18:14.063318 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.989490 31585 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:13.986477 31693 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.974298 31569 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.013015 31286 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:14.042171 31545 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:14.002393 31653 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:14.062849 31590 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:14.069660 31477 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.891s	user 0.012s	sys 0.000s
I20260709 12:18:14.058287 31551 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.063675 31489 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:14.066419 31580 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:14.070355 31476 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.093983 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:13.978952 31688 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:13.986896 31527 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.059541 31669 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:14.154644 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 60814.9us
I20260709 12:18:14.155771 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:14.156664 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:14.085912 31573 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.093433 31307 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:14.092391 31585 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 12:18:14.092607 31286 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.128636 31536 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:14.070540 31654 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:13.993276 31696 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.190142 31545 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.154845 31580 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:14.198282 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 205015us
I20260709 12:18:14.197476 31652 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:14.137703 31704 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.139s
I20260709 12:18:14.167371 31689 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:14.209573 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 158225us
W20260709 12:18:14.254822 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 60360us
W20260709 12:18:14.215170 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 136361us
W20260709 12:18:14.256323 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.095726 31131 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.155710 31683 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:14.217566 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:18:14.228793 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 72199.1us
I20260709 12:18:14.204664 31286 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:14.251806 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.254161 31653 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.175860 31423 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:18:14.205173 31629 raft_consensus.cc:2764] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:14.258548 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:14.250597 31652 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:14.252823 31659 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:14.261771 28435 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:14.091876 31484 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.204955 31536 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:14.252496 31580 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:14.253129 31715 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:14.272583 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 55120.4us
W20260709 12:18:14.216271 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.278918 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:14.280323 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:14.255517 31589 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:14.297674 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:14.304256 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:14.305514 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:14.306663 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:14.286983 28283 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:14.310254 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:14.274336 31671 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:14.311337 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.315654 28367 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: [6.6-8.7]
I20260709 12:18:14.311640 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.275486 31587 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:14.276490 31628 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.317620 31569 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:14.319959 31717 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:14.285403 31536 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.324340 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:14.320247 31698 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 12:18:14.282599 31582 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:14.287097 31683 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:14.253273 31551 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.302253 31715 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:14.266144 31689 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.284257 31652 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.328325 31538 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:14.256129 31476 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.320849 31515 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:14.333078 31703 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:18:14.250797 31669 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.356873 31669 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.343087 31715 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:14.283391 31712 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:18:14.318694 31629 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:18:14.335419 31639 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:14.318959 31307 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:14.355347 31719 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:18:14.337208 31683 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:14.356130 31708 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:18:14.362241 31495 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:14.336726 31725 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:18:14.320390 31714 consensus_queue.cc:1059] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:18:14.368820 31688 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:14.375259 31495 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:14.370016 31710 consensus_queue.cc:1175] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f526369c0d0743b192be6e205540a059" responder_term: 8 status { last_received { term: 8 index: 7 } last_committed_idx: 5 last_received_current_leader { term: 8 index: 7 } }
I20260709 12:18:14.380003 31684 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:14.384981 31654 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:14.377429 31587 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:14.387938 31587 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:14.389187 31598 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.388835 31587 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.390199 31587 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:14.390554 31581 consensus_meta.cc:287] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.000s
I20260709 12:18:14.391029 31581 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.393018 31590 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.384994 31684 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:14.397661 31683 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.400023 31683 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.392160 31690 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.398299 31684 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.415364 31707 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:14.417456 31713 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:14.418339 31713 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.419036 31713 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.420495 31713 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:14.451025 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.330238 28330 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:14.461508 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 131157us
I20260709 12:18:14.463892 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:14.474355 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:14.333017 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 66413.4us
I20260709 12:18:14.366624 28986 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:14.484887 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:14.485574 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56626 (request call id 371) took 1322 ms. Trace:
I20260709 12:18:14.486308 28358 rpcz_store.cc:276] 0709 12:18:13.162757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.163290 (+   533us) service_pool.cc:224] Handling call
0709 12:18:14.485061 (+1321771us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54003bbfd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:14.485285 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033611136}
I20260709 12:18:14.488550 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.488817 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:14.364658 28982 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.000s
I20260709 12:18:14.490844 28982 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:14.492816 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.494545 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:14.498009 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:14.498517 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:14.500105 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:14.503183 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2100) took 1188 ms. Trace:
I20260709 12:18:14.412660 31673 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:14.522282 31380 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:14.523818 31700 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.453s	user 0.000s	sys 0.004s
I20260709 12:18:14.505059 28435 rpcz_store.cc:276] 0709 12:18:13.314482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.529200 (+214718us) service_pool.cc:224] Handling call
0709 12:18:13.550386 (+ 21186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:14.281017 (+730631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.502554 (+221537us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b540026a2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:14.502894 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624447360}
I20260709 12:18:14.522393 31495 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.548291 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.552386 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:14.553037 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:14.554500 28365 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:14.554148 31513 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:14.555191 31639 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:14.555711 31639 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.557163 31673 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.562230 28989 raft_consensus.cc:2410] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:14.560894 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 420) took 1244 ms. Trace:
I20260709 12:18:14.561107 28364 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:14.568286 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.417522 31707 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.577502 31515 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:14.587162 31594 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:14.587790 31594 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.589087 31689 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:14.589854 31520 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.896s	user 0.002s	sys 0.011s
I20260709 12:18:14.590936 31612 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:14.591336 31612 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:14.591632 31590 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:14.593029 31581 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:14.592738 31612 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.591636 28441 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:14.597824 31380 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.599262 31714 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:14.599706 31380 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.567513 28290 rpcz_store.cc:276] 0709 12:18:13.271061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.349388 (+ 78327us) service_pool.cc:224] Handling call
0709 12:18:13.380706 (+ 31318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.983630 (+602924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.475494 (+491864us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b540009ecd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:14.515223 (+ 39729us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329194880}
I20260709 12:18:14.602690 31522 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:14.603338 31522 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:14.573599 28988 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:14.602994 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 114419us
I20260709 12:18:14.575750 31545 raft_consensus.cc:2764] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:18:14.610049 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.570937 31673 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.579891 31598 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.603617 31513 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:14.590608 31520 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:14.625622 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.628229 31477 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:14.613444 31515 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.628437 31714 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.620054 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:14.626446 31513 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.638438 28367 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:14.644721 31527 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:14.589316 31594 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.646081 31654 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.646760 31520 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:14.648129 31612 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:14.648296 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:14.632208 31477 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:14.656522 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.657642 28367 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:14.658176 31617 consensus_meta.cc:287] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.855s	user 0.016s	sys 0.000s
I20260709 12:18:14.647075 31594 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:14.658821 31617 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.662602 31653 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:14.663339 28434 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:14.663519 31653 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.667030 31690 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:14.668049 31653 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:14.670691 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.628561 31495 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.670029 31707 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.658895 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.674966 31586 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:14.656929 31732 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:18:14.673919 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:14.643790 31513 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:14.687156 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 52505.3us
I20260709 12:18:14.685504 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:14.693531 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:14.693681 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:14.704196 28367 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 5: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:14.705586 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:14.705592 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 46317.1us
W20260709 12:18:14.779084 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 71694.2us
W20260709 12:18:14.780992 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.647141 31654 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:14.778406 28257 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.000s	sys 0.034s
W20260709 12:18:14.796101 28257 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.000s	sys 0.035s
I20260709 12:18:14.791368 28985 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:14.799400 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:14.802842 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.803896 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.805433 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.703951 31586 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.796929 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:14.817409 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.817708 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.817863 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 90309.8us
W20260709 12:18:14.897967 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.898459 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.900437 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.901821 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.857609 31714 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:14.822028 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:14.906176 28434 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:14.907285 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:14.907894 28434 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:14.824821 31707 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:14.825047 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.825897 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:14.827821 28437 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:14.912472 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:14.913280 28437 raft_consensus.cc:2368] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:14.830298 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:14.832381 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:14.833428 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:14.833498 31304 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:14.834926 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:14.835254 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:14.840091 31743 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.053s	user 0.002s	sys 0.000s
I20260709 12:18:14.695118 31522 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.857920 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:14.861179 31654 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:14.932876 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:14.933804 28436 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:14.927529 31673 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:14.935782 31733 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.936443 31733 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:14.941038 31286 consensus_meta.cc:287] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.662s	user 0.001s	sys 0.015s
I20260709 12:18:14.941608 31286 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.943051 28990 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7.
I20260709 12:18:14.946143 31286 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:14.867307 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1872) took 1051 ms. Trace:
I20260709 12:18:14.868615 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1864) took 1070 ms. Trace:
W20260709 12:18:14.869488 31536 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.529s	user 0.005s	sys 0.006s
I20260709 12:18:14.969961 31536 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.972088 31536 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:14.962357 28441 rpcz_store.cc:276] 0709 12:18:13.815933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.817629 (+  1696us) service_pool.cc:224] Handling call
0709 12:18:13.836456 (+ 18827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:14.594275 (+757819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.866987 (+272712us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b54002e40d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:14.867212 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1625902336}
W20260709 12:18:14.973997 31652 consensus_meta.cc:287] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.614s	user 0.008s	sys 0.008s
I20260709 12:18:14.974417 28441 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:14.962649 28365 rpcz_store.cc:276] 0709 12:18:13.798137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.799358 (+  1221us) service_pool.cc:224] Handling call
0709 12:18:14.868360 (+1069002us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54001d8154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:14.868551 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703233792}
I20260709 12:18:14.976327 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:14.977809 31587 consensus_meta.cc:287] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.008s
I20260709 12:18:14.979146 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:14.980479 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:14.987546 31588 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:14.988372 31588 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:14.988814 31588 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:14.991739 31597 consensus_meta.cc:287] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.007s	user 0.012s	sys 0.006s
I20260709 12:18:14.992194 31597 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.993445 28363 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:14.869870 31617 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:14.995101 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.995755 28363 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:14.996708 31714 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:14.997414 31714 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.999486 31714 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:14.999933 31714 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.000075 31588 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.000452 31714 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.002108 31674 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.002645 31674 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:14.871160 31586 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:14.871798 31693 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.006470 28439 raft_consensus.cc:2410] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:14.799409 31694 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:18:14.877835 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:14.881089 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:14.881255 28435 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:18:14.883081 28982 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:14.884781 31515 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:14.886749 28366 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.888844 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:15.007544 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.893556 31585 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.909504 28291 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:14.919760 31527 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:14.920835 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.921325 31704 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:14.921954 31684 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:14.922531 31520 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:14.947465 31725 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:14.921876 31690 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.953006 28433 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:14.957175 31701 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:14.957903 31639 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.032143 31639 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.033915 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1851) took 1201 ms. Trace:
I20260709 12:18:15.034324 28285 rpcz_store.cc:276] 0709 12:18:13.832657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.833517 (+   860us) service_pool.cc:224] Handling call
0709 12:18:15.033517 (+1200000us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b540005a0d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:15.033688 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1824848128}
W20260709 12:18:14.958853 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.864941 28359 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
W20260709 12:18:14.963881 31580 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.647s	user 0.015s	sys 0.000s
W20260709 12:18:14.964433 31715 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.606s	user 0.000s	sys 0.015s
I20260709 12:18:14.965277 28437 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:14.966812 31733 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:15.004528 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.037415 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:15.039290 31586 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:15.045603 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.038523 28438 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:15.046576 31527 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:15.047672 31704 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.049167 31527 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:15.049940 31704 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:15.050164 28432 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:15.050590 31706 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.046602 31440 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.051194 31706 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.051451 31440 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.053406 31706 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.054338 31585 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.055080 31694 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:15.055786 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:15.058192 31440 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.059015 31515 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.059490 31520 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.060436 31587 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.060496 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:15.047380 31538 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:15.040010 31690 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:15.063671 31683 consensus_meta.cc:287] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.650s	user 0.007s	sys 0.005s
I20260709 12:18:15.064129 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:15.066776 31674 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:14.974478 31652 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.069790 31715 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.049800 31693 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.071086 31580 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.072408 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:15.073372 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 192417us
I20260709 12:18:15.073964 31733 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:15.058373 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 169829us
I20260709 12:18:15.083868 28363 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:15.089677 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:15.093312 28436 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:15.095933 28360 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:15.097602 28366 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
W20260709 12:18:15.099475 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 191380us
I20260709 12:18:15.084574 31694 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:15.107877 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:15.108573 28363 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:15.108667 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:15.106467 28365 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:15.116592 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:15.092166 31587 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.103331 31440 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.107177 31286 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.134003 28366 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.117228 31538 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.103276 31706 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.138051 28987 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:15.134490 28360 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:18:15.098445 31513 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:15.143131 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 79178.4us
I20260709 12:18:15.148705 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:15.144467 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:15.118057 31733 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:15.115254 31674 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:15.167271 31585 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:15.169206 31690 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:15.170488 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:15.171041 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:15.171551 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 476) took 1853 ms. Trace:
W20260709 12:18:15.171393 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55996.6us
I20260709 12:18:15.198410 28434 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:15.171836 28990 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:15.137600 31696 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.171945 28440 rpcz_store.cc:276] 0709 12:18:13.256450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.260033 (+  3583us) service_pool.cc:224] Handling call
0709 12:18:13.260284 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:14.043762 (+783478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.090895 (+1047133us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54002e36d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:15.109482 (+ 18587us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1709370240}
I20260709 12:18:15.142880 31714 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:15.206905 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49229.3us
W20260709 12:18:15.208215 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 74332.9us
I20260709 12:18:15.151500 31527 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:15.210712 31689 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:15.211501 31694 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.212347 31689 raft_consensus.cc:3055] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:15.212828 31689 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:15.210981 31527 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:15.215018 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.171511 31476 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:15.258030 31713 consensus_meta.cc:287] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.683s	user 0.002s	sys 0.011s
I20260709 12:18:15.257805 31527 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.261477 31380 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:15.262133 31380 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:15.262568 31380 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:15.270923 31742 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:18:15.272754 31733 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.276194 31666 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:15.277593 31477 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.278790 31477 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.279656 31587 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:15.284269 31522 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.284693 28359 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:15.287737 31580 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:15.288671 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 228257us
I20260709 12:18:15.289474 31612 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:15.290001 31612 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:15.290236 28359 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:15.292196 31597 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.171442 31538 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:15.296969 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.317571 28367 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:15.324605 31580 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:15.277652 31739 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:18:15.329459 28358 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:15.330641 28436 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:15.332563 28434 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:15.333479 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 79097.2us
W20260709 12:18:15.334491 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.334838 28432 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:15.198222 31286 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.337580 28332 leader_election.cc:400] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:15.338022 28332 leader_election.cc:403] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:18:15.341387 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.342635 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:15.344460 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:15.339138 31580 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:15.346431 31580 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.348765 31580 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:15.351830 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 97983.7us
I20260709 12:18:15.354259 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1855) took 1028 ms. Trace:
W20260709 12:18:15.233804 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.356978 28436 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:15.222095 31706 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.354756 28364 rpcz_store.cc:276] 0709 12:18:14.311253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.311390 (+   137us) service_pool.cc:224] Handling call
0709 12:18:15.331474 (+1020084us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54003babd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:15.339745 (+  8271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389391616}
I20260709 12:18:15.307557 31693 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.359809 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:15.222955 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.362634 31589 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.362692 28440 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:15.364650 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 156775us
I20260709 12:18:15.296182 31476 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.362648 31580 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:15.373499 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:15.376724 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:15.378268 31580 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:15.353703 31587 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:15.358805 31694 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.381985 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.384300 28363 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:15.371243 31617 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.307822 31689 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.200991 31440 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.399973 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:15.422256 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.392637 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:15.427006 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:15.386557 31538 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.389962 31484 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.426755 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.432487 28436 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:15.427340 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.389065 31580 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:15.435796 31617 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.436200 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.436767 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.390801 31652 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:15.437515 28285 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:18:15.438031 31634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.061s	sys 0.110s
I20260709 12:18:15.438573 28364 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:15.440198 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.441001 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:15.441670 28291 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 6: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:15.363374 31589 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.447796 31380 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.450958 31286 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.453501 28362 raft_consensus.cc:2410] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:15.453223 31484 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:15.457649 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.481783 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.482606 28440 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:15.472172 28292 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.473322 28432 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:15.422209 31715 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:15.478411 28367 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:15.479861 28365 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:18:15.468484 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 91928.9us
I20260709 12:18:15.488410 28286 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:15.492157 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.492233 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.475412 31634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.061s	sys 0.111s
I20260709 12:18:15.494422 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.474311 31683 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.501124 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:15.502337 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.495404 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 102863us
I20260709 12:18:15.506166 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:15.497601 28366 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:15.508311 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:15.510221 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.510375 28366 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:15.478173 31597 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:15.511730 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 70829.3us
W20260709 12:18:15.514567 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.494966 31585 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:15.517138 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.520139 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:15.430698 31612 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.529016 28291 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:15.510823 28436 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:15.517249 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.525250 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:15.531446 28364 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:15.498948 31513 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:15.511516 31515 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:15.534454 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:15.511286 31666 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:15.518247 31689 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:15.543959 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:15.545238 28990 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:15.548125 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:15.553326 28332 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:15.557197 28366 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:15.543756 31588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.037s	sys 0.030s
I20260709 12:18:15.539849 31652 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:15.566395 28283 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:15.556521 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:15.562134 28290 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:15.536512 31515 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:15.571050 28285 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:18:15.564672 28364 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:15.512543 31683 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.534461 31639 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:18:15.580745 28985 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:15.571573 28285 pending_rounds.cc:85] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 3
I20260709 12:18:15.584365 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1869) took 1079 ms. Trace:
I20260709 12:18:15.584807 28364 rpcz_store.cc:276] 0709 12:18:14.492674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.545255 (+1052581us) service_pool.cc:224] Handling call
0709 12:18:15.561334 (+ 16079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.571881 (+ 10547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.572120 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:15.562971 31588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.038s	sys 0.031s
I20260709 12:18:15.558542 31597 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.592093 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1870) took 1092 ms. Trace:
W20260709 12:18:15.595510 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 81944.9us
I20260709 12:18:15.564270 31706 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.598435 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:15.599251 28290 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:15.584287 28285 pending_rounds.cc:107] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:18:15.600746 28285 raft_consensus.cc:2889] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:15.594842 28366 rpcz_store.cc:276] 0709 12:18:14.493075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.550401 (+1057326us) service_pool.cc:224] Handling call
0709 12:18:15.550720 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.557689 (+  6969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.585274 (+ 27585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.580351 31515 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.583133 31612 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.610491 28990 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:18:15.613582 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.558518 31585 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.621306 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.562300 31666 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.622543 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.622463 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.621543 28440 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:15.632493 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:15.632772 28433 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:15.597239 31477 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:15.609084 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.632925 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:15.631330 28987 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 6.
I20260709 12:18:15.593535 31652 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:15.636101 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:15.642321 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 117660us
I20260709 12:18:15.610527 31484 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.612151 31131 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:15.651633 28284 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:15.658674 28291 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:15.660049 28364 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:15.640027 31513 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:15.646034 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:15.648027 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.668128 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:15.671782 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.642989 31477 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:15.682561 31477 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.683286 31683 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:15.650228 31131 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.660990 31440 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:15.643846 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:15.677942 28441 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:15.672739 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:15.673959 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:15.688949 28286 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:15.662674 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:15.692133 28982 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:15.696576 31683 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.697227 31683 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.685768 31484 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.698629 31484 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.699482 31683 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:15.685293 31617 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.702080 31652 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:15.706903 31750 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:18:15.708246 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1859) took 1792 ms. Trace:
I20260709 12:18:15.647929 31597 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:15.712503 31597 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.643985 31684 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:18:15.640489 31588 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:18:15.708690 28288 rpcz_store.cc:276] 0709 12:18:13.849084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.903939 (+ 54855us) service_pool.cc:224] Handling call
0709 12:18:13.947582 (+ 43643us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:13.960884 (+ 13302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.960919 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:13.968276 (+  7357us) log.cc:844] Serialized 26 byte log entry
0709 12:18:14.466914 (+498638us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b1800326020. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:18:14.677154 (+210240us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:14.686163 (+  9009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.695575 (+  9412us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.035258 (+339683us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b5400286020. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7fcdbe 00007f85be807930 00007f85be8078d8 00007f85be807858 00007f85be807813 00007f85be806a9c 00007f85c46f22f6 00007f85c46ecaa8 00007f85be8437cb 00007f85be8462c0
0709 12:18:15.058553 (+ 23295us) raft_consensus.cc:1685] finished
0709 12:18:15.058578 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.634516 (+575938us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b540005a5d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:15.641720 (+  7204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12846,"raft.queue_time_us":21530,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":7075,"spinlock_wait_cycles":76862720,"thread_start_us":80014,"threads_started":1,"wal-append.queue_time_us":682384}
I20260709 12:18:15.673089 31513 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:15.669966 31380 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.718248 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:15.717128 28292 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:15.652626 31715 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 12:18:15.733760 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 61120us
W20260709 12:18:15.739502 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 95734.6us
I20260709 12:18:15.740513 31520 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:15.740756 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.742264 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.689057 31131 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:15.747982 28441 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:15.752367 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:15.745818 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 71978.6us
I20260709 12:18:15.753979 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:15.744740 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:15.755241 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:15.759243 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:15.701591 31484 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:15.737272 31513 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:15.735795 31617 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:15.762173 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.746986 31286 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:15.765380 28358 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1867) took 1273 ms (client timeout 1676 ms). Trace:
W20260709 12:18:15.772365 28358 rpcz_store.cc:269] 0709 12:18:14.491720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.515153 (+ 23433us) service_pool.cc:224] Handling call
0709 12:18:15.765190 (+1250037us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54003bc4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:15.765311 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628859776}
I20260709 12:18:15.762609 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:15.753254 31683 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:15.780467 28287 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:15.780989 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:15.783420 28437 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:18:15.758450 31733 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:15.784787 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.788980 28358 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:15.768388 31587 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:15.768698 31520 raft_consensus.cc:2764] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:15.783861 28437 pending_rounds.cc:85] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 3
I20260709 12:18:15.801596 28437 pending_rounds.cc:107] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:18:15.802034 28437 raft_consensus.cc:2889] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:15.772854 31527 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:15.783970 31580 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:15.810343 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.811012 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:15.812139 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:15.783545 31286 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:15.804899 31527 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.821344 28436 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:15.834404 31286 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:15.825889 31588 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
W20260709 12:18:15.839905 31586 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.755s	user 0.010s	sys 0.001s
I20260709 12:18:15.840469 31586 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:15.841142 28284 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:15.842594 31527 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.843878 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1856) took 1532 ms. Trace:
I20260709 12:18:15.844326 28363 rpcz_store.cc:276] 0709 12:18:14.311567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.252932 (+941365us) service_pool.cc:224] Handling call
0709 12:18:15.335982 (+ 83050us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:15.387106 (+ 51124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.843488 (+456382us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b54001f9ad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:15.843753 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110476032}
I20260709 12:18:15.845724 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:15.846716 28363 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:15.847755 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:15.848582 28363 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:15.850033 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:15.850737 28358 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.853339 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:15.854128 28985 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
W20260709 12:18:15.857013 31756 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:18:15.780566 31617 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.853181 31380 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:15.858587 31748 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 12:18:15.858758 31380 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.859386 31617 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:15.859633 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 115462us
W20260709 12:18:15.860812 31522 consensus_meta.cc:287] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.000s
I20260709 12:18:15.861191 31380 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:15.866178 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 73218us
I20260709 12:18:15.866426 31666 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:15.867408 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:15.868405 31666 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:15.813710 31580 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.869979 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:15.870832 31585 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:15.871052 31666 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:15.860635 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:15.874958 28361 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:15.832530 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:15.876904 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:15.849606 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:15.878546 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.879747 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.871733 31733 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:15.880829 28364 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:15.884403 31380 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:15.884894 31380 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.885241 31380 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.888586 28285 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:15.889314 28330 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:15.890707 31639 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:15.890875 31522 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:15.891273 31639 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:15.891700 31751 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.528s	user 0.005s	sys 0.002s
I20260709 12:18:15.891919 31639 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:15.894095 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1866) took 1219 ms. Trace:
I20260709 12:18:15.895526 28437 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:15.895953 28437 pending_rounds.cc:85] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 4
W20260709 12:18:15.896211 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.896179 28437 pending_rounds.cc:107] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:15.896760 28437 raft_consensus.cc:2889] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:15.894539 28362 rpcz_store.cc:276] 0709 12:18:14.674848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.805090 (+130242us) service_pool.cc:224] Handling call
0709 12:18:15.893713 (+1088623us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b540027db54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:15.893998 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486116736}
I20260709 12:18:15.870224 31580 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.878328 31538 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:15.903563 28363 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:15.891597 31522 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:15.885811 31597 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:15.907929 28364 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:15.910938 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1871) took 1006 ms. Trace:
I20260709 12:18:15.911181 28364 rpcz_store.cc:276] 0709 12:18:14.904526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.882438 (+977912us) service_pool.cc:224] Handling call
0709 12:18:15.903356 (+ 20918us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:15.910670 (+  7314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.910879 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.906924 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1869) took 1231 ms. Trace:
I20260709 12:18:15.909204 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.912335 28363 rpcz_store.cc:276] 0709 12:18:14.675952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.895373 (+1219421us) service_pool.cc:224] Handling call
0709 12:18:15.903362 (+  7989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.906612 (+  3250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.906845 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:15.913812 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:15.923818 28365 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:15.929328 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.920089 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.941330 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.944730 28292 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:15.945343 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:15.947724 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:15.951798 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.952402 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.954401 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 93886.1us
I20260709 12:18:15.955295 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:15.959422 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:15.934657 31760 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:15.963384 31527 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.965646 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.966586 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.967303 28288 raft_consensus.cc:2368] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:15.966110 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.966114 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:15.972339 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:15.973045 28985 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:15.975124 28358 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:15.980475 28360 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:15.985862 28289 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:15.986083 28358 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:15.986554 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:15.987113 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1872) took 1052 ms. Trace:
I20260709 12:18:15.978962 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:15.989498 28285 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:15.917963 31597 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:15.989248 28432 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:15.994952 28437 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:15.969515 31476 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:15.999267 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:16.003949 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:15.987923 28358 rpcz_store.cc:276] 0709 12:18:14.934421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.976056 (+1041635us) service_pool.cc:224] Handling call
0709 12:18:15.976424 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:15.986704 (+ 10280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.987031 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:16.007450 31694 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:16.008214 31694 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.010722 28433 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:15.985833 31760 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:16.020285 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.020231 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.023023 28286 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:16.000247 31538 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:15.995294 31520 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:16.034298 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.003078 31684 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.034096 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:16.041568 28332 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:15.963451 31513 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.042445 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:16.042711 28285 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:16.014662 31527 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:16.047638 28292 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:16.057441 28288 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:16.048002 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:16.055269 31704 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:16.050578 28439 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
W20260709 12:18:16.062438 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 117225us
I20260709 12:18:16.043509 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:16.066774 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.032435 31587 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:16.040539 31538 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.038081 31520 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.073403 31307 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:16.074628 31759 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:18:16.044828 31440 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:16.076104 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1866) took 1585 ms. Trace:
I20260709 12:18:16.015686 31693 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:18:16.078073 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:16.084964 28359 rpcz_store.cc:276] 0709 12:18:14.491185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.325986 (+834801us) service_pool.cc:224] Handling call
0709 12:18:15.326423 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:15.326972 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:15.327003 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:15.339392 (+ 12389us) log.cc:844] Serialized 26 byte log entry
0709 12:18:15.763123 (+423731us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:15.763167 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.763246 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:16.075790 (+312544us) raft_consensus.cc:1685] finished
0709 12:18:16.075814 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.076027 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":871,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":2524,"thread_start_us":375439,"threads_started":1,"wal-append.queue_time_us":544200}
I20260709 12:18:16.061856 31706 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:16.065340 28330 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.579s (SENT) [suppressed 5 similar messages]
I20260709 12:18:16.068876 31696 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.063998 31652 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.060657 31704 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:16.105470 28285 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:16.071650 31538 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.043351 31476 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:18:16.015977 31522 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.080569 31307 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:16.113175 28359 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:16.100349 31706 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.106775 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.079049 31767 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.079748 31440 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.127669 31440 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.107575 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:16.087299 31477 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:18:16.130709 31761 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:16.058739 31684 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.071775 31520 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.133944 28359 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:16.134277 31617 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:16.131274 31380 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.134832 31617 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:16.136358 31758 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:18:16.139513 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.142485 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:16.143504 28284 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:16.142977 31760 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.144728 31589 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:16.109985 31612 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:16.147969 28361 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1858) took 1824 ms (client timeout 1946 ms). Trace:
W20260709 12:18:16.102283 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66511.4us
I20260709 12:18:16.149055 31612 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:18:16.096731 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 321921us
W20260709 12:18:16.130224 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44816.7us
I20260709 12:18:16.089524 31527 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.149905 31754 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:16.120507 31767 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.150380 31754 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.150868 31754 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.125849 31704 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:16.114766 31689 consensus_meta.cc:287] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.001s
I20260709 12:18:16.084374 31694 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:16.152253 28956 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:16.153148 31307 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.131315 31761 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:16.153795 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:16.154605 28982 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:16.154989 31761 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.152244 31704 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.108709 31652 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.130074 31587 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.159682 31689 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:16.152446 31706 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.163091 31597 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:16.163527 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:16.148438 28361 rpcz_store.cc:269] 0709 12:18:14.323556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.326867 (+  3311us) service_pool.cc:224] Handling call
0709 12:18:16.147767 (+1820900us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001d7c54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:16.147901 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2503460736}
I20260709 12:18:16.167038 28361 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:16.148707 28358 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:16.151088 31586 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:16.107287 31286 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.163094 28359 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:16.122001 31476 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:16.167202 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:16.122826 31513 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:16.172617 28367 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:16.175522 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:16.178632 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 119345us
I20260709 12:18:16.149636 31612 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.179744 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:16.181097 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:16.182476 28332 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:16.218135 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.218865 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 68310.8us
I20260709 12:18:16.221626 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:16.225512 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1870) took 1409 ms. Trace:
I20260709 12:18:16.171856 31476 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:16.253456 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:16.229887 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.226049 28360 rpcz_store.cc:276] 0709 12:18:14.816416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.216807 (+400391us) service_pool.cc:224] Handling call
0709 12:18:16.225325 (+1008518us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b5400126954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:16.225443 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1268963072}
W20260709 12:18:16.274662 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.276616 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:16.277356 28364 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:16.275660 31683 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:16.292349 31683 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.294232 31617 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:16.278214 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:16.278100 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:16.298151 28361 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:16.218739 31597 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:16.299130 31597 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.299537 31597 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.290241 31689 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:16.308512 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 141392us
I20260709 12:18:16.313714 28956 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:16.342357 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 62437.4us
W20260709 12:18:16.347559 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 282218us
W20260709 12:18:16.349129 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:16.349599 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.353820 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.354779 28292 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:16.355047 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:16.355805 28367 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.178129 31527 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.230729 31684 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:16.361122 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:16.176173 31286 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.388001 28433 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:16.386919 28285 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:16.398447 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 223033us
I20260709 12:18:16.403035 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:16.403757 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 53579.3us
I20260709 12:18:16.354049 31484 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:18:16.392720 31588 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:16.376430 31771 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:16.416253 28292 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:16.417652 31588 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.400009 31652 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:16.430675 28367 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:16.406047 31683 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.436362 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.437170 28285 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:16.437633 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:16.375370 31772 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:18:16.400622 31617 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:18:16.397579 31586 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:18:16.485810 31769 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.428233 31589 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:16.420182 31773 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:16.490259 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 58016.1us
W20260709 12:18:16.437001 31515 consensus_meta.cc:287] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.800s	user 0.012s	sys 0.001s
I20260709 12:18:16.531162 28956 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:16.528862 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 108917us
I20260709 12:18:16.435460 31588 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.552556 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:16.473182 31706 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:16.534884 31639 consensus_meta.cc:287] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.642s	user 0.012s	sys 0.001s
I20260709 12:18:16.562551 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:16.567634 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 347908us
I20260709 12:18:16.571504 31771 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:16.573623 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:16.539741 31587 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:16.588310 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:16.545979 31760 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:16.589931 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.594497 28292 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:16.592502 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:16.546527 31513 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:16.594594 28284 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:16.595240 28287 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.439898 31782 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.153s
I20260709 12:18:16.598948 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:16.598995 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.601933 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:16.441085 31689 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.542529 31515 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.603103 28291 raft_consensus.cc:2410] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:18:16.604471 28432 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:16.607627 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1873) took 1612 ms. Trace:
I20260709 12:18:16.532902 31683 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.608433 28358 rpcz_store.cc:276] 0709 12:18:14.994863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.006209 (+1011346us) service_pool.cc:224] Handling call
0709 12:18:16.022495 (+ 16286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.167688 (+145193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.607346 (+439658us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400125a54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:16.607550 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289770880}
I20260709 12:18:16.529294 31476 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.547596 31589 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.619935 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.574026 31639 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.593179 31684 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:16.623615 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 228447us
I20260709 12:18:16.623181 28284 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:16.604681 31477 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.624969 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.626255 28366 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:16.561699 31770 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:18:16.587301 31706 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:18:16.602653 31786 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:16.607234 31772 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:16.607060 31760 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:16.611940 31587 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:16.638002 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.638434 28987 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
W20260709 12:18:16.628613 31768 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.377s	user 0.000s	sys 0.001s
I20260709 12:18:16.604326 31513 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.639348 31586 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:16.635344 31477 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:16.640071 31780 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:18:16.640499 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.659019 28292 raft_consensus.cc:2410] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:16.642776 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:18:16.643141 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.636647 31786 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:16.645912 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.637286 31476 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:16.649097 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 96618.8us
I20260709 12:18:16.657287 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.668668 28286 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:16.644114 28365 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 12. Ops: []
I20260709 12:18:16.629259 31781 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.640115 31733 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:16.665463 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:16.670470 28358 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:16.665853 31484 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:16.667100 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:16.644985 31527 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:16.661863 31666 consensus_meta.cc:287] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.789s	user 0.014s	sys 0.000s
I20260709 12:18:16.681991 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:16.662617 31286 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:16.669380 31786 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:16.696394 31440 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.568s	user 0.010s	sys 0.000s
I20260709 12:18:16.671666 31733 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:16.675354 31484 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:16.680302 31617 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:16.687070 31797 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.684188 31666 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:16.705935 31794 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.706449 31783 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:18:16.707258 31589 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:16.691807 31774 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:16.707641 31587 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.653594 31771 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.731721 31791 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.732314 31791 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:16.707126 31790 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:18:16.714861 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 47798.6us
I20260709 12:18:16.654371 31772 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.728609 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1875) took 1733 ms. Trace:
I20260709 12:18:16.690106 31527 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:18:16.734705 28283 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:16.692315 31652 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:18:16.733795 31788 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:18:16.739506 31799 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.797894 31799 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.740195 31767 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:16.742460 31683 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:16.800623 31597 consensus_meta.cc:287] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
W20260709 12:18:16.743134 31754 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.000s
I20260709 12:18:16.801370 31754 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.801797 31798 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:16.802366 31798 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.802723 31798 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.744030 31634 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:16.803809 31634 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:16.801121 31683 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:16.805007 31683 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:16.736604 31589 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:18:16.805927 31683 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.806219 31733 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.807201 31683 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:16.809060 31538 consensus_meta.cc:287] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.000s
I20260709 12:18:16.723152 31706 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:16.809567 31538 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.810060 31706 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.751636 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 380) took 1826 ms. Trace:
I20260709 12:18:16.811110 28365 rpcz_store.cc:276] 0709 12:18:14.925463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.949141 (+1023678us) service_pool.cc:224] Handling call
0709 12:18:16.157550 (+208409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.673893 (+516343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.750748 (+ 76855us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b5400127354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:16.751417 (+   669us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b5400127350. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:16.751565 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1513769984}
I20260709 12:18:16.811832 31538 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:16.812196 31706 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:16.737457 31774 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:16.706158 31760 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.813555 31774 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.814711 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:16.815356 28987 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:16.815760 31580 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.901s	user 0.010s	sys 0.001s
I20260709 12:18:16.753269 28255 leader_election.cc:400] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7.
I20260709 12:18:16.792063 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:16.749594 31696 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.643s	user 0.005s	sys 0.003s
I20260709 12:18:16.795706 28285 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:16.709542 31440 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.726053 31784 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.820775 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:16.729360 31586 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 12:18:16.781893 31693 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.801101 31597 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.825404 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:16.911528 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1872) took 2385 ms. Trace:
I20260709 12:18:16.826165 28287 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
W20260709 12:18:16.912235 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 86949.1us
I20260709 12:18:16.759610 28362 rpcz_store.cc:276] 0709 12:18:14.995466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.061236 (+1065770us) service_pool.cc:224] Handling call
0709 12:18:16.071610 (+ 10374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.118561 (+ 46951us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:16.118602 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:16.177275 (+ 58673us) log.cc:844] Serialized 26 byte log entry
0709 12:18:16.365972 (+188697us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:16.366024 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.366112 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:16.728382 (+362270us) raft_consensus.cc:1685] finished
0709 12:18:16.728404 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:16.728542 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48247,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":180722,"threads_started":2,"wal-append.queue_time_us":174093}
I20260709 12:18:16.837179 28987 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:16.912050 28366 rpcz_store.cc:276] 0709 12:18:14.536313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.601893 (+1065580us) service_pool.cc:224] Handling call
0709 12:18:15.602464 (+   571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.630611 (+1028147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.890176 (+259565us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400134f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:16.905328 (+ 15152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2354056064}
I20260709 12:18:16.792793 31704 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.818578 31520 consensus_meta.cc:287] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.000s
I20260709 12:18:16.807422 31733 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:16.726204 31781 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.172s
W20260709 12:18:16.909426 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.816275 31580 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:16.895539 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 221945us
W20260709 12:18:16.916850 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.920050 28286 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:16.822113 31754 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:16.823534 31696 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:16.796291 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.800670 31799 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:16.889047 31286 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.721269 31797 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.907085 31767 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.727583 31794 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:16.936808 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 247190us
W20260709 12:18:16.938809 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.907447 31515 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:16.933683 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:16.941893 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:16.913239 31782 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.917150 31617 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:16.920720 31804 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:16.990756 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 237647us
W20260709 12:18:16.991255 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.920416 31706 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.929314 31131 consensus_meta.cc:287] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.163s	user 0.015s	sys 0.000s
W20260709 12:18:16.999627 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.831652 31776 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
W20260709 12:18:17.006672 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 64921.8us
W20260709 12:18:17.011772 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.937938 31639 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:17.020098 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:16.925973 31704 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:16.924041 31520 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:17.008117 31715 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:17.013499 31754 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.129208 28360 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:17.129580 28358 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:16.990960 31286 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:17.014775 31617 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:16.939240 31693 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:17.086777 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 77254.6us
I20260709 12:18:17.017612 31522 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.106422 28365 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:17.020546 31785 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.018092 31513 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:17.128180 28359 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:16.907362 31784 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:17.074460 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:17.105819 31801 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:17.003407 31767 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:17.158010 28362 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:17.140378 31805 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 12:18:17.140908 28363 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:17.142375 31794 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:17.025717 31131 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:17.143672 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:17.145450 31715 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:17.018733 31652 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 12:18:17.022003 31706 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:16.988035 31484 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:17.106261 31765 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.045133 31527 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.106s
W20260709 12:18:17.138368 31612 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.914s	user 0.018s	sys 0.001s
I20260709 12:18:17.156788 31762 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:17.161207 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 171678us
I20260709 12:18:17.160121 31801 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:17.162744 28437 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:17.161859 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 140894us
I20260709 12:18:17.169055 28361 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:17.144479 31806 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:17.018962 31782 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:17.197700 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:17.206264 31811 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:18:17.210100 31694 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:17.211678 31617 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:17.150000 31522 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:17.216840 31769 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.672s	user 0.010s	sys 0.000s
I20260709 12:18:17.173228 31380 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:17.218485 31793 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:18:17.218179 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:17.148756 31585 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:17.155068 31307 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.220206 28434 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:17.220515 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:17.226918 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:17.209501 31588 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.636s	user 0.006s	sys 0.002s
I20260709 12:18:17.137280 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:17.230646 28360 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:17.233448 28361 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:18:17.171182 31477 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:17.228873 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:17.157141 31785 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:17.811879 28364 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:17.813649 28359 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:17.560298 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:17.819693 28367 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:17.293869 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.669477 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:17.486172 28435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.020s	sys 0.011s
I20260709 12:18:17.824173 28362 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:17.384850 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:17.899921 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:17.968855 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 831660us
W20260709 12:18:17.823200 28435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.023s	sys 0.018s
I20260709 12:18:17.899921 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:17.828313 28436 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:17.969338 28439 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:17.811740 28361 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:17.707513 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1889) took 1601 ms. Trace:
I20260709 12:18:17.823036 28437 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:17.970651 28365 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:17.972792 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 740019us
I20260709 12:18:17.561020 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:17.979197 28366 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:17.980274 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 595522us
I20260709 12:18:17.824712 28363 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
W20260709 12:18:17.817648 31683 consensus_meta.cc:287] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.010s	user 0.012s	sys 0.001s
I20260709 12:18:17.978127 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:17.996906 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:17.997629 28987 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:17.997913 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:17.999142 31781 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:17.999545 31781 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.999888 31781 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.999899 31791 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:18.002470 31791 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.003046 31807 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:18.003656 31807 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:18.003005 31791 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.004604 31807 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:17.954542 31733 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.027s	user 0.008s	sys 0.003s
W20260709 12:18:17.492542 31798 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.004s
I20260709 12:18:18.018577 31798 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:17.872851 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.974283 28288 rpcz_store.cc:276] 0709 12:18:16.105720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.105938 (+   218us) service_pool.cc:224] Handling call
0709 12:18:16.114915 (+  8977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.118804 (+  3889us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:16.276339 (+157535us) log.cc:844] Serialized 14 byte log entry
0709 12:18:17.382859 (+1106520us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b180032f2c0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb7682a6 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1
0709 12:18:17.383550 (+   691us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b180032f2c0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:17.706971 (+323421us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b180032f2c0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287
0709 12:18:17.707101 (+   130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:17.707161 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:17.707215 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.707294 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.707444 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3034,"prepare.queue_time_us":269664,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":37776,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":35676,"spinlock_wait_cycles":93929856,"thread_start_us":547693,"threads_started":3,"wal-append.queue_time_us":448006}
I20260709 12:18:18.002354 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:18.021193 31813 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.507s	user 0.009s	sys 0.000s
I20260709 12:18:18.006992 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:18.001641 31784 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:17.358927 31786 consensus_meta.cc:287] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.000s
I20260709 12:18:17.824506 31286 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:17.594852 31715 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:17.488946 31760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.001s	sys 0.022s
I20260709 12:18:17.947688 31823 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:17.699941 31812 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.354s	user 0.000s	sys 0.000s
I20260709 12:18:17.547973 31815 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.536s
W20260709 12:18:17.948271 31476 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.272s	user 0.007s	sys 0.004s
W20260709 12:18:17.700695 31800 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.317s	user 0.000s	sys 0.000s
W20260709 12:18:17.308537 31787 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 12:18:17.895797 31585 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:17.939356 31477 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.084266 31585 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.084475 31477 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.086951 31477 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:17.824514 31588 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:17.347051 31597 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:17.310205 31634 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:17.345855 28982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 443) took 1136 ms. Trace:
I20260709 12:18:18.017118 28290 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:17.834470 31484 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.387526 31778 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:17.387020 31788 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:18.113667 28433 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:17.269899 31666 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.748s
I20260709 12:18:17.293882 31589 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.724s
I20260709 12:18:17.947958 31795 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.022261 31527 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:18.002640 31586 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:18.021751 28438 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.027365 28985 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
W20260709 12:18:17.958822 31754 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.816s	user 0.003s	sys 0.007s
W20260709 12:18:17.782600 31773 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 1.235s	user 0.012s	sys 0.001s
I20260709 12:18:17.257313 31440 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:17.822407 31520 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:17.243683 31789 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.560370 31814 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.963680 31767 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:17.669591 31774 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.559809 31693 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:17.966140 31580 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:18.116725 31597 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.115247 31791 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.024705 31380 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.046568 31821 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:18.046967 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:17.981690 31587 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:17.972237 31694 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:18.051761 28358 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.015s	sys 0.011s
I20260709 12:18:18.013011 28987 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:18.174037 28438 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:17.595748 31797 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.183344 28989 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:18.055914 28439 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:17.245082 31612 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:17.558997 31772 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:17.980829 28441 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:18.014701 31769 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:18.055289 28257 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.008s	sys 0.013s
I20260709 12:18:18.032088 31131 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:18.054517 31760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.235s	user 0.011s	sys 0.045s
I20260709 12:18:18.055620 31617 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:18.087455 28283 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 448) took 1441 ms (client timeout 1800 ms). Trace:
I20260709 12:18:17.920869 31307 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.089830 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2139) took 1093 ms. Trace:
I20260709 12:18:18.173112 31440 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.294871 31806 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.079738 31798 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:18.224609 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.076560 31704 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:18.111456 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 104741us
I20260709 12:18:18.157361 31789 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.096446 31477 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.070324 31786 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.122581 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.171247 31380 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.233974 28366 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:18.108836 31778 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:18.225878 31440 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.236876 31778 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.238215 31778 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:18.238657 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:18.113116 31634 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:18.132212 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:18.135310 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 363506us
I20260709 12:18:18.246205 31832 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:18.246989 31832 consensus_queue.cc:1243] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:18:18.005992 31761 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:18.247967 31761 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:18.248924 31761 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.249933 31760 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:18.044142 31784 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:18.154908 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 444) took 2147 ms. Trace:
I20260709 12:18:18.126147 31476 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.113116 31788 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:18.159926 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:18.164166 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:18.166656 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.128051 31586 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:18.169623 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 153942us
I20260709 12:18:18.170783 28440 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:18.168033 31819 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:18:18.104691 28982 rpcz_store.cc:276] 0709 12:18:16.158723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.158893 (+   170us) service_pool.cc:224] Handling call
0709 12:18:16.159290 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.159711 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.159743 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:16.160239 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 12:18:16.290926 (+130687us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:16.290971 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.291063 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:17.146452 (+855389us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b1800188aa0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:17.219612 (+ 73160us) raft_consensus.cc:1685] finished
0709 12:18:17.226649 (+  7037us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.295082 (+ 68433us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":187323,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":30260,"spinlock_wait_cycles":12174208,"thread_start_us":115664,"threads_started":1,"wal-append.queue_time_us":230872}
I20260709 12:18:18.245777 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.338529 28367 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:18.153136 31773 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:18.175918 28358 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.015s	sys 0.011s
I20260709 12:18:18.342286 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:18.073539 31823 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.343628 28288 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:18.345264 31823 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.151080 31795 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.176999 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:18.346155 28365 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:18.156504 31527 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:18.346323 31789 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.346612 31527 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:18.327497 31694 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.182417 28990 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:18.327373 31786 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.348136 31823 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.007589 31652 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.186684 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:18.191565 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:18.192227 31822 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.325s	user 0.002s	sys 0.000s
W20260709 12:18:18.192848 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.349607 28436 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
W20260709 12:18:18.350229 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.349097 28438 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:18.351928 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.193588 31831 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.194157 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:18.194883 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.194172 31612 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.197348 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 449) took 1551 ms. Trace:
W20260709 12:18:18.198328 28257 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.008s	sys 0.013s
I20260709 12:18:18.191051 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:18.072438 31689 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:18:18.215646 28283 rpcz_store.cc:269] 0709 12:18:16.645782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.650516 (+  4734us) service_pool.cc:224] Handling call
0709 12:18:18.087267 (+1436751us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b54000affd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:18.087388 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117792640}
I20260709 12:18:18.173856 31587 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:18.216707 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 496) took 1394 ms. Trace:
I20260709 12:18:18.218887 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.217092 28434 rpcz_store.cc:276] 0709 12:18:16.996425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.996974 (+   549us) service_pool.cc:224] Handling call
0709 12:18:17.019090 (+ 22116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:17.811410 (+792320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.089520 (+278110us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400346f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:18.089750 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461682048}
I20260709 12:18:17.295840 31765 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:17.271564 31762 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.192143 31706 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.187686 31828 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:18.160338 31597 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.227411 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:18.109443 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2137) took 1457 ms. Trace:
I20260709 12:18:18.230054 28988 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:17.501050 31801 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.166306 31774 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:18.234602 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:18.251083 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.225302 31522 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:18.334233 28289 rpcz_store.cc:276] 0709 12:18:16.003359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.004202 (+   843us) service_pool.cc:224] Handling call
0709 12:18:16.029872 (+ 25670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.061675 (+ 31803us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.061710 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:16.076961 (+ 15251us) log.cc:844] Serialized 26 byte log entry
0709 12:18:16.793119 (+716158us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b1800450880. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:18:17.043548 (+250429us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:17.043589 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.043671 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.129025 (+1085354us) raft_consensus.cc:1685] finished
0709 12:18:18.129051 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.151112 (+ 22061us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18506,"raft.queue_time_us":33555,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":10394,"spinlock_wait_cycles":7534336,"thread_start_us":250238,"threads_started":1,"wal-append.queue_time_us":1085114}
I20260709 12:18:18.325594 31477 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.185297 31814 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.337175 31785 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:18.238152 31440 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.130328 31754 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:18.339272 31131 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.329625 28983 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:18.342267 31806 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:18.347357 31795 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:18.190946 31820 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:18:18.361228 28439 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:18.361264 28366 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:18:18.482419 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.871589 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.381623 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:18.872792 28285 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:18.873328 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:18.873898 28255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.010s	sys 0.008s
I20260709 12:18:18.874235 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2138) took 2180 ms. Trace:
I20260709 12:18:18.770058 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1891) took 1565 ms. Trace:
I20260709 12:18:18.874769 28435 rpcz_store.cc:276] 0709 12:18:16.693454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.693612 (+   158us) service_pool.cc:224] Handling call
0709 12:18:16.694697 (+  1085us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.700391 (+  5694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.700426 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:16.714405 (+ 13979us) log.cc:844] Serialized 26 byte log entry
0709 12:18:17.978156 (+1263751us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:17.978199 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.978276 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.873993 (+895717us) raft_consensus.cc:1685] finished
0709 12:18:18.874016 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.874145 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5285,"raft.queue_time_us":63394,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":19204,"thread_start_us":1259425,"threads_started":1,"wal-append.queue_time_us":1364151}
I20260709 12:18:18.875638 28360 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:18.456135 28365 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:18.878665 31791 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:18.301750 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.879283 31791 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:18.880292 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:18.880807 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.874081 28366 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:18:18.881693 31791 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.881805 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.348727 31823 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.883153 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.883801 28366 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:18.884676 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:18.884934 31823 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:18.496408 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.887238 31791 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.428314 31784 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:18.331230 31380 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.888170 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:18.890544 28286 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:18:18.871855 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.879396 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.895327 28365 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:18.897063 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.771809 31788 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.877822 31586 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:18.885448 28366 raft_consensus.cc:2393] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:18.900449 31477 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:18.901283 31762 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:18.900363 28366 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:18.903721 28366 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:18.915108 31788 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:18.361552 31832 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:18.919231 31832 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.555022 31782 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.249607 31484 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.676s
I20260709 12:18:18.426803 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.506673 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:18.250813 31829 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:18.310523 31816 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:18.326859 31760 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:18.459599 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:18.422222 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:18.515188 28985 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:18.422845 28288 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:18.868978 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.870565 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:18.873449 31828 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.870244 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:18.901161 31587 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.871127 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:18.340379 31307 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.890043 28989 raft_consensus.cc:2368] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
I20260709 12:18:18.924105 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2145) took 1028 ms. Trace:
I20260709 12:18:18.877955 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:18.984514 28360 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:18.874316 28255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.010s	sys 0.009s
W20260709 12:18:18.987686 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 740006us
I20260709 12:18:18.889707 28287 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:18.888000 31791 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:18.897519 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.982864 28439 rpcz_store.cc:276] 0709 12:18:17.942166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.193822 (+251656us) service_pool.cc:224] Handling call
0709 12:18:18.924015 (+730193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:18.988286 28367 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:18.989930 28287 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:18.354110 28404 connection.cc:256] RPC call timeout handler was delayed by -1.9962731660002646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
W20260709 12:18:18.993008 28404 connection.cc:256] RPC call timeout handler was delayed by -1.8540731660002621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19630000000000003s for a response
W20260709 12:18:18.993392 28404 connection.cc:256] RPC call timeout handler was delayed by -1.8369731660002344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
I20260709 12:18:18.922418 28433 rpcz_store.cc:276] 0709 12:18:16.820248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.820444 (+   196us) service_pool.cc:224] Handling call
0709 12:18:18.214821 (+1394377us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540026d4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:18.214985 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2818363776}
W20260709 12:18:18.992694 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.991004 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:18.991868 28441 raft_consensus.cc:2393] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:18.927374 28984 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:18.927018 28364 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:18:18.995599 28404 connection.cc:256] RPC call timeout handler was delayed by -1.7280731660000583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
W20260709 12:18:18.999030 28404 connection.cc:256] RPC call timeout handler was delayed by -1.691173166000226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 12:18:18.999420 28404 connection.cc:256] RPC call timeout handler was delayed by -1.6731731660001969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
I20260709 12:18:18.380400 31652 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.980183 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:18.930397 28363 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:19.002151 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2147) took 1059 ms. Trace:
I20260709 12:18:18.870831 28290 rpcz_store.cc:276] 0709 12:18:16.646072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.646195 (+   123us) service_pool.cc:224] Handling call
0709 12:18:16.657296 (+ 11101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.104862 (+1447566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.197063 (+ 92201us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54003daed4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:18.197268 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3126532224}
W20260709 12:18:18.995826 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 62077.1us
I20260709 12:18:18.869457 28432 rpcz_store.cc:276] 0709 12:18:16.652496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.652661 (+   165us) service_pool.cc:224] Handling call
0709 12:18:16.661600 (+  8939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:16.679418 (+ 17818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:16.679455 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:16.693529 (+ 14074us) log.cc:844] Serialized 26 byte log entry
0709 12:18:16.926718 (+233189us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:16.926765 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:16.926843 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.109182 (+1182339us) raft_consensus.cc:1685] finished
0709 12:18:18.109206 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.109372 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13397,"raft.queue_time_us":13608,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":13699,"thread_start_us":186923,"threads_started":1,"wal-append.queue_time_us":307130}
W20260709 12:18:18.930662 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 471165us
I20260709 12:18:18.888860 31784 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.941082 28358 rpcz_store.cc:276] 0709 12:18:17.204456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.219530 (+ 15074us) service_pool.cc:224] Handling call
0709 12:18:17.242870 (+ 23340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:17.243104 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:18.343194 (+1100090us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.769547 (+426353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:18.769623 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:18.769683 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.769794 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.769968 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":871764,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":126,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":14494,"thread_start_us":1473393,"threads_started":2}
W20260709 12:18:18.931334 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 509180us
I20260709 12:18:19.010998 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:18.998198 28286 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:19.013934 31782 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.014343 31782 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.014689 31782 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.015543 31761 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.000524 28364 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:18:19.017432 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.018607 28364 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:19.019939 28364 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:18.869843 31778 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.631s	user 0.008s	sys 0.003s
I20260709 12:18:19.002553 28441 rpcz_store.cc:276] 0709 12:18:17.942800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.190735 (+247935us) service_pool.cc:224] Handling call
0709 12:18:19.002081 (+811346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:19.004135 28987 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
W20260709 12:18:18.862668 31781 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.862s	user 0.012s	sys 0.000s
I20260709 12:18:18.506618 31704 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:18:18.869072 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 1.63732e+06us
I20260709 12:18:18.935413 28438 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:18:19.037114 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2142) took 1142 ms. Trace:
I20260709 12:18:18.941006 28436 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:19.022689 28287 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.927771 28292 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:19.022630 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:19.037971 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2143) took 1143 ms. Trace:
I20260709 12:18:19.005661 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:18.958540 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.038437 28432 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.037501 28438 rpcz_store.cc:276] 0709 12:18:17.894561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.382545 (+487984us) service_pool.cc:224] Handling call
0709 12:18:19.037043 (+654498us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:18.930573 31477 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:18.929783 31760 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:19.005658 28360 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:18.934019 28362 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:18.506213 31825 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.035607 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.037201 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:18.934689 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.037317 28289 pending_rounds.cc:85] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 7
I20260709 12:18:18.887621 31774 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:18.921674 31832 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.931521 31754 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:18.830295 31687 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:19.041358 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.982266 31689 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:18.308743 31821 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.737s
I20260709 12:18:19.153376 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.042484 28287 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:19.157119 31840 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.157718 31840 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.993948 31785 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:18.940066 28440 consensus_meta.cc:287] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.000s
I20260709 12:18:19.036399 31762 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:18.928313 31829 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:18.939392 31440 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.121449 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:18.938644 31828 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:18.943096 31694 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.125056 28289 pending_rounds.cc:107] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:18:19.187655 28289 raft_consensus.cc:2889] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:19.125903 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 100317us
I20260709 12:18:18.388047 31476 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.124985 28283 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:18:18.931891 31597 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:19.131568 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:19.130427 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 93318.6us
I20260709 12:18:18.929416 31816 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:19.004910 31807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.023s	sys 0.071s
I20260709 12:18:19.183300 31440 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:18.414950 31831 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.203409 28365 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:19.203868 31440 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:19.204525 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:19.205225 28434 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:18.926523 31131 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.214090 31131 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:19.040067 31781 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.218573 31781 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:19.219264 31781 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.219775 31781 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.220439 31652 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:19.220836 31617 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.003s	user 0.008s	sys 0.000s
I20260709 12:18:19.152318 31687 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.221827 31307 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.222442 31307 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.223682 31687 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:19.153548 31821 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.224500 31821 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.224917 31821 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.227964 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56626 (request call id 394) took 1038 ms. Trace:
I20260709 12:18:19.228399 28362 rpcz_store.cc:276] 0709 12:18:18.189265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.189424 (+   159us) service_pool.cc:224] Handling call
0709 12:18:19.227740 (+1038316us) spinlock_profiling.cc:243] Waited 60.2 ms on lock 0x7b54001d8654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:19.227892 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138451328}
I20260709 12:18:19.038372 28436 rpcz_store.cc:276] 0709 12:18:17.894802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.381130 (+486328us) service_pool.cc:224] Handling call
0709 12:18:19.037903 (+656773us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:19.230083 31588 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:19.230011 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.230857 28436 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:19.231783 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.232239 31801 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:19.232879 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:19.235504 31829 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.237098 31829 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.043809 31704 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.239522 31307 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:19.242789 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:19.205813 31760 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.244271 31760 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.134421 31806 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:18:19.022835 31612 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 12:18:19.169798 28440 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
W20260709 12:18:19.259083 28440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 502) took 1407 ms (client timeout 1819 ms). Trace:
I20260709 12:18:19.171911 28287 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:18:19.259723 28287 pending_rounds.cc:85] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 7
I20260709 12:18:19.259984 28287 pending_rounds.cc:107] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:18:19.260223 28287 raft_consensus.cc:2889] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:19.128736 31380 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:19.259518 28440 rpcz_store.cc:269] 0709 12:18:17.851774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.851943 (+   169us) service_pool.cc:224] Handling call
0709 12:18:18.046857 (+194914us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b58001d4698. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762f8c 00007f85bb762ef6 000000000036eb39 00007f85c37d61e0 00007f85c389f0c4 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:18:19.259012 (+1212155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111157888}
W20260709 12:18:19.174266 28330 connection.cc:256] RPC call timeout handler was delayed by -1.06351447499992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
I20260709 12:18:19.130729 31754 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.135892 31824 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:19.134871 31774 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.279089 31824 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:19.283304 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:19.283996 28989 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:19.282696 31774 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.285198 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:19.190415 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:19.192113 28404 connection.cc:256] RPC call timeout handler was delayed by -1.4743516819999058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 12:18:19.283943 31824 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.289757 31824 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.347653 31527 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.199098 28439 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:19.191916 28435 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:19.174916 31762 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.303674 31762 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.285902 28984 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:18:19.200424 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:19.210166 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.133440 31825 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:19.201937 31807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.023s	sys 0.071s
W20260709 12:18:19.251693 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.251055 31706 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.259181 28361 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:19.185096 31785 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.246439 31597 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.239807 31704 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:19.295190 28434 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
W20260709 12:18:19.303120 31837 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 12:18:19.210516 31839 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:18:19.307379 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 107081us
I20260709 12:18:19.307364 28441 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:19.282781 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:19.311257 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.926123 31814 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:19.314779 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.334817 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:19.314862 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 309289us
W20260709 12:18:19.267932 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.341049 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:19.279584 31754 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:19.341229 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.340229 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:19.322975 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2141) took 1428 ms. Trace:
I20260709 12:18:19.312829 31586 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:19.311709 31477 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:19.347218 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.292093 31380 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:19.352119 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:19.365906 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.334353 31652 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.345106 28437 rpcz_store.cc:276] 0709 12:18:17.894222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.421977 (+527755us) service_pool.cc:224] Handling call
0709 12:18:18.422381 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:18.422788 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.422819 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.423332 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.577687 (+154355us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:18.577763 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.577856 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.322754 (+744898us) raft_consensus.cc:1685] finished
0709 12:18:19.322776 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.322908 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":153862,"threads_started":1,"wal-append.queue_time_us":767066}
W20260709 12:18:19.366745 28359 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.008s	sys 0.008s
W20260709 12:18:19.367144 28359 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.008s	sys 0.008s
I20260709 12:18:19.343480 31666 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:19.375480 28365 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:19.377193 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:19.377919 31798 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:19.372047 31440 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.378583 31774 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:19.419905 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.364630 31307 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:18:19.331014 31706 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.379902 31765 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:19.382205 28332 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:19.386672 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.387642 28286 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:19.387697 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.388232 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:19.349997 31586 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:19.405860 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.417644 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 391) took 1292 ms. Trace:
W20260709 12:18:19.421303 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 407994us
I20260709 12:18:19.444501 28361 rpcz_store.cc:276] 0709 12:18:18.125148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.125311 (+   163us) service_pool.cc:224] Handling call
0709 12:18:18.125645 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.314091 (+1188446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.417376 (+103285us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400078d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:19.417568 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2605828480}
I20260709 12:18:19.445298 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:19.350862 31814 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:19.337548 31597 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.342538 31484 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:18:19.366546 31704 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:19.438138 28291 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:19.440549 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.454448 28284 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:19.441215 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.446072 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.456300 28283 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:19.382555 31791 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:19.442982 28287 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 5.
I20260709 12:18:19.445986 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.449819 31832 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:18:19.450438 31842 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:18:19.450095 31380 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.451817 31477 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.446283 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.455681 28290 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:19.419160 31440 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:19.477869 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.478227 28437 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:19.396615 31666 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:19.469935 31791 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:19.478008 31440 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.480280 31666 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.482110 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.482797 28283 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:19.483757 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.484371 28283 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:19.479489 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.459465 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1914) took 1035 ms. Trace:
I20260709 12:18:19.462218 28292 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:19.484908 31840 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:19.485275 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.459978 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:19.462810 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.486986 28291 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:19.486023 28364 rpcz_store.cc:276] 0709 12:18:18.423719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.020661 (+596942us) service_pool.cc:224] Handling call
0709 12:18:19.020975 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.021353 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.021394 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.155810 (+134416us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b2c0000e678. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:18:19.156275 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.352442 (+196167us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:19.352485 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.352898 (+   413us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.453099 (+100201us) raft_consensus.cc:1685] finished
0709 12:18:19.453120 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.459388 (+  6268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":22475008,"thread_start_us":195753,"threads_started":1,"wal-append.queue_time_us":294320}
I20260709 12:18:19.488308 28291 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:19.465931 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.485703 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:19.490836 31816 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:19.491323 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.467231 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.452903 31814 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.492769 31814 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:19.495321 31825 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:19.446961 31586 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:19.490707 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.474411 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.474581 31689 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:19.474890 31838 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:18:19.475965 31476 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:19.480130 31791 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.473654 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:19.500308 28404 connection.cc:256] RPC call timeout handler was delayed by -1.2628516820000186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
I20260709 12:18:19.517849 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:19.374522 31652 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.457119 31778 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.529075 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:19.513546 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 204819us
I20260709 12:18:19.512116 28288 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:19.531584 28437 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:19.469548 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 87406.6us
W20260709 12:18:19.518290 28404 connection.cc:256] RPC call timeout handler was delayed by -1.2304516819999662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
I20260709 12:18:19.472508 31781 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:19.485812 31831 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:19.525426 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.524904 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51319.9us
I20260709 12:18:19.526481 28290 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:19.525002 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:19.528354 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.455907 31704 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.547719 28437 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:19.534767 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1701516820000961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
W20260709 12:18:19.548653 28404 connection.cc:256] RPC call timeout handler was delayed by -1.0556985990001522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
I20260709 12:18:19.549449 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:19.520249 31791 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:19.528475 31689 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:19.541054 28432 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
W20260709 12:18:19.544493 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 95706.3us
I20260709 12:18:19.545097 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:19.543975 28956 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:19.553241 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.533022 31778 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.562275 28283 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:19.564222 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1916) took 1140 ms. Trace:
I20260709 12:18:19.559785 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:19.567035 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:19.568817 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:19.568935 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:19.568413 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.564599 28358 rpcz_store.cc:276] 0709 12:18:18.424280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.040242 (+615962us) service_pool.cc:224] Handling call
0709 12:18:19.127063 (+ 86821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.240752 (+113689us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:19.496892 (+256140us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.540818 (+ 43926us) log.cc:844] Serialized 14 byte log entry
0709 12:18:19.541013 (+   195us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:19.541058 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:19.541111 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.563978 (+ 22867us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540027e554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:19.564026 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.564156 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11592,"raft.queue_time_us":17072,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5158,"spinlock_wait_cycles":260806528,"thread_start_us":287548,"threads_started":2}
W20260709 12:18:19.571343 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.579524 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.570358 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1904) took 1217 ms. Trace:
W20260709 12:18:19.569927 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.575140 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.583896 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.584065 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:19.575191 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:19.591821 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:19.588778 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:19.592679 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:19.593282 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:19.593488 31476 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:19.593941 31476 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.594123 28984 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 7: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:19.594411 31848 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:19.594792 31848 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.595173 31848 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.595464 31789 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.595973 31789 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.596000 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:19.598096 28984 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.598176 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:19.599270 31617 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.599689 31589 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.600109 31476 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:19.600245 31589 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.593282 28360 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:19.602492 31589 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:19.598191 31789 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:19.589308 28432 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:19.604516 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:19.583292 28366 rpcz_store.cc:276] 0709 12:18:18.353340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.904553 (+551213us) service_pool.cc:224] Handling call
0709 12:18:18.904861 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:18.905254 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.905284 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.905828 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.176315 (+270487us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:19.176376 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.176473 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.570128 (+393655us) raft_consensus.cc:1685] finished
0709 12:18:19.570154 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.570293 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1223,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"thread_start_us":270026,"threads_started":1,"wal-append.queue_time_us":401001}
I20260709 12:18:19.605072 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:19.602644 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:19.605417 28988 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 12:18:19.606716 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.552196 31704 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:19.607707 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.605947 28360 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:19.568487 31687 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:19.569815 31778 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.563336 31781 raft_consensus.cc:2764] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:18:19.614630 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.620429 28432 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:19.569573 31831 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:19.615339 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:19.621294 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.621693 28292 raft_consensus.cc:2368] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:19.622120 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.616950 28362 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:19.613942 28255 leader_election.cc:400] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
I20260709 12:18:19.544145 31821 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.612119 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:19.618379 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:19.584951 31798 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.622872 28255 leader_election.cc:403] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:19.599781 31617 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.588264 31612 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.623945 28360 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:19.626873 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:19.632120 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:19.605309 31785 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:19.635279 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.636830 28362 raft_consensus.cc:2410] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:19.639283 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:19.635892 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:19.622822 31589 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:19.640908 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:19.647528 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.644434 31867 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:19.646155 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54408.1us
I20260709 12:18:19.652235 28984 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:19.645695 31784 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:19.648131 28286 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:19.623498 31831 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.629649 31798 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.630970 31612 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.655063 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:19.655179 31867 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:19.612731 31789 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:18:19.654290 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.676544 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.686210 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:19.698357 28432 raft_consensus.cc:2368] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
W20260709 12:18:19.690541 31852 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
W20260709 12:18:19.690835 31634 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.449s	user 0.007s	sys 0.004s
I20260709 12:18:19.691912 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 447) took 1426 ms. Trace:
I20260709 12:18:19.698359 28292 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:19.648780 31761 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
I20260709 12:18:19.611694 31851 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 12:18:19.709008 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.766712 31666 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:19.767331 28292 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 5.
I20260709 12:18:19.768956 31666 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:19.770365 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1907) took 1348 ms. Trace:
W20260709 12:18:19.771122 31856 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 12:18:19.683187 31612 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.684564 31798 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:19.654141 31784 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:19.778079 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.706689 28986 rpcz_store.cc:276] 0709 12:18:18.265442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.308098 (+ 42656us) service_pool.cc:224] Handling call
0709 12:18:18.331734 (+ 23636us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.185831 (+854097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.185875 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.225933 (+ 40058us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.454816 (+228883us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:19.454865 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.574243 (+119378us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b5400634cd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:19.574297 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.691609 (+117312us) raft_consensus.cc:1685] finished
0709 12:18:19.691631 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.691844 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1963505536,"thread_start_us":228445,"threads_started":1,"wal-append.queue_time_us":306574}
I20260709 12:18:19.653996 31706 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:19.781890 28437 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:18:19.765884 31865 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 12:18:19.790169 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 189389us
W20260709 12:18:19.713470 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 99845.6us
I20260709 12:18:19.657752 31589 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:19.654808 31831 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.711556 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:19.762673 31687 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:19.823778 28367 rpcz_store.cc:276] 0709 12:18:18.421679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.993616 (+571937us) service_pool.cc:224] Handling call
0709 12:18:19.024876 (+ 31260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:19.026580 (+  1704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.026613 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:19.211334 (+184721us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b2c0000e678. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:18:19.211785 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.454495 (+242710us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:19.454533 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.454598 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.770021 (+315423us) raft_consensus.cc:1685] finished
0709 12:18:19.770043 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.770284 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1223,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":6948,"spinlock_wait_cycles":11198720,"thread_start_us":242326,"threads_started":1,"wal-append.queue_time_us":257898}
I20260709 12:18:19.689581 31597 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:20.151103 31858 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.509s	user 0.003s	sys 0.003s
W20260709 12:18:20.151579 31863 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.475s	user 0.000s	sys 0.008s
I20260709 12:18:20.018260 31617 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:20.053650 31801 consensus_meta.cc:287] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.821s	user 0.010s	sys 0.000s
I20260709 12:18:19.872648 28433 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:18:20.155258 31687 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:19.892797 31689 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:20.156261 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:20.109615 31814 consensus_meta.cc:287] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.616s	user 0.007s	sys 0.003s
I20260709 12:18:19.849629 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.122650 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:19.951102 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 392199us
I20260709 12:18:20.122012 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1896) took 1770 ms. Trace:
I20260709 12:18:20.157326 31689 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.002597 31821 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.004897 31778 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:20.121907 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.164599 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.165344 28292 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 6.
I20260709 12:18:20.165966 31883 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.935163 31868 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:20.005945 31870 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.360s	user 0.000s	sys 0.002s
I20260709 12:18:20.123092 28362 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:20.154453 31484 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:18:20.068426 31782 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 1.053s	user 0.008s	sys 0.002s
W20260709 12:18:19.938750 31872 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
W20260709 12:18:20.006110 31855 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.443s	user 0.001s	sys 0.000s
W20260709 12:18:20.122458 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:20.017505 31760 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.773s	user 0.005s	sys 0.001s
W20260709 12:18:19.894268 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:19.822638 31829 consensus_meta.cc:287] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.000s
W20260709 12:18:19.861299 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:20.062983 31824 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.773s	user 0.008s	sys 0.003s
I20260709 12:18:20.170251 28285 raft_consensus.cc:2410] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:19.839851 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.170964 28291 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
W20260709 12:18:19.873333 31694 consensus_meta.cc:287] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.680s	user 0.004s	sys 0.008s
I20260709 12:18:20.171275 28287 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:20.046073 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.171319 28291 pending_rounds.cc:85] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 4
W20260709 12:18:20.172313 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.172358 28291 pending_rounds.cc:107] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:19.934036 31867 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.062753 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:20.127030 31848 consensus_meta.cc:287] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.000s
I20260709 12:18:20.172772 28291 raft_consensus.cc:2889] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:20.062182 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:20.108316 31476 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.002s
I20260709 12:18:20.173954 31816 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.171315 28290 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:20.174561 31476 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.174707 31816 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.158699 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.175738 31848 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:20.169391 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.176939 28364 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 11. Ops: []
I20260709 12:18:20.177026 28432 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:20.099246 31587 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.158s	user 0.008s	sys 0.004s
I20260709 12:18:20.178103 31587 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.160311 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.179076 31848 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.179324 28440 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:19.870893 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:19.822712 31869 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:20.149973 28361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.013s	sys 0.010s
I20260709 12:18:20.134478 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:20.180799 28361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.013s	sys 0.010s
I20260709 12:18:20.181327 28434 raft_consensus.cc:2410] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:18:20.177165 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.155925 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.183166 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.183337 28433 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:20.182798 28186 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 5 to 6, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.167851 31484 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:20.184391 31527 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:20.106997 31762 consensus_meta.cc:287] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.803s	user 0.008s	sys 0.004s
I20260709 12:18:20.175277 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:20.184937 28186 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 6, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.187088 31861 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:20.187521 31861 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.187919 31861 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.187642 28186 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.160085 28359 rpcz_store.cc:276] 0709 12:18:18.351209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.351351 (+   142us) service_pool.cc:224] Handling call
0709 12:18:18.423255 (+ 71904us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:18.459976 (+ 36721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.460016 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.460588 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 12:18:19.367549 (+906961us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:19.367591 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.367661 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:20.121620 (+753959us) raft_consensus.cc:1685] finished
0709 12:18:20.121649 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:20.121938 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36224,"raft.queue_time_us":93960,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":99100,"thread_start_us":861052,"threads_started":1,"wal-append.queue_time_us":949990}
I20260709 12:18:20.189960 31694 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:20.190614 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.190536 31795 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.179316 28437 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:20.192082 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.192378 28437 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:18:20.191198 31795 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.194135 28954 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.183848 28290 raft_consensus.cc:2376] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12. Re-sending same reply.
I20260709 12:18:20.196241 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.196952 28290 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:20.180557 31869 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.166307 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.172056 31589 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.133581 31307 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:20.234107 31307 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:20.235409 31307 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.236717 31307 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:20.123662 31440 consensus_meta.cc:287] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.635s	user 0.006s	sys 0.003s
I20260709 12:18:20.237823 31440 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:20.239789 28332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.006s	sys 0.017s
W20260709 12:18:20.240195 28332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.006s	sys 0.017s
I20260709 12:18:20.240587 31440 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.168291 31782 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.241032 31877 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.241688 31877 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.243366 31782 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.243969 31877 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:19.799299 31832 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.247550 31832 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.250617 31832 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.251221 31784 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.251473 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.184842 31484 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.185611 31762 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.252333 28362 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:20.253162 28283 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:20.254225 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.255007 28366 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:20.255282 31762 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.256460 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:20.121018 31754 consensus_meta.cc:287] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.000s
I20260709 12:18:20.158600 31814 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.192025 31868 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.174022 28435 raft_consensus.cc:2410] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:20.197718 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.259477 31814 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.171087 31778 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:20.210120 31816 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:20.211117 28360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.010s	sys 0.022s
I20260709 12:18:20.259481 31868 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:20.260573 28360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.010s	sys 0.022s
I20260709 12:18:20.261531 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.262182 31484 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:20.172760 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.264871 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.265583 28432 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:20.216709 28434 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.221095 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.266901 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.267676 28437 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 7: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:20.189270 28186 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 4 to 6, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.268194 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:20.198457 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.269752 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:20.170697 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.181984 31883 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.198959 31869 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.199471 31849 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.180567 31587 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:20.278834 28441 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:20.157687 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.280110 28286 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
W20260709 12:18:20.244221 31477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.004s	sys 0.020s
I20260709 12:18:20.199911 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.200219 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.284540 28292 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:20.284710 28285 raft_consensus.cc:2410] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:20.199191 28284 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:20.159894 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:20.203671 31704 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.001s
I20260709 12:18:20.118856 31612 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.177474 31476 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:20.195235 31795 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.290902 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:20.264600 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.291729 28367 raft_consensus.cc:2368] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
W20260709 12:18:20.291736 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.292915 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:20.094332 31586 consensus_meta.cc:287] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.000s
I20260709 12:18:20.270430 28432 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:20.298363 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.299458 28440 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:20.256136 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.300384 28362 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:20.273916 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:20.303798 28954 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:20.104858 31791 consensus_meta.cc:287] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.000s
W20260709 12:18:20.243896 31131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.012s	sys 0.010s
W20260709 12:18:20.271052 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.257462 31754 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.293610 28285 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:20.299266 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.300594 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.308068 28366 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:18:20.305140 31131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.012s	sys 0.011s
W20260709 12:18:20.286692 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 122552us
I20260709 12:18:20.341387 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.202862 31694 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:20.290553 31880 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.056s	user 0.001s	sys 0.003s
I20260709 12:18:20.323983 28433 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:20.274534 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:20.298918 31879 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.063s	user 0.010s	sys 0.000s
I20260709 12:18:20.189111 31652 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:20.342837 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1080584490000547s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25930000000000003s for a response
I20260709 12:18:20.360955 28987 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:20.323866 28432 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:20.285269 31477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.004s	sys 0.021s
I20260709 12:18:20.349290 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.349529 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.323405 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:20.383064 28286 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:20.348838 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.353104 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:20.337456 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.384763 28285 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.356583 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.357656 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.386205 28439 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:20.360332 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.387060 28433 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:20.362390 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 458) took 1475 ms. Trace:
I20260709 12:18:20.362612 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.365914 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.389565 28292 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:20.390579 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:20.390865 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 67557.9us
I20260709 12:18:20.391605 28286 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:20.367825 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.369650 28440 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:20.371889 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:20.374087 31892 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:20.375411 31883 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.298355 31849 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.378882 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1910) took 1956 ms. Trace:
I20260709 12:18:20.341387 31816 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:20.379913 28187 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 237) took 24692 ms (client timeout 15000 ms). Trace:
I20260709 12:18:20.381769 31612 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.200678 31861 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.348713 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.403100 28284 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:20.389467 28283 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:20.396944 28987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 532) took 1140 ms. Trace:
I20260709 12:18:20.404412 28438 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:20.354573 31832 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.398998 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.404645 28987 rpcz_store.cc:276] 0709 12:18:19.242370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.242484 (+   114us) service_pool.cc:224] Handling call
0709 12:18:20.382419 (+1139935us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54004c36d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:20.382559 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2569479040}
I20260709 12:18:20.400924 28285 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:20.401185 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.409332 28287 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.395542 28363 rpcz_store.cc:276] 0709 12:18:18.422522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.013134 (+590612us) service_pool.cc:224] Handling call
0709 12:18:19.013503 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:19.017887 (+  4384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.017923 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:19.235101 (+217178us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b2c0000e678. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:18:19.235764 (+   663us) log.cc:844] Serialized 52 byte log entry
0709 12:18:19.438401 (+202637us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:19.438662 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.439131 (+   469us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:20.378487 (+939356us) raft_consensus.cc:1685] finished
0709 12:18:20.378510 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:20.378797 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":30480256,"thread_start_us":200644,"threads_started":1,"wal-append.queue_time_us":279798}
I20260709 12:18:20.411690 28257 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.411866 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:20.382885 28330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.007s	sys 0.014s
I20260709 12:18:20.411825 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.387895 28289 rpcz_store.cc:276] 0709 12:18:18.887398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.995633 (+108235us) service_pool.cc:224] Handling call
0709 12:18:19.037075 (+ 41442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:19.321263 (+284188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:19.321299 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:19.321958 (+   659us) log.cc:844] Serialized 52 byte log entry
0709 12:18:19.455124 (+133166us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:19.455163 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.455226 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:20.150219 (+694993us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:20.358321 (+208102us) raft_consensus.cc:1685] finished
0709 12:18:20.358346 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:20.362302 (+  3956us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10929,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6956,"spinlock_wait_cycles":7920512,"thread_start_us":85616,"threads_started":1,"wal-append.queue_time_us":202038}
W20260709 12:18:20.412978 28330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.007s	sys 0.014s
I20260709 12:18:20.349062 31694 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:20.414837 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.417143 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.404804 28438 pending_rounds.cc:85] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 4
W20260709 12:18:20.413354 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 777519us
I20260709 12:18:20.413362 28363 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:20.418099 28438 pending_rounds.cc:107] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:18:20.418466 28438 raft_consensus.cc:2889] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:20.307559 31704 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.326133 31586 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.421422 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.421962 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:20.422122 28284 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:20.415107 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.423134 28292 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:20.422056 28288 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:20.305155 31791 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.425582 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.426956 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.441663 28285 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:20.443728 28287 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:20.445403 31762 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.400449 31816 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.445098 28436 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:20.447006 28437 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:20.467221 28186 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 4 to 7, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 7 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.488929 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.415126 31851 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:18:20.469916 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:20.489593 28440 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:20.470991 28288 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:20.410178 31795 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.492789 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.420106 31781 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:18:20.473892 28359 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:20.467020 31765 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.407094 31832 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:20.402163 31883 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:20.476727 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:20.500936 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.476873 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.502214 28362 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:20.419919 31760 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:20.477870 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 70185.8us
I20260709 12:18:20.478437 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:20.419701 31694 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.481063 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.507604 28984 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:20.481580 28360 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.482458 28365 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:20.481803 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.484510 28291 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:20.389405 31687 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 12:18:20.511444 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 277809us
I20260709 12:18:20.487627 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.412046 31587 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:18:20.405668 31831 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.491333 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:20.414999 31652 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:18:20.469674 31814 raft_consensus.cc:2764] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:20.468290 31816 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.473112 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.414819 31843 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:18:20.535549 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.439837 31131 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:18:20.499128 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.499676 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.540473 28435 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.533424 28363 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:18:20.506249 28364 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:20.541500 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.542253 28363 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:20.543227 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:20.543773 28363 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.507795 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.545140 28439 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:20.447827 28257 leader_election.cc:400] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
W20260709 12:18:20.513211 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.503029 31832 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.550087 31831 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.527511 31652 consensus_queue.cc:1243] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:20.526011 28186 catalog_manager.cc:5697] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.553587 28186 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:20.506830 31760 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.536425 28362 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:20.558521 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.558720 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.559342 31760 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:20.559612 28358 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.559973 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.501168 31765 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.508741 28291 pending_rounds.cc:85] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 4
I20260709 12:18:20.496484 31795 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.561156 31798 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.561110 28291 pending_rounds.cc:107] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:20.562260 28291 raft_consensus.cc:2889] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:20.396947 28187 rpcz_store.cc:269] 0709 12:17:55.687014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.689909 (+  2895us) service_pool.cc:224] Handling call
0709 12:17:56.021767 (+331858us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f1090 00007f85be88ff3a 00007f85c4645e9c 00007f85c474404e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc 00007f85c2c04392 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:20.379652 (+24357885us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:13.646783 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:13.647370 (+   587us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:13.647413 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:18:13.647431 (+    18us) tablet.cc:662] Decoding operations
0709 12:18:13.648485 (+  1054us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:13.648550 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:13.648575 (+    25us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:18:13.649499 (+   924us) tablet.cc:701] Row locks acquired
0709 12:18:13.649514 (+    15us) write_op.cc:260] PREPARE: finished
0709 12:18:13.649680 (+   166us) write_op.cc:270] Start()
0709 12:18:13.660710 (+ 11030us) write_op.cc:276] Timestamp: P: 1783599493649624 usec, L: 0
0709 12:18:13.660737 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:18:13.742030 (+ 81293us) log.cc:844] Serialized 4365 byte log entry
0709 12:18:14.415162 (+673132us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:14.856170 (+441008us) op_driver.cc:464] REPLICATION: finished
0709 12:18:15.374358 (+518188us) write_op.cc:301] APPLY: starting
0709 12:18:15.374478 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:15.374795 (+   317us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:15.374830 (+    35us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:15.376436 (+  1606us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:15.376826 (+   390us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:18:15.376872 (+    46us) write_op.cc:312] APPLY: finished
0709 12:18:15.377329 (+   457us) log.cc:844] Serialized 83 byte log entry
0709 12:18:15.377563 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:15.400868 (+ 23305us) write_op.cc:454] Released schema lock
0709 12:18:15.442719 (+ 41851us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:15.443289 (+   570us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:15.443322 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":358121,"reported_tablets":46,"spinlock_wait_cycles":472135424,"child_traces":[["op",{"apply.queue_time_us":517801,"apply.run_cpu_time_us":4392,"apply.run_wall_time_us":68908,"num_ops":11,"prepare.queue_time_us":97557,"prepare.run_cpu_time_us":35814,"prepare.run_wall_time_us":298962,"raft.queue_time_us":195106,"raft.run_cpu_time_us":10010,"raft.run_wall_time_us":245923,"replication_time_us":1195184,"spinlock_wait_cycles":20816128,"thread_start_us":635706,"threads_started":4,"wal-append.queue_time_us":238429,"wal-append.run_cpu_time_us":32445,"wal-append.run_wall_time_us":2398925}]]}
W20260709 12:18:20.546324 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 102886us
I20260709 12:18:20.559252 28364 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
W20260709 12:18:20.547907 31889 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:20.560600 28362 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:20.564329 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:20.564325 28435 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:20.567417 28364 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:20.569644 31131 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:20.570888 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.569919 31760 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:20.565523 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 53586.2us
I20260709 12:18:20.557020 28186 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 7, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 7 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.576005 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.576054 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 507) took 1385 ms. Trace:
I20260709 12:18:20.581516 28436 rpcz_store.cc:276] 0709 12:18:19.191078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.232619 (+ 41541us) service_pool.cc:224] Handling call
0709 12:18:20.575857 (+1343238us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400184054. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:20.575983 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2682577664}
I20260709 12:18:20.583069 31801 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:20.583503 31586 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.583739 31754 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:20.584160 31586 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:20.584728 28257 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.584916 31762 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.585399 31762 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.579267 31814 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:20.586196 31849 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.585799 31762 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.579591 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.582070 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.591391 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.592307 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.587939 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.595175 28363 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:20.596577 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.586853 31849 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:20.599707 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 252218us
I20260709 12:18:20.602030 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.602697 28985 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:20.601631 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.605077 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.605346 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.606568 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.608587 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.616670 28988 raft_consensus.cc:2368] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
I20260709 12:18:20.617234 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:20.617713 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.604463 31704 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.623119 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.623390 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.623390 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.576725 31131 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:20.585894 31791 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.621920 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.625167 28186 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 6 to 9, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 9 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.627460 28990 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:18:20.625101 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 156449us
I20260709 12:18:20.618327 31829 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.600205 31801 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:20.634549 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.618491 31843 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:18:20.639665 31617 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.600656 31587 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:20.642390 31814 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:20.642774 31587 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:20.600517 31781 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:18:20.643256 31587 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.600401 31652 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.621307 31825 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:20.628106 28330 leader_election.cc:336] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.579s (SENT)
I20260709 12:18:20.584575 31589 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:18:20.631166 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.628494 28988 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.633176 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:20.651103 28359 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:20.629248 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.636356 31806 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:18:20.600772 31845 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:20.638514 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.643158 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.655957 28286 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.635264 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.638123 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.630779 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:20.656988 28285 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.589987 31760 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.629855 31821 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:20.604854 31586 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.635429 31861 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:20.584636 31784 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.622355 31851 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:20.654917 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.619462 31527 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:18:20.638824 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.657188 28363 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.645923 31814 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.677340 31814 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:20.677572 28286 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
W20260709 12:18:20.683427 31893 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.288s	user 0.003s	sys 0.000s
I20260709 12:18:20.657841 31845 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:20.688962 31845 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:20.689365 31829 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:20.691536 31587 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.696086 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1913) took 1038 ms. Trace:
I20260709 12:18:20.694028 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.696488 28365 rpcz_store.cc:276] 0709 12:18:19.657611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.776142 (+118531us) service_pool.cc:224] Handling call
0709 12:18:19.776507 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:20.508137 (+731630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:20.695817 (+187680us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54003babd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:20.696018 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622984064}
I20260709 12:18:20.692766 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.699947 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.659654 31795 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.702800 28255 leader_election.cc:400] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
I20260709 12:18:20.703009 31795 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:20.703493 31795 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:20.703900 31795 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:20.696622 31587 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:20.667862 31851 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:20.709846 31851 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.710034 31131 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.711453 31131 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.712172 28255 leader_election.cc:403] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:20.717970 28288 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:20.660195 31824 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:18:20.725626 31694 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.751678 31484 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:20.752179 31484 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.752522 31484 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:20.737426 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:20.753597 28359 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:20.753981 31823 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.737051 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:20.758204 31589 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.758773 31589 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.759191 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:20.759768 28365 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:20.741981 31307 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.738359 28367 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:20.762844 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.651507 28361 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:20.662412 28330 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.606s (SENT)
I20260709 12:18:20.763633 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:20.764638 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 102278us
W20260709 12:18:20.765247 28330 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.689s (SENT)
I20260709 12:18:20.765640 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:20.766057 28367 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:20.766721 31867 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:20.767360 28367 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:20.767437 31867 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:20.767817 28330 leader_election.cc:336] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.241s (SENT)
I20260709 12:18:20.633904 31801 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.768419 31801 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:20.647212 31652 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:20.768608 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.768895 31652 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:20.769110 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.643136 31798 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.648808 31825 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.668632 28290 raft_consensus.cc:2410] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:20.662149 31821 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.763046 28986 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 6: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:20.738967 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.790882 28360 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:20.746919 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1905) took 1179 ms. Trace:
I20260709 12:18:20.791736 28358 rpcz_store.cc:276] 0709 12:18:19.567570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.649500 (+ 81930us) service_pool.cc:224] Handling call
0709 12:18:19.682708 (+ 33208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:20.559998 (+877290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:20.746624 (+186626us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54003bc4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:20.746836 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016255872}
I20260709 12:18:20.744226 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.792972 28366 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
W20260709 12:18:20.772014 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 163494us
I20260709 12:18:20.750051 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:20.794729 28364 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:20.776068 28437 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.776798 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.659528 28186 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 5 to 8, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 8 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:20.786481 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.788187 28363 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:20.803006 28186 catalog_manager.cc:5697] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 6, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.803974 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:20.805321 28186 catalog_manager.cc:5697] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 2 to 7, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 7 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:20.807479 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:20.807863 28186 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 6, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 6 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:20.810276 28359 raft_consensus.cc:2410] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:18:20.812661 28285 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:20.814334 28361 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:20.815977 31765 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:20.816599 31587 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:20.734783 31869 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.818008 31527 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:20.818528 31527 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.818863 31527 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.753531 31694 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.819185 31869 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:20.822569 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 167724us
I20260709 12:18:20.823429 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.823832 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.825809 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:20.846846 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 216448us
I20260709 12:18:20.847478 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:20.801203 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.849591 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:20.850592 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.850950 28365 raft_consensus.cc:2410] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:20.851649 28437 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:20.847723 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:20.858100 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.858842 28441 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:20.859238 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.861146 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.862450 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:20.864907 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:20.736357 31784 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:20.810444 31825 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:20.748593 31760 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:20.872457 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:20.875795 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.799582 31821 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.875068 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:20.879652 28284 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.875355 28285 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:20.880438 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.821223 31781 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.884546 28364 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 6: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:20.849095 31861 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:20.821323 31843 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:18:20.849532 31687 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:20.887657 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.888219 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.850232 31788 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:18:20.856557 31782 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:20.887044 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:20.903296 28437 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.891871 28989 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:20.870985 31694 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:20.885486 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:20.899221 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.747764 31832 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.906628 31782 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.911646 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.912462 31782 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.913564 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:20.914623 31883 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:20.914631 28288 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.915740 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.916501 28283 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:20.907374 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.871385 31754 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.877849 31899 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:20.885190 31689 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:20.919168 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.935128 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.935750 28434 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:20.890858 31907 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:20.919159 28432 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:20.889627 31778 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.921397 28255 leader_election.cc:400] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
I20260709 12:18:20.922384 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.922554 31754 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:20.922979 31899 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.924098 31816 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.889889 31861 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:20.938516 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:20.929380 31860 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:20.890682 31587 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.888937 31760 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:20.962407 31760 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.962847 31760 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.945036 31869 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:18:20.912140 31781 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:18:20.906069 31484 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.937216 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.967623 28286 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:20.968652 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.919340 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.969064 31762 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.969522 28283 raft_consensus.cc:2376] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9. Re-sending same reply.
I20260709 12:18:20.970779 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.937585 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:20.937975 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.972617 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.972795 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:20.938536 28284 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 6.
I20260709 12:18:20.933566 31814 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.974373 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:20.974578 31860 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.974990 28284 raft_consensus.cc:2376] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9. Re-sending same reply.
I20260709 12:18:20.975382 31860 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.975915 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.961293 31861 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:20.977094 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.931978 31851 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.977876 31825 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:20.904994 31687 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:20.978734 31687 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.979475 31823 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
I20260709 12:18:20.980077 28441 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:20.981189 28983 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:20.954578 28987 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.955708 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.873265 31765 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:18:20.958629 28288 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:20.981978 31754 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:20.982908 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:20.983724 28288 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:20.958643 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.960577 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1933) took 1117 ms. Trace:
I20260709 12:18:20.939196 31843 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:20.921165 31589 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.975956 31814 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.942489 31784 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:18:20.978169 31832 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.940580 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.019177 28436 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.986927 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:20.944492 28255 leader_election.cc:403] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:21.022377 28987 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:20.978236 28286 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:21.024071 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 332714us
I20260709 12:18:20.989894 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:21.027298 28289 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:20.992017 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:21.028542 28288 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:20.988284 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:20.989223 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:21.031237 28436 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:21.031445 28284 raft_consensus.cc:2410] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:18:21.032591 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44385.8us
I20260709 12:18:21.034159 28330 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.016559 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:21.022521 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 101254us
I20260709 12:18:20.985572 28290 rpcz_store.cc:276] 0709 12:18:19.843044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.197466 (+354422us) service_pool.cc:224] Handling call
0709 12:18:20.950057 (+752591us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54002422d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:20.960504 (+ 10447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939019136}
I20260709 12:18:20.979631 31851 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:20.978521 31899 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:21.024886 28440 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:20.989529 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.036424 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:21.062065 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:21.040838 28286 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:21.041833 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.053557 28330 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:21.057467 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.058866 28285 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:21.062816 28289 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:21.071358 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.072260 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:20.967662 31484 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:20.969501 31801 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:20.990103 31843 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:21.079474 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:21.100319 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.146365 28441 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.080185 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:21.150550 28363 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:21.113374 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.100625 28440 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:18:21.159902 28332 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.3:33721 timed out after 1.509s (SENT) [suppressed 41 similar messages]
W20260709 12:18:21.110291 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 149271us
I20260709 12:18:21.079681 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:21.025282 31823 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10.
I20260709 12:18:21.165647 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:21.166541 28366 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.080924 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.177019 31840 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:21.052757 31860 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:21.051769 31824 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
W20260709 12:18:21.183350 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 105675us
I20260709 12:18:21.183132 31484 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.183960 31911 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:21.184748 31911 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:21.185868 31911 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.187367 31911 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.188349 28366 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:21.165014 31762 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:21.192600 31816 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:21.061623 31798 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:21.161551 31851 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.182088 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:21.097617 31617 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.982975 31754 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.203227 31798 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.205016 31798 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:21.200116 31795 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.210101 31795 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:21.213919 31851 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.215024 31484 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:18:21.216017 31484 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:21.216379 31484 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:21.217468 31484 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.217999 31851 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:21.200762 28287 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:21.218758 31484 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:21.201303 28433 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:21.223985 31652 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.225243 28435 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:20.990058 31869 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.171s
I20260709 12:18:21.124107 31806 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:21.217816 31912 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:18:21.096619 31477 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.204245 28364 raft_consensus.cc:606] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.202510 28283 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:21.213541 31798 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:21.233618 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.229664 28437 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:21.237587 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.243674 28364 raft_consensus.cc:614] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:21.254321 31912 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:21.254786 31912 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:21.233467 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.255667 31912 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.235960 31806 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.257093 31912 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:21.257098 31806 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:21.256345 28433 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:21.260123 28437 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:21.263296 28361 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 7: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:21.195329 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:21.265013 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 69801.1us
I20260709 12:18:21.266093 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.268153 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:21.275591 28332 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:21.234038 31849 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.003s
I20260709 12:18:21.277696 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.278537 28436 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:21.248126 28289 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:21.287873 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.291986 28436 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
W20260709 12:18:21.304836 28367 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.000s	sys 0.025s
I20260709 12:18:21.305151 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:21.305960 28364 raft_consensus.cc:606] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.306356 28364 raft_consensus.cc:614] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:21.306459 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 440061us
W20260709 12:18:21.307124 28367 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.000s	sys 0.025s
I20260709 12:18:21.202695 28363 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:21.254801 31825 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:21.321105 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:21.336695 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:21.342088 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:21.239846 31843 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.340050 31689 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:21.265029 31861 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.257184 31687 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.304301 31784 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:18:21.259608 31527 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.342167 31883 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:21.267117 31694 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:21.407881 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 494340us
I20260709 12:18:21.313520 31484 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:21.300815 31823 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:21.345146 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.334637 31867 raft_consensus.cc:2764] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
W20260709 12:18:21.421173 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 112801us
I20260709 12:18:21.432869 28441 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
W20260709 12:18:21.460997 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 116303us
I20260709 12:18:21.304649 31849 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.462092 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.462797 28435 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:21.352350 31907 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:18:21.464416 28434 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:21.434755 31617 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:21.436959 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 367108us
I20260709 12:18:21.334897 31860 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 12:18:21.339004 31788 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 12:18:21.442163 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:21.451650 31484 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.447811 28954 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:21.421754 31829 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:21.421715 31587 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.352896 31477 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.097s
I20260709 12:18:21.352531 31795 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:21.352780 31906 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:18:21.456410 31778 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:18:21.486487 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:21.489813 28364 raft_consensus.cc:606] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:21.516337 28364 raft_consensus.cc:614] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:21.390931 31652 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:21.490450 28292 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:21.498378 31484 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.501473 31829 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.496222 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:21.501401 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.467264 31586 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.468111 31825 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:21.419997 31851 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 12:18:21.508759 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 61035.8us
I20260709 12:18:21.490998 31762 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:21.501708 31823 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:21.507863 31869 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:21.531546 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:21.530851 31849 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:21.509898 31801 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:21.523620 31845 consensus_meta.cc:287] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.834s	user 0.009s	sys 0.002s
I20260709 12:18:21.552202 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:21.522111 31900 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:21.502367 31617 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.564392 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:21.586716 28289 raft_consensus.cc:2368] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11.
I20260709 12:18:21.521768 31795 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:21.532196 31829 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:21.536415 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.507344 31587 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.492929 31914 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:18:21.576378 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45052.4us
I20260709 12:18:21.531150 31798 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:21.582577 28285 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:21.503450 31689 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.550413 31906 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:21.587803 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:21.550151 31586 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.562260 31907 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.563150 31823 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:21.582645 31821 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.000s
W20260709 12:18:21.597543 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 61228.8us
I20260709 12:18:21.519205 31883 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.614668 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.586086 31754 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:21.616169 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.508616 31694 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.601874 28364 raft_consensus.cc:606] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.618122 28364 raft_consensus.cc:614] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:21.575297 31849 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:21.580370 31926 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:21.590708 31900 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:21.620098 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:21.599961 31801 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.650969 28364 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:21.651711 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:18:21.657191 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:21.653441 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:21.660715 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:21.662904 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:21.663554 28364 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:21.617765 31762 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:21.586575 31845 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.619026 31754 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.665819 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:21.630421 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.668751 28363 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:21.668843 28436 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:21.619215 31860 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:21.630015 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:21.651262 31923 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:18:21.621727 31825 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:21.672580 28441 raft_consensus.cc:2376] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10. Re-sending same reply.
W20260709 12:18:21.673293 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 188387us
I20260709 12:18:21.621906 31798 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:21.690238 28441 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:21.659526 31906 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:21.664928 31527 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:21.707881 31931 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:18:21.674371 31754 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:21.710465 31930 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:18:21.711795 31920 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 12:18:21.712652 31915 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 12:18:21.714051 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 468) took 1085 ms. Trace:
W20260709 12:18:21.714653 31782 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.802s	user 0.011s	sys 0.001s
I20260709 12:18:21.715145 31782 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.715852 31586 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:21.666141 31816 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:18:21.716887 31911 consensus_meta.cc:287] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.001s
I20260709 12:18:21.646538 31687 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:21.718716 31845 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:21.693393 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 206635us
I20260709 12:18:21.726564 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:21.674451 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.622152 31821 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.618642 31907 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.690135 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:21.714417 28291 rpcz_store.cc:276] 0709 12:18:20.629114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.630294 (+  1180us) service_pool.cc:224] Handling call
0709 12:18:20.639345 (+  9051us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:20.790870 (+151525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:20.790913 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:20.815444 (+ 24531us) log.cc:844] Serialized 26 byte log entry
0709 12:18:21.241520 (+426076us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:21.241560 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.242569 (+  1009us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:21.713804 (+471235us) raft_consensus.cc:1685] finished
0709 12:18:21.713824 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:21.713984 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8029,"raft.queue_time_us":53292,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":500129,"threads_started":2,"wal-append.queue_time_us":588916}
I20260709 12:18:21.729460 28433 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:21.674762 31860 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:21.630555 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.610551 31784 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:18:21.728047 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:21.686306 31926 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:21.725551 31922 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
W20260709 12:18:21.725754 31908 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 12:18:21.690997 31798 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:21.740623 28441 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:21.741714 28364 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:21.726483 31477 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:21.725271 31906 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:21.748683 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:21.739115 31816 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:21.747759 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1933) took 1011 ms. Trace:
I20260709 12:18:21.752539 28436 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:21.691284 31762 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:21.751477 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:21.772018 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.742954 31934 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:21.777138 31762 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.778883 31782 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:21.779181 31806 consensus_meta.cc:287] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.001s
I20260709 12:18:21.779626 31806 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.747303 31900 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.786329 31926 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.693331 31849 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.790115 28287 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:21.738389 31921 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:18:21.774828 31825 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:21.697805 31795 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:21.790889 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.793211 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:21.799278 28441 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:21.792845 28330 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:21.801153 28330 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:21.759528 31477 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:21.790580 31929 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:18:21.691725 31823 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.795207 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:21.808785 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:21.819419 31933 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:18:21.820438 31687 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:21.821098 31687 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:21.822185 31687 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.775329 31816 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:21.824390 31816 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.824756 31906 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.795111 31860 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:21.795352 31760 consensus_meta.cc:287] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.832s	user 0.012s	sys 0.001s
I20260709 12:18:21.797382 31938 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.777227 31934 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:21.846868 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:21.854297 28332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.003s	sys 0.011s
I20260709 12:18:21.790941 31936 raft_consensus.cc:2764] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:18:21.848625 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44967.3us
W20260709 12:18:21.849098 28440 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.012s	sys 0.015s
I20260709 12:18:21.849334 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 409) took 1202 ms. Trace:
I20260709 12:18:21.794411 31825 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:21.854732 28332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.003s	sys 0.011s
W20260709 12:18:21.879029 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 535057us
I20260709 12:18:21.877409 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:21.879475 28362 rpcz_store.cc:276] 0709 12:18:20.646620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.646780 (+   160us) service_pool.cc:224] Handling call
0709 12:18:20.647153 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:20.648366 (+  1213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:20.648399 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:20.656740 (+  8341us) log.cc:844] Serialized 26 byte log entry
0709 12:18:21.016081 (+359341us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:21.016129 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.016462 (+   333us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:21.849037 (+832575us) raft_consensus.cc:1685] finished
0709 12:18:21.849067 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:21.849240 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1283,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5724,"thread_start_us":327449,"threads_started":1,"wal-append.queue_time_us":704550}
I20260709 12:18:21.881491 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:21.882362 28440 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.013s	sys 0.015s
I20260709 12:18:21.885421 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:21.886129 28441 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:21.887594 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:21.847858 31782 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:21.856775 31932 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.140s	user 0.005s	sys 0.002s
W20260709 12:18:21.852790 31899 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.684s	user 0.014s	sys 0.000s
I20260709 12:18:21.844460 31937 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:21.804785 31911 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:18:21.901629 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:21.884987 31825 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:21.878026 31823 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.859654 31934 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:21.909482 31587 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:21.907232 31912 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.646s	user 0.012s	sys 0.000s
I20260709 12:18:21.893013 31586 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:21.913151 31937 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:21.886816 31617 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:21.901021 31860 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.853883 31760 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.903234 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:21.901523 31806 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:21.911590 31765 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:21.915380 31936 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:21.912559 31939 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:21.851523 31798 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.911643 31899 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.928527 31788 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:21.928380 31586 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:21.976308 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 243653us
W20260709 12:18:21.979947 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.914011 31754 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:21.912716 31782 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:22.037370 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.047036 28438 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:21.955421 31936 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:22.047780 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 239083us
I20260709 12:18:22.043645 31926 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.080749 31936 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.951905 31765 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:22.049430 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 168241us
I20260709 12:18:21.914333 31477 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:21.981065 31907 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:21.980811 31586 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.081668 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:22.048041 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:22.078694 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 177164us
I20260709 12:18:21.977852 31821 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:21.976400 31788 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.982167 31806 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.944849 31937 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:22.088858 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 171525us
I20260709 12:18:22.090051 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:22.092653 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:22.049890 31798 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:22.095427 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 47474.1us
W20260709 12:18:22.099197 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.099651 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:22.102288 28432 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.081979 31765 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.110180 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:22.110487 28983 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:22.110946 28359 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:22.112183 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:22.113372 28439 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:22.117365 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:22.093806 31788 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.078117 31829 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.154356 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:22.162123 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.162420 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:22.164116 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.166880 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:22.166527 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:22.167912 28288 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:22.168165 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:22.103514 31652 raft_consensus.cc:2764] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:22.170038 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.178251 31587 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.178551 28983 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:22.178951 31587 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.179409 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.180405 28987 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:18:22.181285 31587 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:22.182060 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.182928 28289 raft_consensus.cc:2368] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7.
I20260709 12:18:22.184094 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.154088 31907 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:22.191115 31754 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:22.193152 28432 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
W20260709 12:18:22.196768 31941 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:18:22.199506 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.200389 28292 raft_consensus.cc:2368] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
I20260709 12:18:22.201287 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:22.211596 28292 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:22.149138 31821 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:18:22.178261 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:22.185412 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.213080 28289 raft_consensus.cc:2368] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
I20260709 12:18:22.206972 28286 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:22.208742 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.109025 31911 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.218159 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:22.219161 28286 raft_consensus.cc:2368] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
I20260709 12:18:22.220108 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.220359 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.215278 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:22.220860 28285 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:21.194543 31307 consensus_queue.cc:1175] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6634fa3fbd594726a5ec3c1686900284" responder_term: 9 status { last_received { term: 9 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:22.226872 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.227813 28292 raft_consensus.cc:2368] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
I20260709 12:18:22.229839 31952 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
W20260709 12:18:22.223439 31484 consensus_meta.cc:287] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.006s
I20260709 12:18:22.093194 31942 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
W20260709 12:18:22.251821 28404 connection.cc:256] RPC call timeout handler was delayed by -1.8949879569997847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 12:18:22.252841 28404 connection.cc:256] RPC call timeout handler was delayed by -1.0051500879999367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260709 12:18:22.253813 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:22.240813 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.100834 31806 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.106712 31845 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.110s
W20260709 12:18:22.243476 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 161878us
I20260709 12:18:22.243844 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.257191 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:22.259066 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:22.261026 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:22.263891 28292 raft_consensus.cc:2368] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:22.263695 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:22.265664 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 175685us
I20260709 12:18:22.267658 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:22.279268 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:22.289633 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:22.254343 31821 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:22.301605 28292 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:22.257926 31652 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:22.309804 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1939) took 1034 ms. Trace:
I20260709 12:18:22.310253 28435 rpcz_store.cc:276] 0709 12:18:21.275768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.275947 (+   179us) service_pool.cc:224] Handling call
0709 12:18:21.462474 (+186527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:21.463283 (+   809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.309540 (+846257us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540026a2d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:22.309705 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427925376}
I20260709 12:18:22.306751 28288 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:22.314385 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.318427 28288 raft_consensus.cc:2376] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8. Re-sending same reply.
W20260709 12:18:22.321472 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 62480.3us
I20260709 12:18:22.322993 28288 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:18:22.323436 28283 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:22.297250 31955 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:22.330114 28288 pending_rounds.cc:85] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 3
I20260709 12:18:22.340817 28288 pending_rounds.cc:107] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:18:22.258810 31484 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.323940 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:22.336930 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.330463 28283 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:22.341244 28288 raft_consensus.cc:2889] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:22.327826 28285 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:22.366389 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.367271 28364 raft_consensus.cc:2368] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11.
I20260709 12:18:22.367740 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:22.369329 28363 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:22.303030 31821 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:22.264657 31953 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:18:22.371857 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.372562 28363 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.349712 28283 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:22.375888 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.376547 28364 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.377153 28285 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:18:22.383594 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.390980 28289 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:22.394068 28364 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:22.402784 28285 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:22.348435 31760 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:22.399498 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:22.405409 28283 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:18:22.406590 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.407348 28358 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.370889 31867 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:22.409933 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.415192 28366 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:22.365572 31955 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:22.389215 31942 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:22.430949 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.389719 31899 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:22.431799 28364 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.450275 28363 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:22.406162 31970 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:18:22.462544 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 94924.6us
W20260709 12:18:22.426532 31944 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
I20260709 12:18:22.464349 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:22.468829 28366 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:22.411979 31911 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:22.424155 31867 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.425071 31906 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.472221 31952 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:22.486672 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.487792 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:22.464571 31760 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.465132 31970 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:22.483214 31867 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.485967 31911 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:22.491344 31782 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:22.441960 31821 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.492681 31958 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.493335 31907 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:22.498886 31947 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 12:18:22.538834 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 1924) took 1657 ms. Trace:
I20260709 12:18:22.548941 28438 rpcz_store.cc:276] 0709 12:18:20.881333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.881471 (+   138us) service_pool.cc:224] Handling call
0709 12:18:20.890047 (+  8576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.047520 (+1157473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.538535 (+491015us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003cb254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:22.538743 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2660365696}
I20260709 12:18:22.551764 28360 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:22.488775 31906 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.541141 28358 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:22.540750 28290 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:22.521381 28364 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:22.530220 31952 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:22.497789 31829 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:22.536424 31946 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:18:22.570147 28360 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:22.507350 31806 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:22.576797 28358 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:22.579136 28954 leader_election.cc:400] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11.
I20260709 12:18:22.580237 28954 leader_election.cc:403] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 12:18:22.538192 31961 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:18:22.531214 31971 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.516342 31977 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:22.583775 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.584347 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:22.585182 28360 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:22.586112 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 320006us
W20260709 12:18:22.571285 31929 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:18:22.582643 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.500535 31760 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.593966 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:22.596516 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:22.596553 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:22.601924 28360 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:22.544096 31912 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:22.602928 31760 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:22.603153 31912 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:22.604169 31912 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.600987 31974 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:22.602979 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.605705 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.604811 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:22.581640 31821 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.582085 31687 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.593885 31694 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11.
I20260709 12:18:22.612475 28364 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:22.605257 31912 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.572193 31907 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.605881 31484 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:22.670472 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.674562 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:22.615703 31921 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:22.537650 31911 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.593645 31754 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:18:22.679778 28364 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:22.632066 31926 consensus_meta.cc:287] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.002s
I20260709 12:18:22.675844 31975 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:22.675992 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 71395.9us
I20260709 12:18:22.695879 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:22.654735 31816 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:22.584605 31958 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.713871 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:22.612155 31587 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
W20260709 12:18:22.691035 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 203340us
I20260709 12:18:22.652809 31980 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:22.681990 31974 consensus_queue.cc:1243] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer f526369c0d0743b192be6e205540a059 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:18:22.729137 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 132714us
I20260709 12:18:22.690407 31527 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:22.730100 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:22.754348 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:22.755079 28283 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.755652 28289 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:22.684118 31952 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.627564 31970 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.720582 31926 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:22.757966 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:22.697098 31921 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:22.720085 31975 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.584774 31955 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.752583 31816 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:22.729471 31851 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:22.675154 31687 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.763561 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:22.773674 28285 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:22.817272 28287 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:22.635082 31987 consensus_queue.cc:1059] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.145s
W20260709 12:18:22.778277 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49007.7us
I20260709 12:18:22.818403 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:22.764910 31484 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.753794 31527 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:22.772571 28364 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:22.805250 28283 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
W20260709 12:18:22.776577 31937 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.000s
I20260709 12:18:22.838500 31937 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.841374 31937 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:22.803107 31967 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
W20260709 12:18:22.862977 31765 consensus_meta.cc:287] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.714s	user 0.009s	sys 0.003s
I20260709 12:18:22.869071 31765 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:22.823693 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 65802.5us
I20260709 12:18:22.827746 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:22.874692 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47057us
I20260709 12:18:22.875286 31937 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:22.876080 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:22.876266 31687 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:22.876945 31937 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:22.879251 28989 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:18:22.828378 31973 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:18:22.837656 31906 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:22.802421 28289 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:22.843027 28365 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.845860 31977 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.847990 31914 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:22.854035 31861 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:22.871325 31958 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:22.873466 31765 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:22.883083 28330 connection.cc:256] RPC call timeout handler was delayed by -2.0300826079999297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 12:18:22.919756 28330 connection.cc:256] RPC call timeout handler was delayed by -1.0689241259999562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
I20260709 12:18:22.765872 31694 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:18:22.887460 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:22.924623 28989 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:22.804580 31975 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.880090 31937 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:22.910133 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:22.954408 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.910948 28987 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:22.838085 31971 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:22.822208 31798 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.718s	user 0.006s	sys 0.009s
I20260709 12:18:22.917374 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:22.803534 31917 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.145s
W20260709 12:18:22.888309 31964 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 12:18:22.806823 31978 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.880486 31527 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:22.962136 31983 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:18:22.962806 31937 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.882977 31484 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.967010 31958 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.967681 31958 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:22.970150 31921 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.971230 31976 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:22.971693 31976 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:22.972563 31976 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:22.974397 31816 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:22.901100 31825 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.975s	user 0.009s	sys 0.004s
I20260709 12:18:22.801112 31985 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:22.942982 28365 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:22.911895 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:22.954473 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:22.955508 28364 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:22.899359 31869 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:22.916659 31587 consensus_queue.cc:1243] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer f526369c0d0743b192be6e205540a059 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:18:22.959749 31968 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.343s	user 0.000s	sys 0.002s
W20260709 12:18:22.960167 31965 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:18:22.921245 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:22.979858 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.043143 28292 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.967758 31484 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:22.998694 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 244788us
W20260709 12:18:22.945664 31934 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.998s	user 0.009s	sys 0.005s
I20260709 12:18:23.023988 28289 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:23.024596 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.050253 28285 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.983506 28360 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
W20260709 12:18:23.046797 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.048519 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 127368us
I20260709 12:18:23.052264 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:22.998306 31307 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.052608 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:23.013937 31694 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.928987 31914 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:18:23.014268 31976 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.059894 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:23.062237 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.957893 31765 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 12:18:23.026597 31869 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.020959 31971 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:23.069260 28364 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:18:23.068729 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:23.070129 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:23.054642 31867 consensus_meta.cc:287] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.536s	user 0.013s	sys 0.000s
I20260709 12:18:23.070597 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:23.047627 31954 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.770004 28367 rpcz_store.cc:276] 0709 12:18:20.736590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.768435 (+ 31845us) service_pool.cc:224] Handling call
0709 12:18:20.768746 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:20.768972 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:21.308717 (+539745us) log.cc:844] Serialized 14 byte log entry
0709 12:18:21.740547 (+431830us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:21.740620 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:21.741567 (+   947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.746762 (+  5195us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:21.747686 (+   924us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":687720,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":325,"raft.queue_time_us":198079,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":20564,"thread_start_us":970691,"threads_started":3,"wal-append.queue_time_us":342678}
I20260709 12:18:23.053238 31943 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.912844 31977 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.071077 28986 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:23.018705 31825 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:23.070931 31936 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.989s	user 0.011s	sys 0.003s
I20260709 12:18:23.092226 28292 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:23.063905 31907 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:23.092717 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 212017us
I20260709 12:18:23.085047 28364 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:18:23.123694 31781 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:23.077288 31942 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.017s	sys 0.002s
I20260709 12:18:23.132357 31781 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:23.133008 31942 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.018s	sys 0.002s
I20260709 12:18:23.133361 31942 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:23.134831 31781 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:23.134549 31942 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.135684 31986 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:23.132084 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.137589 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.139271 31825 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:23.123773 31989 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:18:23.140898 31989 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:23.141260 31989 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:23.143124 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:23.143966 28986 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:23.127687 31936 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.146104 28367 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:23.147266 31936 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.149417 28284 raft_consensus.cc:2368] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:18:23.142156 31989 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.154883 31989 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:23.157974 28292 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:23.076994 31798 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:23.159653 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1946) took 1009 ms. Trace:
W20260709 12:18:23.077911 31788 consensus_meta.cc:287] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.875s	user 0.011s	sys 0.003s
I20260709 12:18:23.159881 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.161863 28365 rpcz_store.cc:276] 0709 12:18:22.150040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.150470 (+   430us) service_pool.cc:224] Handling call
0709 12:18:23.159475 (+1009005us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b5400103454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:23.159587 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1441372928}
W20260709 12:18:23.080669 31586 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.984s	user 0.007s	sys 0.007s
I20260709 12:18:23.063889 31307 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.056679 31918 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:18:23.173893 28364 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:23.069648 31694 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.168947 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:23.025308 31985 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:23.171941 28434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.022s	sys 0.035s
I20260709 12:18:23.172403 28285 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:23.083950 31795 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.177244 28434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.022s	sys 0.035s
I20260709 12:18:23.091228 31484 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:23.073231 31845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.001s	sys 0.019s
I20260709 12:18:23.163626 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:23.057313 31934 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.186705 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:23.192685 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:23.183169 31845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.002s	sys 0.020s
I20260709 12:18:23.190877 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.192792 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:23.194681 28432 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:23.074910 31867 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.201555 28989 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:23.079296 31954 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.208724 28404 leader_election.cc:400] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 6 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7.
I20260709 12:18:23.181216 31899 raft_consensus.cc:2764] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:23.214439 31975 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.090708 31978 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 12:18:23.216624 31975 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.217476 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:23.178467 31586 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.219046 28291 raft_consensus.cc:2368] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:23.182587 31795 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.220094 31586 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.220789 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1915) took 1110 ms. Trace:
I20260709 12:18:23.221138 31795 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.194084 31955 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:23.223215 31955 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.223587 31955 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:23.211621 31990 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:18:23.221174 28291 rpcz_store.cc:276] 0709 12:18:22.110792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.147494 (+ 36702us) service_pool.cc:224] Handling call
0709 12:18:23.220576 (+1073082us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002422d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:23.220724 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2417744640}
I20260709 12:18:23.227881 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.228278 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:23.194023 31991 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.229096 31991 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.216696 31904 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:18:23.232908 31904 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:23.233335 31904 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:23.233067 31958 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.213114 31985 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:23.171450 31788 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.228675 28291 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:18:23.213510 31982 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.454s	user 0.004s	sys 0.000s
I20260709 12:18:23.212536 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.259560 28285 raft_consensus.cc:2410] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:23.218827 31851 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:18:23.260308 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 197722us
I20260709 12:18:23.240160 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:23.242115 28361 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:23.191802 31484 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:23.246642 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:23.231676 31991 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:23.221252 32002 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:23.264232 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.216180 31867 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.302953 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:23.304072 28291 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:23.263401 31851 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:23.274389 28432 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:23.288668 28330 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:23.285813 28439 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:23.268129 28361 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:23.306465 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.306733 28954 leader_election.cc:400] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11.
W20260709 12:18:23.319017 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.293648 31911 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.306689 31823 consensus_meta.cc:287] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.379s	user 0.013s	sys 0.003s
I20260709 12:18:23.352816 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:23.363621 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 123553us
W20260709 12:18:23.358906 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 52324.3us
I20260709 12:18:23.308292 31991 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:23.360664 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 72152.9us
I20260709 12:18:23.305440 31993 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:23.352183 28441 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
I20260709 12:18:23.200984 31971 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.398465 28361 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:23.399212 28404 leader_election.cc:400] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
I20260709 12:18:23.308293 31527 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.399454 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:23.403218 28432 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:23.405489 28404 leader_election.cc:400] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
I20260709 12:18:23.405864 28404 leader_election.cc:403] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:18:23.353168 32003 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:23.374327 31992 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:23.460856 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 55509.8us
I20260709 12:18:23.437336 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2184) took 1051 ms. Trace:
I20260709 12:18:23.461922 31694 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:23.462087 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.463506 31527 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:23.466740 32004 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:18:23.467001 31998 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:18:23.467310 31988 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:18:23.467622 31851 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:23.470626 32006 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:18:23.408099 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.473151 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:23.473289 28439 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:23.400908 31843 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.474956 28361 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:23.473527 28435 rpcz_store.cc:276] 0709 12:18:22.386071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.386261 (+   190us) service_pool.cc:224] Handling call
0709 12:18:22.386621 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.386995 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.387025 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.387517 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.576497 (+188980us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:22.576542 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.576615 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.212735 (+636120us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:23.361935 (+149200us) raft_consensus.cc:1685] finished
0709 12:18:23.361959 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.437205 (+ 75246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1245,"spinlock_wait_cycles":19163648,"thread_start_us":146303,"threads_started":1,"wal-append.queue_time_us":283933}
I20260709 12:18:23.396898 31907 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.482062 28432 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:23.483498 28441 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:23.503007 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:23.400565 31917 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 12:18:23.504355 28404 leader_election.cc:400] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
I20260709 12:18:23.504173 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.506212 28404 leader_election.cc:403] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:23.503109 28441 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:23.506794 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.506515 28432 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:23.400700 31911 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:23.509629 31995 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:18:23.510581 31997 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:18:23.511647 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.513827 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.527604 28283 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:23.527812 28285 raft_consensus.cc:2410] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:18:23.475375 31937 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.474473 31914 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
I20260709 12:18:23.518532 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:23.255090 31904 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.504590 31988 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:23.507510 28441 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.516505 31954 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.524958 31788 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:23.400044 31484 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.525558 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:23.402560 31991 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.522166 28432 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:23.526784 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1920) took 1364 ms. Trace:
W20260709 12:18:23.505085 31860 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.553s	user 0.014s	sys 0.002s
I20260709 12:18:23.528713 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:23.528712 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 130428us
I20260709 12:18:23.530607 28404 leader_election.cc:400] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
I20260709 12:18:23.532089 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.508157 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.503438 31307 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.534140 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:23.574819 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:23.536586 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.548127 28441 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.539268 28434 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:23.549526 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 145475us
I20260709 12:18:23.550136 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.562758 28438 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:23.577757 28358 raft_consensus.cc:2410] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:23.519977 31987 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:23.582985 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:23.499620 31694 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:18:23.562600 28284 rpcz_store.cc:276] 0709 12:18:22.161936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.162081 (+   145us) service_pool.cc:224] Handling call
0709 12:18:23.526570 (+1364489us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b54003fa054. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:23.526717 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2267671296}
I20260709 12:18:23.584060 28288 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:23.518796 31527 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:23.584547 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.566856 28285 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:23.571298 28433 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:23.578078 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:23.586347 28432 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:23.586802 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.587675 28433 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:23.588778 28284 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:23.577361 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.566510 28404 leader_election.cc:403] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:23.591301 28434 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:23.591735 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.592087 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:23.592918 28288 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:23.516913 31907 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.539636 31851 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:23.600422 28433 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:23.611294 28285 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:23.544227 31992 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:23.597553 31934 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:23.617237 31788 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
W20260709 12:18:23.615939 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 85521.2us
I20260709 12:18:23.616549 28434 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:23.583076 31942 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:23.618301 28364 raft_consensus.cc:2410] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:18:23.620932 28432 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:23.617278 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.612576 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.640900 28404 leader_election.cc:400] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:23.643471 28288 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 6: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:23.643795 28404 leader_election.cc:403] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:23.582511 31784 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:23.644502 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:23.647504 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 70570.8us
I20260709 12:18:23.647949 28288 raft_consensus.cc:2410] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:23.614126 31860 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.539021 31937 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.549544 31869 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.547616 31988 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:23.658813 31988 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.660370 31937 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:23.564472 31904 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:23.636622 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:23.636725 28283 raft_consensus.cc:2410] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:23.635528 28441 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:18:23.665277 28441 pending_rounds.cc:85] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 4
I20260709 12:18:23.665604 28441 pending_rounds.cc:107] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:18:23.665978 28441 raft_consensus.cc:2889] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:23.591089 31527 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:23.642717 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:23.645592 28284 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:23.637213 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1941) took 1048 ms. Trace:
I20260709 12:18:23.646220 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:23.668488 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.667894 28361 rpcz_store.cc:276] 0709 12:18:22.588728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.503271 (+914543us) service_pool.cc:224] Handling call
0709 12:18:23.527918 (+ 24647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.528160 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:23.592822 (+ 64662us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.636284 (+ 43462us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.636784 (+   500us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:23.636834 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.637001 (+   167us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.637131 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":104287,"raft.queue_time_us":152,"raft.run_wall_time_us":1924,"spinlock_wait_cycles":9934848,"thread_start_us":70232,"threads_started":2}
I20260709 12:18:23.669660 28284 raft_consensus.cc:2410] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:23.647538 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:23.670862 28432 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:23.610932 31307 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:23.649953 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:23.671137 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:23.671916 28432 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:23.650246 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:23.647863 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:23.672876 28288 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:23.652700 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:23.656361 28433 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:23.672330 28432 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:23.675319 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:23.674266 28433 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:23.679813 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:23.674266 28288 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:18:23.653713 31912 consensus_meta.cc:287] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.975s	user 0.010s	sys 0.003s
I20260709 12:18:23.647065 31784 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.679491 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:23.683192 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 115133us
I20260709 12:18:23.674743 28285 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:23.684553 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:23.650316 31914 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:23.682138 28433 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.639569 31918 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:18:23.685880 28285 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:23.677451 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:23.660418 31988 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:23.695726 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:23.676733 28432 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.687817 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:23.688546 28433 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:23.675369 32007 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.170s	user 0.001s	sys 0.001s
I20260709 12:18:23.689755 28291 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:23.669900 31527 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.690892 28361 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:23.701000 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1942) took 1112 ms. Trace:
I20260709 12:18:23.678696 31937 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:23.678392 31917 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:23.688431 31617 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:23.694825 28285 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 12:18:23.708470 28285 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:18:23.649842 32016 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:18:23.698661 28432 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:23.690688 31845 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:23.701408 28361 rpcz_store.cc:276] 0709 12:18:22.588909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.669568 (+1080659us) service_pool.cc:224] Handling call
0709 12:18:23.687325 (+ 17757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.700778 (+ 13453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.700945 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:23.682461 28438 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:23.682983 31784 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.681620 31956 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:18:23.701172 31942 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:23.688611 32005 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:23.681767 31823 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:23.695444 31781 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:18:23.686687 31307 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.701015 31934 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:23.690644 31899 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:18:23.690793 31788 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:18:23.725445 28432 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:23.719159 31694 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:23.688175 32000 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:18:23.703537 31917 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.702843 31586 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:23.692850 31985 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:18:23.717243 32017 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:18:23.710258 28285 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [8.5-9.6]
W20260709 12:18:23.754331 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 69858.3us
I20260709 12:18:23.713085 31617 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:23.723950 31821 consensus_meta.cc:287] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.051s	user 0.003s	sys 0.012s
I20260709 12:18:23.759725 28361 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:23.729009 31860 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:23.797871 28284 raft_consensus.cc:2410] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:23.700435 31914 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:18:23.706614 31911 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.731739 32016 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:23.732020 31845 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:23.753894 28433 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:23.770785 31942 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:23.779695 32022 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:23.716166 31912 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:18:23.781378 31307 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:23.754914 31823 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.794163 28285 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:23.776592 31917 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:23.791883 31617 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:23.805138 28361 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:23.792970 31991 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:23.821471 31849 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:23.825623 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.807327 28285 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
I20260709 12:18:23.829892 28285 pending_rounds.cc:85] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 9
I20260709 12:18:23.830159 28285 pending_rounds.cc:107] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 12:18:23.830405 28285 raft_consensus.cc:2889] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:23.829169 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:18:23.828670 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 139360us
I20260709 12:18:23.833678 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:23.800259 32016 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:23.828840 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 128589us
I20260709 12:18:23.854938 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:23.856396 31527 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.858027 31845 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:23.859603 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.860093 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.860450 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.861369 31991 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:23.862171 31934 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.839033 31586 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:23.802513 31821 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.870230 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:23.876950 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:23.813175 31823 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:23.826704 31587 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.836777 31860 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:23.851783 31937 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.858716 31527 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:23.899497 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 111255us
I20260709 12:18:23.890327 31985 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:23.875063 31934 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.883961 31821 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.915321 31527 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:23.936434 31985 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:23.937522 31985 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:23.939201 32039 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:18:23.940415 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.942106 28286 raft_consensus.cc:2368] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
I20260709 12:18:23.942935 31911 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.943467 31911 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.943791 31987 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
I20260709 12:18:23.944528 28441 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:23.897063 31992 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:18:23.893353 31587 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.947319 31934 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:23.911461 32023 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:23.960527 28358 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1925) took 1377 ms (client timeout 1750 ms). Trace:
W20260709 12:18:23.961817 28358 rpcz_store.cc:269] 0709 12:18:22.583125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.583409 (+   284us) service_pool.cc:224] Handling call
0709 12:18:23.960296 (+1376887us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b5400127d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:23.960451 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258022144}
I20260709 12:18:23.963052 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:23.964092 28358 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:23.896008 31860 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.961266 32023 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:23.912009 31845 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:23.919365 32036 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:23.972967 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:23.894759 32030 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:18:23.973716 31484 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.977166 32025 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.977726 31849 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.898111 31937 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.973449 32037 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:24.007989 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 122856us
I20260709 12:18:23.979499 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:24.077634 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:24.079299 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 100294us
I20260709 12:18:24.079973 28358 raft_consensus.cc:2410] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:24.081199 31823 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.083902 31823 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:24.085703 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.087574 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.105998 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:23.986852 32005 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:24.107551 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:23.995185 31694 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:24.111646 31694 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.084239 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:24.006551 31845 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:24.127420 31845 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.130528 32023 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.131803 32023 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.134577 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 480) took 1381 ms. Trace:
I20260709 12:18:24.135062 28290 rpcz_store.cc:276] 0709 12:18:22.753258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.764660 (+ 11402us) service_pool.cc:224] Handling call
0709 12:18:22.849404 (+ 84744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.858121 (+  8717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.858154 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.858780 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:18:23.081951 (+223171us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:23.081997 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.082075 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.527440 (+445365us) raft_consensus.cc:1685] finished
0709 12:18:23.527459 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.134277 (+606818us) spinlock_profiling.cc:243] Waited 84.3 ms on lock 0x7b54001af250. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:24.134502 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8356,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":193867648,"thread_start_us":222689,"threads_started":1,"wal-append.queue_time_us":401618}
I20260709 12:18:24.137158 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:24.114745 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.033489 31849 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.089866 31989 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.972090 31527 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:24.149447 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:24.121644 32005 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.019591 31784 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.036497 32036 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:23.933633 31586 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.125176 31956 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.980286 31617 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.089181 28358 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:23.913149 31942 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.170639 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.171394 28358 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:24.172385 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:24.172923 28358 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.140187 28438 raft_consensus.cc:2410] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:24.175421 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.176559 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:24.177117 28438 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:24.140738 28290 raft_consensus.cc:2376] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11. Re-sending same reply.
I20260709 12:18:24.136639 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1976) took 1021 ms. Trace:
I20260709 12:18:24.178504 28285 rpcz_store.cc:276] 0709 12:18:23.115231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.831220 (+715989us) service_pool.cc:224] Handling call
0709 12:18:23.831493 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:23.831665 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.831680 (+    15us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:23.853374 (+ 21694us) log.cc:844] Serialized 26 byte log entry
0709 12:18:24.077159 (+223785us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:24.077206 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.077293 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.136421 (+ 59128us) raft_consensus.cc:1685] finished
0709 12:18:24.136442 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.136579 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":182223,"threads_started":1,"wal-append.queue_time_us":249084}
I20260709 12:18:24.180119 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:24.104085 32030 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:24.146342 31938 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:24.146867 32046 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:24.147575 31918 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:24.148497 31849 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:24.184111 31849 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:23.972750 32034 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:24.185246 32034 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:24.186249 32034 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.150457 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:24.150471 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 425) took 1088 ms. Trace:
I20260709 12:18:24.122174 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:24.189291 28361 rpcz_store.cc:276] 0709 12:18:23.062086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.826381 (+764295us) service_pool.cc:224] Handling call
0709 12:18:23.861309 (+ 34928us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:23.861688 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.861720 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:23.862310 (+   590us) log.cc:844] Serialized 52 byte log entry
0709 12:18:24.106732 (+244422us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:24.106780 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.106919 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.150166 (+ 43247us) raft_consensus.cc:1685] finished
0709 12:18:24.150189 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.150385 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13924,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":6215,"thread_start_us":243355,"threads_started":1,"wal-append.queue_time_us":263478}
I20260709 12:18:24.190747 28987 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.191594 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.193130 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.149907 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:24.159426 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 282543us
I20260709 12:18:24.193959 28361 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:24.194464 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:24.152942 31989 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.035946 31484 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.194849 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.195514 28361 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:24.196481 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.196746 31989 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.197772 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.197984 31788 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:24.198506 28404 leader_election.cc:400] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9.
I20260709 12:18:24.103580 31823 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.204425 28358 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:24.109637 31934 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
W20260709 12:18:24.209401 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 125318us
I20260709 12:18:24.210263 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:24.210702 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:24.211755 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:24.198737 28361 raft_consensus.cc:2410] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:24.200053 28404 leader_election.cc:400] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8.
I20260709 12:18:24.213187 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:24.213923 28290 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:24.214645 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:24.215072 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:24.216675 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:24.217355 28290 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
W20260709 12:18:24.218082 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.218493 28987 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:24.073987 32025 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.225876 28438 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:24.223647 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.226655 31985 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:24.225558 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:24.235374 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.233628 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:24.198742 31788 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:24.196915 31484 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:24.235163 28358 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:24.032784 32016 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.106423 32037 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.246896 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.244344 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.231771 32043 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:18:24.249909 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.149714 31527 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.234843 32033 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:24.275754 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.276886 28361 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:24.202050 31956 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:24.281522 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.282045 31956 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:24.282774 31956 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:24.283263 31956 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:24.285279 31988 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.005s
W20260709 12:18:24.286384 31955 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.062s	user 0.013s	sys 0.003s
I20260709 12:18:24.286906 31955 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.225244 31823 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.291325 32025 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.291625 31823 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.292238 31823 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.292358 31937 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.294242 32036 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.294426 28983 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
W20260709 12:18:24.295517 31976 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.225s	user 0.011s	sys 0.003s
I20260709 12:18:24.296041 31617 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.296427 28437 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:18:24.296766 28437 pending_rounds.cc:85] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 9
I20260709 12:18:24.296975 28437 pending_rounds.cc:107] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:18:24.297212 28437 raft_consensus.cc:2889] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:24.298559 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 540) took 1012 ms. Trace:
I20260709 12:18:24.299568 28434 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:24.247633 31985 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.292727 31823 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.302663 28288 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:24.298929 28437 rpcz_store.cc:276] 0709 12:18:23.286665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.286850 (+   185us) service_pool.cc:224] Handling call
0709 12:18:23.287191 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.297633 (+1010442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.298296 (+   663us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400184554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:24.298494 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2320114560}
W20260709 12:18:24.302748 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 193917us
I20260709 12:18:24.289491 31955 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:24.303383 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 88822.9us
I20260709 12:18:24.316843 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:24.316989 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:24.247225 32030 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:24.252100 32051 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:24.306380 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.304744 32033 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:24.302969 32048 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:24.306699 31904 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.250130 31788 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.308246 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.314594 28330 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:24.320358 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:24.297539 32037 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.324945 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.332300 28437 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:24.329429 28358 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:24.333961 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:24.321368 31989 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:18:24.335837 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:24.334782 28437 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.334075 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:24.344841 31617 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:24.349810 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.351037 31989 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:24.351660 31989 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.352787 32033 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.332259 32055 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.322827 31484 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:24.322656 31861 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:18:24.357163 31843 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:24.358049 31989 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:24.338516 31934 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.366117 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:24.349840 28437 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:24.399948 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:24.369633 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:24.400683 28437 raft_consensus.cc:2368] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11.
I20260709 12:18:24.358865 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:24.401381 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:24.326587 32048 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:24.402387 32048 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.381532 32037 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:24.321542 32025 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:24.385175 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.337047 31904 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:24.389413 28286 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1932) took 2206 ms (client timeout 2144 ms). Trace:
I20260709 12:18:24.393039 28363 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:24.421062 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1943) took 1294 ms. Trace:
I20260709 12:18:24.421444 28363 rpcz_store.cc:276] 0709 12:18:23.126408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.362294 (+1235886us) service_pool.cc:224] Handling call
0709 12:18:24.392800 (+ 30506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.420690 (+ 27890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.420987 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:24.401499 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.423440 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.322909 31860 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.321345 32051 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:24.409828 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 43776us
I20260709 12:18:24.431623 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:24.408803 31617 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.329339 32017 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:24.412385 32036 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:24.396178 32057 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.393635 31843 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:24.419872 28286 rpcz_store.cc:269] 0709 12:18:22.176893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.220084 (+ 43191us) service_pool.cc:224] Handling call
0709 12:18:24.382687 (+2162603us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540006f754. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:24.382887 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3957470080}
I20260709 12:18:24.402419 31694 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:24.387948 31587 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:24.434206 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.454190 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.452093 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:24.454886 28437 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:24.433245 32053 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.409166 31484 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.415968 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:24.445363 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44085.3us
W20260709 12:18:24.463022 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47210.4us
I20260709 12:18:24.389637 32055 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.455823 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:24.442153 31784 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.468626 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:24.469508 28437 raft_consensus.cc:2410] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:18:24.416574 31917 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.000s
I20260709 12:18:24.457811 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:24.432919 32051 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.428342 32025 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.447575 32058 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:24.464743 32053 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:24.442391 32017 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:24.394407 31956 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.469805 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:24.450497 32057 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.477249 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:24.444479 31617 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.480326 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:24.480898 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.467909 32036 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:24.460911 28404 leader_election.cc:336] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.144s (SENT)
I20260709 12:18:24.490072 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.490396 28287 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.491145 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1949) took 1503 ms. Trace:
I20260709 12:18:24.493518 28291 rpcz_store.cc:276] 0709 12:18:22.987932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.401341 (+413409us) service_pool.cc:224] Handling call
0709 12:18:24.490886 (+1089545us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54003ab954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:24.491067 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496593280}
I20260709 12:18:24.431125 32030 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.494300 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:24.494119 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.491525 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.499610 28433 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:24.499903 28437 raft_consensus.cc:2410] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:24.486802 32017 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.503469 28287 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:24.504714 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:24.502107 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:24.505076 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44200.8us
I20260709 12:18:24.505297 28291 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.505548 28432 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:24.488129 32057 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.510913 32023 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:18:24.513427 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.498541 32036 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:24.476329 31917 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.499413 31823 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.497888 32067 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.523562 32067 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.523679 31617 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.524022 32067 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.473240 31484 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.518888 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.548921 28433 raft_consensus.cc:2410] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:24.487897 31843 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:24.518376 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55222.4us
I20260709 12:18:24.552402 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:24.521939 31861 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.522794 31904 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:24.507727 32058 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:18:24.556128 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 73062.5us
I20260709 12:18:24.562604 28984 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:24.554000 32050 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.577895 32050 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.578555 32025 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:24.579054 32063 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:18:24.563580 31784 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.570195 31843 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.569262 31860 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:18:24.563815 31861 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.571120 32071 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 12:18:24.604146 32065 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:18:24.604775 32062 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.105s	user 0.001s	sys 0.001s
W20260709 12:18:24.633517 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.636709 28438 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:24.592942 31784 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.583055 32058 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:24.643992 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 91683.7us
I20260709 12:18:24.641424 31938 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.656270 28438 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:24.591486 32036 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.656865 32087 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:24.662861 28360 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1927) took 2072 ms (client timeout 1925 ms). Trace:
W20260709 12:18:24.663322 28360 rpcz_store.cc:269] 0709 12:18:22.585708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.653324 (+ 67616us) service_pool.cc:224] Handling call
0709 12:18:24.657948 (+2004624us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54003d2f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:24.658072 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666264960}
I20260709 12:18:24.667527 28360 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:24.668156 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1946) took 1536 ms. Trace:
I20260709 12:18:24.668409 28360 rpcz_store.cc:276] 0709 12:18:23.131761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.666918 (+1535157us) service_pool.cc:224] Handling call
0709 12:18:24.667323 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.667934 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.668108 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:24.670653 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.678226 31477 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.680039 31917 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.658733 32050 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:24.686107 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.662470 32087 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.697319 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1945) took 1569 ms. Trace:
I20260709 12:18:24.697933 28363 rpcz_store.cc:276] 0709 12:18:23.127591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.422364 (+1294773us) service_pool.cc:224] Handling call
0709 12:18:24.437250 (+ 14886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.437554 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.437583 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.445232 (+  7649us) log.cc:844] Serialized 26 byte log entry
0709 12:18:24.562247 (+117015us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:24.562284 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.562361 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.697027 (+134666us) raft_consensus.cc:1685] finished
0709 12:18:24.697053 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.697227 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":93147,"threads_started":1,"wal-append.queue_time_us":147723}
I20260709 12:18:24.703284 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:24.707604 31861 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.665657 31938 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:24.691135 32048 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.710399 31938 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:24.710770 32061 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:18:24.712368 32048 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:24.713605 28438 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:24.690866 31484 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:24.714780 32053 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:24.717494 32079 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:18:24.718012 32067 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.721010 31970 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.722703 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:24.722986 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:24.700182 32087 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.728417 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:24.721349 32067 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.737236 32051 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.752187 31985 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:24.754050 32058 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.754834 31784 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:24.755064 32017 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:24.756699 31845 consensus_meta.cc:287] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.000s
I20260709 12:18:24.757346 32017 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:24.757205 31845 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:24.762744 32051 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:24.767182 32057 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.769407 32087 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:24.795640 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 138861us
I20260709 12:18:24.803910 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:24.810938 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 426) took 1746 ms. Trace:
I20260709 12:18:24.811013 28363 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:24.819278 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.820987 31845 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:24.813622 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.826848 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:24.831349 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:24.833897 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:24.841382 32036 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:24.843883 32057 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:24.848390 32087 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:24.851351 28432 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:24.854768 28432 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:24.811630 28358 rpcz_store.cc:276] 0709 12:18:23.064690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.334214 (+1269524us) service_pool.cc:224] Handling call
0709 12:18:24.337238 (+  3024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.339223 (+  1985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.339254 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.382063 (+ 42809us) log.cc:844] Serialized 26 byte log entry
0709 12:18:24.589447 (+207384us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:24.589489 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.589564 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.810087 (+220523us) raft_consensus.cc:1685] finished
0709 12:18:24.810193 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.810634 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1524,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"thread_start_us":176714,"threads_started":1,"wal-append.queue_time_us":240437}
I20260709 12:18:24.858103 28363 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.857962 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:24.859531 28358 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:24.858516 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1945) took 1881 ms. Trace:
W20260709 12:18:24.859542 28283 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1957) took 1772 ms (client timeout 2034 ms). Trace:
I20260709 12:18:24.864162 28358 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:24.860807 28289 rpcz_store.cc:276] 0709 12:18:22.976734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.977978 (+  1244us) service_pool.cc:224] Handling call
0709 12:18:23.023698 (+ 45720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:23.049812 (+ 26114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.857540 (+1807728us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b54000f1020. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7fedc0 00007f85be88aa02 00007f85be886dd6 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:18:24.858265 (+   725us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11286400}
I20260709 12:18:24.866525 28432 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:24.866081 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:24.861627 28983 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:18:24.861227 28283 rpcz_store.cc:269] 0709 12:18:23.086804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.528417 (+441613us) service_pool.cc:224] Handling call
0709 12:18:24.859046 (+1330629us) spinlock_profiling.cc:243] Waited 70.2 ms on lock 0x7b54000b1dd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:24.859315 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161511296}
I20260709 12:18:24.862444 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:24.873133 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:24.878921 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.880198 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:24.882211 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:24.877916 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1934) took 2290 ms. Trace:
I20260709 12:18:24.883169 28365 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.877959 28358 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:24.882921 28367 rpcz_store.cc:276] 0709 12:18:22.587225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.075863 (+488638us) service_pool.cc:224] Handling call
0709 12:18:23.076232 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:23.146562 (+ 70330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.874441 (+1727879us) spinlock_profiling.cc:243] Waited 69.6 ms on lock 0x7b5400126e54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:24.877634 (+  3193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159994112}
I20260709 12:18:24.882573 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:24.897235 28367 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:24.896546 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 414) took 1845 ms. Trace:
I20260709 12:18:24.895481 31784 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:24.906507 28989 rpcz_store.cc:276] 0709 12:18:23.050933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.051248 (+   315us) service_pool.cc:224] Handling call
0709 12:18:23.051603 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.202112 (+150509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.895844 (+1693732us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54005bfed4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:24.896268 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344354688}
I20260709 12:18:24.908193 28367 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:24.893631 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 544) took 1237 ms. Trace:
I20260709 12:18:24.897651 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:24.912379 28434 rpcz_store.cc:276] 0709 12:18:23.656056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.656191 (+   135us) service_pool.cc:224] Handling call
0709 12:18:23.656531 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.301981 (+645450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.893026 (+591045us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b5400184054. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:24.893413 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1478384384}
I20260709 12:18:24.913797 28283 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:24.927688 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:24.931430 31914 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.931921 31914 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.932276 31914 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.936643 31956 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.937705 31956 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.952404 31778 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:24.947592 31974 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:18:24.950096 31906 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.955407 31906 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.956725 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:24.962100 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 235678us
I20260709 12:18:24.962960 31993 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:24.963627 31993 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.954984 31778 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
W20260709 12:18:24.967948 32001 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.499s	user 0.005s	sys 0.000s
I20260709 12:18:24.973135 28359 raft_consensus.cc:2410] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 12:18:24.982386 32015 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.352s	user 0.003s	sys 0.001s
W20260709 12:18:24.982563 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.984969 31867 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.985592 31867 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:24.988507 32029 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:24.989017 32029 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:24.990412 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 534) took 1797 ms. Trace:
I20260709 12:18:24.990849 28436 rpcz_store.cc:276] 0709 12:18:23.192591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.192759 (+   168us) service_pool.cc:224] Handling call
0709 12:18:23.193116 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:23.195793 (+  2677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.195823 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:23.208850 (+ 13027us) log.cc:844] Serialized 26 byte log entry
0709 12:18:23.404431 (+195581us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:23.404474 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.404547 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.971503 (+1566956us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:24.989836 (+ 18333us) raft_consensus.cc:1685] finished
0709 12:18:24.989857 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.990222 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40154,"raft.queue_time_us":13701,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":7924,"spinlock_wait_cycles":5250048,"thread_start_us":116731,"threads_started":1,"wal-append.queue_time_us":283961}
W20260709 12:18:25.001586 32010 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.533s	user 0.003s	sys 0.005s
I20260709 12:18:25.012387 28289 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:25.018620 32029 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.020298 32029 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.022925 31974 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.027181 31974 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:25.027558 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:25.028596 28440 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.032222 28332 leader_election.cc:400] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11.
I20260709 12:18:25.032791 28332 leader_election.cc:403] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:18:25.035166 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 545) took 1378 ms. Trace:
I20260709 12:18:25.035643 28441 rpcz_store.cc:276] 0709 12:18:23.656990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.667214 (+ 10224us) service_pool.cc:224] Handling call
0709 12:18:23.684281 (+ 17067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.944902 (+260621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.034625 (+1089723us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400184f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:25.034980 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597294336}
I20260709 12:18:25.037925 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:25.039477 31307 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:25.039970 31307 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:25.040033 32035 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:25.043733 31821 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:25.047533 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:25.049877 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 544) took 1826 ms. Trace:
I20260709 12:18:25.050292 28986 rpcz_store.cc:276] 0709 12:18:23.222971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.223148 (+   177us) service_pool.cc:224] Handling call
0709 12:18:23.223568 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:23.230488 (+  6920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.230521 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:23.231027 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 12:18:23.414693 (+183666us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:23.414745 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.049116 (+1634371us) spinlock_profiling.cc:243] Waited 6.47 ms on lock 0x7b540048fb54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:25.049211 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.049309 (+    98us) raft_consensus.cc:1685] finished
0709 12:18:25.049334 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.049672 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1186,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":115,"spinlock_wait_cycles":14883200,"thread_start_us":183182,"threads_started":1,"wal-append.queue_time_us":282764,"wal-append.run_cpu_time_us":1880,"wal-append.run_wall_time_us":1001874}
I20260709 12:18:25.053725 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:25.054019 31974 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:25.053769 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 427) took 1034 ms. Trace:
I20260709 12:18:25.055054 28362 rpcz_store.cc:276] 0709 12:18:24.019286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.883571 (+864285us) service_pool.cc:224] Handling call
0709 12:18:24.884202 (+   631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.884991 (+   789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.885020 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.889867 (+  4847us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.005918 (+116051us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:25.005967 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.006043 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.053352 (+ 47309us) raft_consensus.cc:1685] finished
0709 12:18:25.053373 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.053529 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":162,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":227,"thread_start_us":110025,"threads_started":1,"wal-append.queue_time_us":128692}
I20260709 12:18:25.057047 28989 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:25.060003 28984 raft_consensus.cc:1217] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Deduplicated request from leader. Original: 4.6->[5.7-5.7]   Dedup: 5.7->[]
I20260709 12:18:25.060709 28984 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:18:25.062691 31907 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:25.041119 31307 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.066721 31869 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:25.067799 31934 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.068178 28366 raft_consensus.cc:2410] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:25.068326 31934 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.070148 31307 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:25.070278 28366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1933) took 2483 ms (client timeout 1881 ms). Trace:
W20260709 12:18:25.070714 28366 rpcz_store.cc:269] 0709 12:18:22.587003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.605392 (+ 18389us) service_pool.cc:224] Handling call
0709 12:18:25.069978 (+2464586us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b54003700d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:25.070161 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5590063488}
I20260709 12:18:25.064052 31911 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:25.072446 28359 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1945) took 2958 ms (client timeout 2218 ms). Trace:
W20260709 12:18:25.072901 28359 rpcz_store.cc:269] 0709 12:18:22.113937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.114132 (+   195us) service_pool.cc:224] Handling call
0709 12:18:25.072189 (+2958057us) spinlock_profiling.cc:243] Waited 2.82 s on lock 0x7b54003bbad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:25.072369 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6476327680}
I20260709 12:18:25.076331 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:25.064098 28984 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:25.081229 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 75891.7us
I20260709 12:18:25.087849 28363 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:25.094420 28367 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:25.094264 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:25.099030 28365 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
W20260709 12:18:25.101303 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.108357 28985 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:25.108816 28366 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:25.070621 31993 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:25.111368 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 79670.3us
I20260709 12:18:25.112949 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:25.116912 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:25.117620 28289 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:25.111665 31867 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:25.134722 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 87395.6us
W20260709 12:18:25.135632 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.115041 31992 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11.
I20260709 12:18:25.141281 31960 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:25.144096 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:25.158263 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 542) took 1807 ms. Trace:
I20260709 12:18:25.158749 28439 rpcz_store.cc:276] 0709 12:18:23.359632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.370903 (+ 11271us) service_pool.cc:224] Handling call
0709 12:18:23.397710 (+ 26807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.502515 (+104805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.157707 (+1655192us) spinlock_profiling.cc:243] Waited 75.3 ms on lock 0x7b54003cb754. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:25.158079 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173225216}
I20260709 12:18:25.161799 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:25.163772 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.170183 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:25.171396 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:25.172737 28291 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.657s	user 0.018s	sys 0.005s
I20260709 12:18:25.172845 32026 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:25.173381 28291 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:25.173509 32026 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:25.175005 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:25.176395 32029 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:25.152729 31914 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:25.185773 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 49171.7us
I20260709 12:18:25.185842 28291 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:25.187008 28289 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.204979 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1961) took 2117 ms. Trace:
I20260709 12:18:25.205868 31914 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:25.206272 28292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1974) took 2091 ms (client timeout 1525 ms). Trace:
W20260709 12:18:25.206735 28292 rpcz_store.cc:269] 0709 12:18:23.114730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.159512 (+ 44782us) service_pool.cc:224] Handling call
0709 12:18:25.194294 (+2034782us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b54003da4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:25.206052 (+ 11758us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1755739008}
I20260709 12:18:25.207929 31977 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:25.208706 31860 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:25.209653 31993 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:25.208411 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 71360.3us
I20260709 12:18:25.212502 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:25.205523 28284 rpcz_store.cc:276] 0709 12:18:23.087851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.670816 (+582965us) service_pool.cc:224] Handling call
0709 12:18:25.194067 (+1523251us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54000028d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:25.204813 (+ 10746us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288087424}
I20260709 12:18:25.208376 31934 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.205068 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:25.217623 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:25.175379 32026 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.222429 32026 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.186133 32029 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.227865 32029 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:25.214789 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:25.208009 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:25.242151 28291 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.231742 31907 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.248227 28292 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.247339 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:25.251636 28361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:25.252518 31960 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:25.254426 31867 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:25.253974 31960 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:25.255837 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1948) took 2123 ms. Trace:
I20260709 12:18:25.259469 28360 rpcz_store.cc:276] 0709 12:18:23.132334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.668987 (+1536653us) service_pool.cc:224] Handling call
0709 12:18:24.671088 (+  2101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.671325 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:24.676879 (+  5554us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.157305 (+480426us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b180047b540. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be42723a
0709 12:18:25.238783 (+ 81478us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.255195 (+ 16412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.255268 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:25.255363 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.255463 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.255636 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62071,"prepare.queue_time_us":40561,"prepare.run_cpu_time_us":175,"prepare.run_wall_time_us":9751,"raft.queue_time_us":59789,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":207,"spinlock_wait_cycles":246049536,"thread_start_us":24574,"threads_started":1,"wal-append.queue_time_us":76422}
I20260709 12:18:25.252737 28361 raft_consensus.cc:606] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:25.261926 28361 raft_consensus.cc:614] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:25.262022 31907 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.262639 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41314 (request call id 40) took 1274 ms. Trace:
I20260709 12:18:25.263010 28361 rpcz_store.cc:276] 0709 12:18:23.988591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.235934 (+1247343us) service_pool.cc:224] Handling call
0709 12:18:25.262554 (+ 26620us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.253638 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:25.266032 31867 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:25.267045 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.267290 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.268538 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.269452 28358 raft_consensus.cc:2410] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:25.257670 32043 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:25.270469 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.270782 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:25.271701 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.273711 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:25.245209 31993 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:18:25.274427 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.274246 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.275169 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:25.257310 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 54036.2us
W20260709 12:18:25.274998 32011 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.806s	user 0.008s	sys 0.000s
I20260709 12:18:25.283310 28440 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:25.284320 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:25.285710 28359 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:25.285534 31993 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:25.288476 28985 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:25.290509 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.291173 31867 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:25.293154 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.293987 28359 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:25.293812 31867 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:25.294466 32055 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:25.294677 31867 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:25.295733 31798 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:25.288460 31993 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:25.296170 31798 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:25.296347 31956 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:25.294902 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.296975 31993 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:25.297945 28359 raft_consensus.cc:2410] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:25.298744 31993 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.254101 31938 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:25.298924 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.299945 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.300948 31798 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.281795 32043 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:25.303174 32016 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.303058 31971 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:25.304373 31977 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:25.305217 31867 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:18:25.304441 28364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1944) took 2177 ms (client timeout 1592 ms). Trace:
I20260709 12:18:25.305948 28330 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:25.312229 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.312548 28358 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:25.314460 28359 raft_consensus.cc:2368] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
W20260709 12:18:25.314266 28440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 532) took 2109 ms (client timeout 1855 ms). Trace:
I20260709 12:18:25.306053 31938 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:25.315027 28440 rpcz_store.cc:269] 0709 12:18:23.176635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.176796 (+   161us) service_pool.cc:224] Handling call
0709 12:18:25.027227 (+1850431us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b58001d4698. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762eee 000000000036eb39 00007f85c37d61e0 00007f85c389f0c4 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:18:25.285941 (+258714us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3284352}
I20260709 12:18:25.322207 28289 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:25.323508 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:25.313867 28364 rpcz_store.cc:269] 0709 12:18:23.126771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.194983 (+ 68212us) service_pool.cc:224] Handling call
0709 12:18:25.304040 (+2109057us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b5400127354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:25.304258 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919616768}
I20260709 12:18:25.326859 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:25.327991 31956 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:25.310698 32043 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:25.339234 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:25.339305 32046 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.021s	sys 0.020s
I20260709 12:18:25.335180 32043 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.349865 28359 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:25.331396 31938 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:25.352015 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:25.353552 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 64421.1us
I20260709 12:18:25.342862 31970 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:25.313118 31860 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:18:25.341456 31869 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:18:25.360383 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.342541 31798 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.367849 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:25.372664 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:25.364087 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.342962 32019 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:25.350168 32046 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.021s	sys 0.020s
W20260709 12:18:25.374791 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.373648 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.384500 28289 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:25.385336 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 496) took 1261 ms. Trace:
I20260709 12:18:25.385654 28289 rpcz_store.cc:276] 0709 12:18:24.123828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.346937 (+1223109us) service_pool.cc:224] Handling call
0709 12:18:25.381688 (+ 34751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.385021 (+  3333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.385252 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.386502 28286 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:25.387102 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:25.385813 28439 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:25.388949 28291 raft_consensus.cc:2410] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:25.388916 28292 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:25.394264 32024 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:25.394681 28436 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:25.395035 32024 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:25.395514 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:25.394397 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.396304 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.384564 28441 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:25.375432 31869 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:25.386193 31977 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:18:25.428370 32070 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.840s	user 0.001s	sys 0.003s
I20260709 12:18:25.430315 31934 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:25.431483 28360 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
W20260709 12:18:25.433877 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:25.434329 32104 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.258s	user 0.000s	sys 0.003s
I20260709 12:18:25.437892 31788 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.438292 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:25.438508 31788 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:25.439998 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:25.440866 31788 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:25.446933 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1956) took 1162 ms. Trace:
I20260709 12:18:25.441668 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:25.448665 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 53254.7us
I20260709 12:18:25.450325 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1951) took 1193 ms. Trace:
I20260709 12:18:25.456339 28367 rpcz_store.cc:276] 0709 12:18:24.256942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.911515 (+654573us) service_pool.cc:224] Handling call
0709 12:18:25.088673 (+177158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.096223 (+  7550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.450093 (+353870us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400079750. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:25.450244 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405679488}
I20260709 12:18:25.443326 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 493) took 1336 ms. Trace:
I20260709 12:18:25.432828 31869 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.411464 31860 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:25.445972 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.434042 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 552) took 1207 ms. Trace:
W20260709 12:18:25.447331 32106 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
W20260709 12:18:25.489396 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.530828 28438 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:25.490474 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:25.491895 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:25.454094 28287 rpcz_store.cc:276] 0709 12:18:24.248901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.358559 (+109658us) service_pool.cc:224] Handling call
0709 12:18:25.362768 (+1004209us) spinlock_profiling.cc:243] Waited 88.3 ms on lock 0x7b54000f2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:25.411526 (+ 48758us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203123968}
I20260709 12:18:25.449208 31907 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:25.537590 31869 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.540011 28285 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:25.448176 31906 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:25.446775 32108 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:25.495905 31971 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.427017 31977 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:25.447350 31956 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:25.510138 31860 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.484638 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:25.513154 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 140672us
I20260709 12:18:25.575688 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.414268 32035 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 12:18:25.421957 32019 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:25.513626 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:25.514297 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 72778.4us
I20260709 12:18:25.442157 31617 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:25.450001 32021 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:25.445853 32045 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:25.519779 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:25.522212 32024 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:25.523669 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:25.524459 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.525180 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:25.525906 32087 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:25.526793 28437 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:18:25.527212 32000 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.616s	user 0.013s	sys 0.023s
I20260709 12:18:25.527661 32055 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:25.522095 32110 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.214s	user 0.001s	sys 0.001s
I20260709 12:18:25.537668 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:25.472290 28290 rpcz_store.cc:276] 0709 12:18:24.107096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.220664 (+113568us) service_pool.cc:224] Handling call
0709 12:18:24.221050 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.371122 (+150072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.371156 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.371657 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:18:24.462010 (+ 90353us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:24.462051 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.462287 (+   236us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.427459 (+965172us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:25.442763 (+ 15304us) raft_consensus.cc:1685] finished
0709 12:18:25.442793 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.443106 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29495,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":45307,"spinlock_wait_cycles":5065088,"thread_start_us":196443,"threads_started":2,"wal-append.queue_time_us":119443}
I20260709 12:18:25.556951 28288 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.575588 28283 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:25.508767 28433 rpcz_store.cc:276] 0709 12:18:24.205135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.572000 (+366865us) service_pool.cc:224] Handling call
0709 12:18:24.589699 (+ 17699us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.590188 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.590219 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.590829 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:18:24.662193 (+ 71364us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:24.663351 (+  1158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:24.663430 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.372152 (+708722us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b2400214568. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6209c4 00007f85bb61eacb 00007f85bb7d9dc9 00007f85bb7d9a55 00007f85bb7d8fb0 00007f85be44f488 00007f85be44fbe8 00007f85be427398 00007f85be42276a 00007f85be421bb6
0709 12:18:25.376002 (+  3850us) raft_consensus.cc:1685] finished
0709 12:18:25.376023 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.412407 (+ 36384us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1298,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":3705,"spinlock_wait_cycles":21746432,"thread_start_us":70982,"threads_started":1,"wal-append.queue_time_us":82420}
I20260709 12:18:25.546592 31992 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.576478 28438 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:25.585584 28330 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:25.586889 28432 raft_consensus.cc:2376] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11. Re-sending same reply.
W20260709 12:18:25.591668 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.594718 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 494) took 1471 ms. Trace:
I20260709 12:18:25.601332 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:25.612272 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1991) took 1017 ms. Trace:
I20260709 12:18:25.618024 28283 rpcz_store.cc:276] 0709 12:18:24.594451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.484282 (+889831us) service_pool.cc:224] Handling call
0709 12:18:25.612201 (+127919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.558065 31971 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.586280 28436 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
W20260709 12:18:25.607837 32000 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.807s	user 0.013s	sys 0.024s
I20260709 12:18:25.611275 28287 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:25.616815 28286 rpcz_store.cc:276] 0709 12:18:24.123283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.271889 (+1148606us) service_pool.cc:224] Handling call
0709 12:18:25.379730 (+107841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.390218 (+ 10488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.594446 (+204228us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54000f2650. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:25.594604 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236428160}
I20260709 12:18:25.608788 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.626595 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.609529 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.621568 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:25.623505 28285 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.627269 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.624046 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.632406 28358 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:25.632556 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.604759 32021 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:25.636746 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:25.651145 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.652298 28441 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:25.653337 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.653618 28286 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:25.655282 28360 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:18:25.654464 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 74024.6us
I20260709 12:18:25.655704 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.655611 28439 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:25.659301 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:25.670756 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:25.653898 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.662595 28440 raft_consensus.cc:2410] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:25.664671 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.666494 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.654099 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.666898 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 495) took 1543 ms. Trace:
I20260709 12:18:25.667618 28286 raft_consensus.cc:606] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:25.674533 28441 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:25.678684 28359 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.654668 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.682157 28286 raft_consensus.cc:614] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:25.665948 32043 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:25.683967 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49742.2us
I20260709 12:18:25.684886 28283 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:25.686182 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.662997 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:25.683530 28433 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:25.689925 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.690820 28364 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:25.691895 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.693032 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.693953 31798 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:25.693813 31992 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.697268 28289 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:25.698426 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1960) took 1446 ms. Trace:
I20260709 12:18:25.698832 28289 rpcz_store.cc:276] 0709 12:18:24.252472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.674285 (+1421813us) service_pool.cc:224] Handling call
0709 12:18:25.696890 (+ 22605us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.697979 (+  1089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.698328 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:25.699803 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 77194.6us
I20260709 12:18:25.700563 28289 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:25.685621 32043 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:25.701377 28436 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:25.702171 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.702867 28358 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.707137 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:25.691135 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.709342 28437 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:25.710054 28438 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:25.654084 31934 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:25.686661 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:25.655787 31477 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.713088 28364 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:25.708617 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.675463 28284 rpcz_store.cc:276] 0709 12:18:24.123584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.301140 (+1177556us) service_pool.cc:224] Handling call
0709 12:18:25.301592 (+   452us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:25.323192 (+ 21600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.323222 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:25.346570 (+ 23348us) log.cc:844] Serialized 52 byte log entry
0709 12:18:25.585974 (+239404us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:25.586011 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.586076 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.666669 (+ 80593us) raft_consensus.cc:1685] finished
0709 12:18:25.666690 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.666837 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8526,"raft.queue_time_us":13427,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":18925,"thread_start_us":227778,"threads_started":1,"wal-append.queue_time_us":306910}
I20260709 12:18:25.701234 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1986) took 1370 ms. Trace:
W20260709 12:18:25.715750 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 53012.8us
I20260709 12:18:25.715502 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:25.715221 28289 rpcz_store.cc:276] 0709 12:18:24.331118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.699753 (+1368635us) service_pool.cc:224] Handling call
0709 12:18:25.700211 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.700964 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.701159 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.719287 28432 raft_consensus.cc:2410] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:18:25.721081 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.722085 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.722394 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.715448 28990 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:25.727577 28288 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:25.726166 28360 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:25.727242 28362 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:25.732936 28361 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:25.734790 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:25.735011 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 48489.9us
I20260709 12:18:25.736236 28360 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:25.737862 32029 consensus_queue.cc:1059] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:25.738276 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:25.738963 31935 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:25.738710 32029 consensus_queue.cc:1243] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:25.737638 28360 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:25.740283 32029 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:25.740754 32029 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:25.740846 32113 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:25.741446 28288 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:25.741937 32029 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.740633 31935 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:25.742197 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.747236 28365 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:25.743863 31617 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:18:25.749616 31617 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:25.744019 31907 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:18:25.744299 31849 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:18:25.743611 32029 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.744983 31935 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:25.713688 31934 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.745364 28284 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:18:25.707386 31977 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.758876 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1989) took 1391 ms. Trace:
I20260709 12:18:25.742144 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 498) took 1338 ms. Trace:
I20260709 12:18:25.745095 28283 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:25.745254 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.759268 28284 rpcz_store.cc:276] 0709 12:18:24.367911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.715027 (+1347116us) service_pool.cc:224] Handling call
0709 12:18:25.732649 (+ 17622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.758518 (+ 25869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.758773 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.739882 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:25.759455 28288 rpcz_store.cc:276] 0709 12:18:24.404012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.728395 (+1324383us) service_pool.cc:224] Handling call
0709 12:18:25.741159 (+ 12764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.741868 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.742069 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.764405 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.765625 28364 raft_consensus.cc:2410] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:25.741325 31869 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:25.775583 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1961) took 1511 ms. Trace:
I20260709 12:18:25.771410 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:25.772640 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.776031 28283 rpcz_store.cc:276] 0709 12:18:24.252747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.711895 (+1459148us) service_pool.cc:224] Handling call
0709 12:18:25.744789 (+ 32894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.763530 (+ 18741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.763769 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.764263 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:25.774168 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:25.715394 32026 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.777833 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:25.740264 32043 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:25.777829 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 328421us
I20260709 12:18:25.780238 28287 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:25.782135 28362 raft_consensus.cc:2410] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:25.758033 31934 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:25.758996 31977 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.759904 32055 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:25.711228 31993 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:18:25.808198 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:25.816418 28287 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:25.835099 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.775892 31869 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:25.778712 31867 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:25.663937 32019 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:25.663826 32021 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:25.807126 31956 raft_consensus.cc:2764] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:18:25.760288 31938 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.814976 32055 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.854025 31869 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.776839 31477 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:25.822729 32043 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:25.807813 32026 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:25.865229 28288 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:25.873008 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 503) took 1070 ms. Trace:
I20260709 12:18:25.873385 28288 rpcz_store.cc:276] 0709 12:18:24.802795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.774966 (+972171us) service_pool.cc:224] Handling call
0709 12:18:25.815551 (+ 40585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.872715 (+ 57164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.872947 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:25.862154 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 122384us
I20260709 12:18:25.884263 28433 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:25.887388 28287 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:25.861928 32055 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:25.890841 28366 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:25.880400 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:25.899437 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 153527us
I20260709 12:18:25.900588 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 504) took 1048 ms. Trace:
I20260709 12:18:25.900992 28287 rpcz_store.cc:276] 0709 12:18:24.847872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.872037 (+1024165us) service_pool.cc:224] Handling call
0709 12:18:25.887198 (+ 15161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.887841 (+   643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.895951 (+  8110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:25.902631 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1995) took 1073 ms. Trace:
I20260709 12:18:25.903071 28283 rpcz_store.cc:276] 0709 12:18:24.829061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.777554 (+948493us) service_pool.cc:224] Handling call
0709 12:18:25.850106 (+ 72552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.889366 (+ 39260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.889398 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.897645 (+  8247us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:25.902397 (+  4752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.902462 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.902570 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:25.909675 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 101562us
I20260709 12:18:25.912734 28441 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:25.888300 31914 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:25.921291 28985 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:25.894953 31860 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:25.933137 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 189762us
I20260709 12:18:25.935526 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:25.937319 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.911861 31960 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:25.912149 31971 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:25.924551 31823 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.255s	user 0.021s	sys 0.017s
I20260709 12:18:25.952771 28283 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:25.903370 32035 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:18:25.974642 28434 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:25.955966 31823 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.433s	user 0.021s	sys 0.018s
I20260709 12:18:25.979823 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:25.979105 28441 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:25.946509 31860 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:25.977141 28433 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:25.982059 28359 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
W20260709 12:18:26.000888 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 82786.1us
I20260709 12:18:25.996168 28358 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:26.004221 28283 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:26.010522 31798 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.011018 31798 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.945813 32043 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.051460 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:26.053592 28283 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:26.060541 32039 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:18:26.061602 28986 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:26.062000 32039 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:26.062418 32039 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:26.060460 28435 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:26.022495 31860 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.111109 28283 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:26.076562 31938 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.083202 31914 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:26.083714 32024 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.117780 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:26.118764 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 81468.3us
I20260709 12:18:26.128989 28285 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:26.060434 31921 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:18:26.136112 28986 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:18:26.083832 32026 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:26.105468 31970 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:26.142815 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 105036us
I20260709 12:18:26.110258 31477 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:26.104518 32000 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.129635 31906 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.165917 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:26.167084 31899 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:26.167533 31899 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:26.168367 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:26.168628 31899 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.159479 31970 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.171339 32137 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:18:26.172657 31970 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:26.174261 32128 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:26.175061 32113 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:26.175475 32129 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:26.175812 32113 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:26.177924 32133 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:26.165940 32039 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.178823 31798 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.179338 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 497) took 2055 ms. Trace:
I20260709 12:18:26.179697 28292 rpcz_store.cc:276] 0709 12:18:24.124059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.390585 (+1266526us) service_pool.cc:224] Handling call
0709 12:18:25.501467 (+110882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.578493 (+ 77026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.578528 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.617797 (+ 39269us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.870693 (+252896us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:25.870739 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.870821 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.179043 (+308222us) raft_consensus.cc:1685] finished
0709 12:18:26.179064 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.179201 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14475,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":3054,"thread_start_us":204601,"threads_started":1,"wal-append.queue_time_us":373611}
W20260709 12:18:26.181053 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.181196 31869 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.183602 31869 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.185153 31899 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.186040 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1955) took 1937 ms. Trace:
I20260709 12:18:26.186443 28291 rpcz_store.cc:276] 0709 12:18:24.248598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.412962 (+1164364us) service_pool.cc:224] Handling call
0709 12:18:25.533988 (+121026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.606683 (+ 72695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.606716 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.626067 (+ 19351us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.933553 (+307486us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:25.933643 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.933723 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.185819 (+252096us) raft_consensus.cc:1685] finished
0709 12:18:26.185839 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.185978 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10662,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3415,"thread_start_us":248141,"threads_started":1,"wal-append.queue_time_us":530263}
I20260709 12:18:26.188892 31798 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.190495 32039 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.192643 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1959) took 1940 ms. Trace:
I20260709 12:18:26.196422 31935 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.199673 31992 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.200315 31992 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.200827 31992 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.193071 28290 rpcz_store.cc:276] 0709 12:18:24.252113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.657856 (+1405743us) service_pool.cc:224] Handling call
0709 12:18:25.658221 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.658429 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:25.658817 (+   388us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.858609 (+199792us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.192081 (+333472us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800478180. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287
0709 12:18:26.192215 (+   134us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.192278 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:26.192364 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.192447 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.192577 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42005,"prepare.queue_time_us":449784,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":8151,"raft.queue_time_us":414354,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":32666,"spinlock_wait_cycles":104197248,"thread_start_us":337374,"threads_started":2,"wal-append.queue_time_us":307955}
I20260709 12:18:26.196434 31860 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.206987 31935 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.212649 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 499) took 1808 ms. Trace:
I20260709 12:18:26.213086 28284 rpcz_store.cc:276] 0709 12:18:24.404239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.762048 (+1357809us) service_pool.cc:224] Handling call
0709 12:18:25.814256 (+ 52208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.869626 (+ 55370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.869659 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.891884 (+ 22225us) log.cc:844] Serialized 26 byte log entry
0709 12:18:26.096923 (+205039us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:26.096971 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.097075 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.212431 (+115356us) raft_consensus.cc:1685] finished
0709 12:18:26.212456 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.212585 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2740,"raft.queue_time_us":197693,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":25384,"thread_start_us":152871,"threads_started":1,"wal-append.queue_time_us":285556}
I20260709 12:18:26.218948 31992 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.219600 31992 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:26.220415 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.220116 31992 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.222070 28360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:18:26.222711 28360 raft_consensus.cc:606] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:26.223029 28360 raft_consensus.cc:614] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:26.153383 31914 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.157457 32024 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.227531 28988 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:26.228685 32024 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.231262 32084 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:26.232311 32084 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:26.232030 31906 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.232663 32084 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:26.232825 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.162770 28956 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:26.233563 32084 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.235164 32129 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.234465 32021 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:26.235603 32129 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.235710 32021 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:26.155411 31849 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.164068 28283 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:26.236269 32021 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:26.246484 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.250953 28288 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:26.251091 28358 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:18:26.234737 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 72080.3us
I20260709 12:18:26.251626 28358 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:26.169658 28404 leader_election.cc:400] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
I20260709 12:18:26.259279 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.260820 28284 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:18:26.259224 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:26.260385 28436 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:26.252482 28358 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.266379 28358 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.272214 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.270516 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.271858 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:26.276674 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:26.273517 28432 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:26.248627 31798 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.276829 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.277356 28291 raft_consensus.cc:606] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
W20260709 12:18:26.280877 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 111356us
I20260709 12:18:26.282608 28291 raft_consensus.cc:614] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:26.283768 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:26.284721 28359 raft_consensus.cc:2410] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:18:26.262799 32084 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.250886 32000 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:26.245388 32029 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:26.292026 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:26.281270 31938 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:18:26.295209 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:26.299742 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:26.305342 28433 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.260456 32045 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:26.272053 31914 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:26.302100 32071 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:26.280911 32133 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.269989 31869 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:26.303581 28432 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:26.301317 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.270891 32039 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 12:18:26.308810 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 356617us
I20260709 12:18:26.310176 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.260010 31921 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:18:26.317591 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:26.308857 32117 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.286s	user 0.001s	sys 0.002s
W20260709 12:18:26.310626 32122 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.337s	user 0.000s	sys 0.002s
I20260709 12:18:26.277927 31849 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.330407 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1963) took 1271 ms. Trace:
I20260709 12:18:26.327536 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:26.331669 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.287576 31798 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.330797 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1990) took 1962 ms. Trace:
I20260709 12:18:26.327328 28332 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:26.332239 28287 rpcz_store.cc:276] 0709 12:18:25.058824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.901834 (+843010us) service_pool.cc:224] Handling call
0709 12:18:25.913623 (+ 11789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.945975 (+ 32352us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:26.038691 (+ 92716us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.328713 (+290022us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.328764 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:26.329088 (+   324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.329156 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.330337 (+  1181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8426,"prepare.queue_time_us":73432,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":1628,"raft.queue_time_us":8684,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":79,"thread_start_us":306491,"threads_started":2}
I20260709 12:18:26.312132 32000 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.338625 31934 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.306007 32016 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:18:26.340554 28289 rpcz_store.cc:276] 0709 12:18:24.368182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.718795 (+1350613us) service_pool.cc:224] Handling call
0709 12:18:25.734368 (+ 15573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.816186 (+ 81818us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:25.948276 (+132090us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.328173 (+379897us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.328235 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:26.328287 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.328364 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.330743 (+  2379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4325,"prepare.queue_time_us":184225,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":9961,"raft.queue_time_us":9342,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":418614,"threads_started":2}
I20260709 12:18:26.346599 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.349531 28358 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:18:26.351641 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.343578 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:26.326479 31914 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.314110 32029 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.332033 31477 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:26.369908 31477 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:26.370365 31477 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.371172 32021 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.365150 32134 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.355324 32132 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:26.317703 32133 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.372339 32134 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.373072 31823 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:26.367412 31798 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.360929 32113 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:26.371553 28257 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:26.377135 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.350414 31992 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.372342 32132 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.381726 31934 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.380882 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.387215 28436 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:26.385288 28440 raft_consensus.cc:2410] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:18:26.388053 28435 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
W20260709 12:18:26.388870 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 100776us
I20260709 12:18:26.382157 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:26.396477 28438 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:26.324687 32045 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.370422 31849 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:26.400034 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:26.384084 32134 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.386606 31823 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:26.403225 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.375131 32017 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.147s
I20260709 12:18:26.386508 32132 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:26.406438 32145 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:26.411474 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 163112us
I20260709 12:18:26.419486 28432 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:26.424634 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:26.426403 28436 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.426129 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:26.427784 32134 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.428328 32134 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:26.429970 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.430524 32134 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:26.429454 28255 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 1.572s (SENT) [suppressed 12 similar messages]
I20260709 12:18:26.405658 31934 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:26.402638 32021 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.423965 32145 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:26.482887 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 83158.6us
I20260709 12:18:26.483741 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:26.437821 31860 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:26.445122 31934 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.495991 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:26.524744 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.482760 32017 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:26.487625 31823 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:26.533322 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:26.485047 32145 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.496802 31992 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.523145 31914 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:26.523243 31934 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:26.556334 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 62333.2us
I20260709 12:18:26.535305 32026 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:26.530972 31798 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.563642 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:26.526800 31935 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:18:26.532981 32017 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:26.534683 31860 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:26.585201 28360 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:18:26.586468 28360 pending_rounds.cc:85] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 5
I20260709 12:18:26.591001 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:26.592569 28358 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:26.595477 28363 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:26.596976 31914 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:26.597548 28366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1956) took 1330 ms (client timeout 1709 ms). Trace:
W20260709 12:18:26.598100 28366 rpcz_store.cc:269] 0709 12:18:25.266832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.266993 (+   161us) service_pool.cc:224] Handling call
0709 12:18:26.597318 (+1330325us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54001d7754. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:26.597459 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409328512}
W20260709 12:18:26.598974 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.598129 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:26.601045 28362 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:26.586732 28360 pending_rounds.cc:107] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:18:26.603165 28360 raft_consensus.cc:2889] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:26.522738 32134 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.564649 32026 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:26.609390 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 121280us
I20260709 12:18:26.561097 32153 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:26.574322 31970 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.557137 31992 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:26.607863 32055 consensus_meta.cc:287] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.695s	user 0.010s	sys 0.002s
I20260709 12:18:26.556979 31823 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.566298 32155 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:18:26.650553 28284 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:18:26.606904 31849 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:26.660761 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 231322us
I20260709 12:18:26.585901 32129 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.663719 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:26.682464 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:26.636703 32026 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.658279 28284 pending_rounds.cc:85] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 8
I20260709 12:18:26.688330 28284 pending_rounds.cc:107] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:18:26.688671 28284 raft_consensus.cc:2889] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:26.644752 32153 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.612252 31907 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:18:26.639319 31869 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:26.597580 31914 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.599368 31798 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:26.697952 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 136955us
W20260709 12:18:26.704304 32143 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:18:26.710343 31823 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:26.712263 32156 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:18:26.713065 32154 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:18:26.713379 32165 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:26.714182 32148 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:18:26.714073 32017 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:26.714897 32152 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:18:26.657372 31970 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:26.715673 32149 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
W20260709 12:18:26.717964 32139 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 12:18:26.721684 28990 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 421) took 2572 ms (client timeout 2219 ms). Trace:
I20260709 12:18:26.647985 32134 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.701165 32133 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:26.727008 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.664440 31849 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.730440 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:26.732772 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 136286us
I20260709 12:18:26.733611 28360 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:18:26.736410 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.741004 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:26.738111 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:26.654842 32055 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.700799 32153 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:26.727437 31992 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:26.755369 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 90281.3us
I20260709 12:18:26.757545 28365 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:26.702096 32137 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
W20260709 12:18:26.760752 28990 rpcz_store.cc:269] 0709 12:18:24.149375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.149521 (+   146us) service_pool.cc:224] Handling call
0709 12:18:26.721434 (+2571913us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b540064f354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:26.721571 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3481868672}
I20260709 12:18:26.770047 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:26.754536 31778 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:26.730544 31869 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:26.777922 32150 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 12:18:26.780035 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:26.780879 28433 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:26.784461 28436 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:26.788818 28361 raft_consensus.cc:2410] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:26.789114 28367 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:26.799160 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:26.810736 28360 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:26.798126 31921 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:18:26.804812 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.815272 28286 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:26.788214 31849 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:26.806309 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.804047 28364 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:26.822294 28360 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:26.807451 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:26.823939 28283 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
W20260709 12:18:26.805343 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.728029 31860 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.825623 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 1989) took 1023 ms. Trace:
I20260709 12:18:26.826112 28367 rpcz_store.cc:276] 0709 12:18:25.802576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.730744 (+928168us) service_pool.cc:224] Handling call
0709 12:18:26.825548 (+ 94804us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:26.828862 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:26.829854 28283 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:26.829663 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:26.831264 28283 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:26.832795 32055 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.833971 32129 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.802454 32017 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.838531 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:26.828928 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:26.807243 31778 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:26.842641 31778 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.843539 31869 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:26.843747 31778 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:26.844980 32119 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.985s	user 0.001s	sys 0.004s
I20260709 12:18:26.846596 31914 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:26.847774 31906 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:26.854499 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:26.855947 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:26.848557 32084 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 12:18:26.756332 32165 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.820708 32133 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:26.872651 28436 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:26.875312 28436 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:26.850157 31798 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:26.877043 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:26.880374 32163 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:18:26.886006 32174 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:26.886694 32174 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:26.887213 32019 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:26.889775 31914 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:26.895133 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:26.895877 28292 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
W20260709 12:18:26.897994 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 76233.1us
I20260709 12:18:26.898919 28361 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:26.876173 28436 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.863348 31974 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:26.901539 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:26.891888 31914 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.919725 28436 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.923520 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:26.925676 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2216) took 1320 ms. Trace:
I20260709 12:18:26.928293 28292 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:26.887683 32019 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:26.927328 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:26.931466 28361 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:26.905512 31869 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:26.887643 32174 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.927345 28439 rpcz_store.cc:276] 0709 12:18:25.605547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.778593 (+173046us) service_pool.cc:224] Handling call
0709 12:18:25.825370 (+ 46777us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:25.968271 (+142901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.968340 (+    69us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:26.056180 (+ 87840us) log.cc:844] Serialized 582 byte log entry
0709 12:18:26.311841 (+255661us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:26.311881 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.311948 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.925429 (+613481us) raft_consensus.cc:1685] finished
0709 12:18:26.925451 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.925610 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5226,"raft.queue_time_us":8189,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":8417,"thread_start_us":302187,"threads_started":2,"wal-append.queue_time_us":391638}
I20260709 12:18:26.944384 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:26.949070 32159 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:26.954712 32181 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:26.955451 32181 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:26.956357 31914 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:26.957068 31477 consensus_meta.cc:287] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.002s
I20260709 12:18:26.960075 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:26.961076 28292 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:26.959991 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:26.934571 31974 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:26.962138 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:26.901345 32133 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:26.972649 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:26.989566 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.987246 28283 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:26.957540 31477 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:26.989420 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.000339 28292 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:27.001574 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.006910 32179 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:27.007572 32179 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.007957 32179 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:27.011194 32180 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.011901 32180 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.007761 31960 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.014117 31960 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.014633 28439 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:27.015038 28439 pending_rounds.cc:85] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 4
I20260709 12:18:27.015293 28439 pending_rounds.cc:107] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:27.007139 31869 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:27.015532 28439 raft_consensus.cc:2889] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:27.006781 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.016364 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.044126 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:27.045791 28437 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:27.049708 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:26.989573 32132 consensus_meta.cc:287] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.003s
I20260709 12:18:27.055698 28367 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:18:27.058282 28436 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:27.058667 28436 pending_rounds.cc:85] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
I20260709 12:18:27.058851 28436 pending_rounds.cc:107] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:27.059023 28436 raft_consensus.cc:2889] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:26.987754 31798 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.961887 32181 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:27.077224 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:27.078720 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.071609 31798 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.085142 31869 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.086975 32180 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.087543 31798 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.088792 31960 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.090914 32182 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.091202 32133 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:27.091674 32178 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:18:27.092274 32165 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:27.092025 31974 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.091617 32182 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.070286 31477 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.102641 32182 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:27.103507 32145 consensus_meta.cc:287] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.000s
I20260709 12:18:27.103986 32145 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.105856 32134 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.074451 32019 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:27.110842 32019 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:27.112514 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.113947 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.115716 32170 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:27.116442 32170 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:27.118911 32124 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.148s	user 0.002s	sys 0.000s
I20260709 12:18:27.119083 32170 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:27.119633 32133 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.119771 32170 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:27.120715 32171 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.120399 32170 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:27.121243 32171 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.123178 32026 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.123742 32171 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:27.065824 32132 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.146826 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.160168 32132 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.150352 28289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:27.152570 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:27.161145 28289 raft_consensus.cc:606] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:27.161558 28289 raft_consensus.cc:614] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:27.161921 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:27.162766 28435 raft_consensus.cc:2410] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:18:27.154268 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:27.163188 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.163420 28362 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:27.164520 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:27.164767 32153 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:18:27.156397 31934 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.001s
I20260709 12:18:27.120232 32133 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:27.166003 31934 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.153602 32026 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:27.158044 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.170660 28989 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:27.172765 28435 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:27.152446 32170 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:27.174574 32133 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:27.175266 32179 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:27.178100 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.179332 28989 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:27.181562 32179 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:27.169714 31934 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.169684 32145 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.175841 31970 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.162892 28289 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:27.185465 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.186127 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.186180 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.186436 28289 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:27.187054 28990 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:27.188355 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.189144 28988 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:27.191648 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:27.192859 32017 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:27.193420 32017 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.193881 32000 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:27.187868 28289 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:27.197429 31849 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.198977 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.199832 31849 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:27.200754 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 191033us
I20260709 12:18:27.200358 28361 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:27.203435 28361 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:27.206204 28361 pending_rounds.cc:85] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 5
I20260709 12:18:27.206940 28361 pending_rounds.cc:107] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:27.207610 28361 raft_consensus.cc:2889] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:27.222150 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 226165us
I20260709 12:18:27.236810 32000 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:27.193895 32017 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:27.237612 32000 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.240027 32000 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:27.251442 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.255177 31617 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.255807 31617 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.258368 31617 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.258980 31617 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:27.260505 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:27.262732 31823 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:27.278069 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 367388us
W20260709 12:18:27.288583 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.284201 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.290338 32000 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.291147 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:27.290977 32000 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.293969 32000 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:27.298401 31823 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:27.304414 32000 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.305299 32000 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.307880 32000 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:27.309896 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:27.310561 28434 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:27.310988 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:27.311930 32000 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:27.312321 32000 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.312644 32000 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:27.328181 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 67776us
I20260709 12:18:27.330252 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.331025 28289 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:27.382077 32153 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.479030 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:27.480684 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 158248us
I20260709 12:18:27.481621 32170 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:27.481945 31914 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:27.482756 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:27.483407 32170 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:27.483831 32170 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:27.484544 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:27.484097 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:27.485558 28436 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:27.530862 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:27.531646 28436 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:27.532666 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:27.533330 28436 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
W20260709 12:18:27.486189 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.534574 28956 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:27.530961 28287 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:27.535686 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:27.491194 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.491214 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:27.538213 28435 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:27.538300 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:27.539232 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:27.539669 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.496320 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.538589 32133 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.540712 28435 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:27.540997 32133 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.541687 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:27.542338 28435 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.543339 32133 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:27.545382 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.496666 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.546813 32179 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:27.547385 32179 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.548709 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:27.497766 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.539505 32120 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:27.539114 32019 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:27.498308 28438 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:27.551987 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:27.552337 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:27.498906 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:27.553020 28441 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
W20260709 12:18:27.553635 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 54887us
I20260709 12:18:27.554373 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:27.554987 28441 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.556313 31907 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:27.556813 31907 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.557196 31907 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:27.557981 32153 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:27.560158 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:27.561851 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.564292 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:27.565286 28436 raft_consensus.cc:2410] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:27.565711 31934 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:27.566339 31934 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:27.569334 31934 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:27.499274 32126 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.529s	user 0.002s	sys 0.000s
I20260709 12:18:27.488420 28434 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:27.484963 32170 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.512086 32000 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.514564 32017 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:27.590505 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.517011 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.522305 32026 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.523669 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.506286 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:27.572815 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:27.539494 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.592504 28989 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:27.593333 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:27.591910 28984 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:27.573558 28438 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:27.566258 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:27.597014 32179 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.584338 31823 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:27.598232 32017 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:27.600131 32000 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:27.588229 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:27.589156 32170 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:27.516732 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.594060 28283 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:27.579077 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.815953 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:27.622308 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.595949 28436 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:27.641762 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 81652.4us
I20260709 12:18:27.630587 32174 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:18:27.691298 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:27.625113 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:27.824509 28438 raft_consensus.cc:2410] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:27.698237 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 512) took 1054 ms. Trace:
I20260709 12:18:27.827273 28291 rpcz_store.cc:276] 0709 12:18:26.643782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.648665 (+  4883us) service_pool.cc:224] Handling call
0709 12:18:26.684349 (+ 35684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.691394 (+  7045us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:26.727867 (+ 36473us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.946018 (+218151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.946071 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:26.946116 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.689526 (+743410us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b540035fcd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:27.689581 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.698152 (+  8571us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36450,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":3330,"raft.queue_time_us":1939,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":33609,"spinlock_wait_cycles":15536896,"thread_start_us":198605,"threads_started":1,"wal-append.queue_time_us":344253}
I20260709 12:18:27.717291 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:27.734088 28285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.009s	sys 0.011s
W20260709 12:18:27.830848 28285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.009s	sys 0.011s
I20260709 12:18:27.831631 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 515) took 1029 ms. Trace:
I20260709 12:18:27.832056 28285 rpcz_store.cc:276] 0709 12:18:26.802224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.802473 (+   249us) service_pool.cc:224] Handling call
0709 12:18:26.802846 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.815051 (+ 12205us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:26.815083 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:26.850660 (+ 35577us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.831271 (+980611us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:27.831319 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.831392 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.831435 (+    43us) raft_consensus.cc:1685] finished
0709 12:18:27.831447 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:27.831570 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16502,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":31989,"thread_start_us":980197,"threads_started":1,"wal-append.queue_time_us":249694}
I20260709 12:18:27.820792 32194 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:18:27.829576 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.746353 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.835258 28292 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:27.627889 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:27.602011 32026 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.983824 32026 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:27.630935 31934 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.135s
I20260709 12:18:27.985526 31934 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:27.986001 31934 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.986351 31934 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:27.771916 32179 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:27.991799 32179 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:27.782588 32045 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:27.654945 32175 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:27.797812 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.996429 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.996199 32175 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:27.800765 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:27.997699 28358 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:27.998281 28365 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
W20260709 12:18:27.801961 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.806185 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:27.999588 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.000646 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.001291 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:27.820271 31970 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.018337 31970 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.019052 32153 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:28.020658 31970 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:27.822387 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.019066 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:28.022147 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 199855us
I20260709 12:18:28.022573 28365 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:28.022982 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.023551 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.024257 28365 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:28.025177 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:27.831467 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:27.735276 28364 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.011s	sys 0.024s
W20260709 12:18:28.026068 28364 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.107s	user 0.011s	sys 0.024s
I20260709 12:18:28.026882 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1996) took 1302 ms. Trace:
I20260709 12:18:28.027266 28364 rpcz_store.cc:276] 0709 12:18:26.723998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.822674 (+ 98676us) service_pool.cc:224] Handling call
0709 12:18:26.823009 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.823377 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.823405 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:26.863906 (+ 40501us) log.cc:844] Serialized 26 byte log entry
0709 12:18:28.026493 (+1162587us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:28.026539 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.026614 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.026661 (+    47us) raft_consensus.cc:1685] finished
0709 12:18:28.026672 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.026819 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60647,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":1107039,"threads_started":1,"wal-append.queue_time_us":152885}
I20260709 12:18:27.738855 28433 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:28.031508 28364 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:28.034242 32153 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:27.835429 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.035231 32153 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.036408 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.037096 28291 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:27.836351 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.039609 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.040304 28285 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:27.839215 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.843789 32194 consensus_queue.cc:1059] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:27.850534 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.042860 32194 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:28.043025 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 192569us
I20260709 12:18:28.039860 31934 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:27.857479 28283 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:28.044641 28330 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.044801 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.046075 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.046208 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:27.871891 28435 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
W20260709 12:18:28.046449 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.885044 31988 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.894117 32129 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:27.908171 32000 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:18:27.909273 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 93867.7us
I20260709 12:18:27.914247 31823 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:27.919790 28436 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:27.937036 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:27.939316 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2229) took 1627 ms. Trace:
I20260709 12:18:28.052773 28440 rpcz_store.cc:276] 0709 12:18:26.317060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.317185 (+   125us) service_pool.cc:224] Handling call
0709 12:18:27.939035 (+1621850us) spinlock_profiling.cc:243] Waited 52.7 ms on lock 0x7b54002e40d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:27.939218 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121219840}
I20260709 12:18:27.940668 28437 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:18:27.929517 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:27.955529 32029 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:27.973625 32046 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.974751 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1993) took 3307 ms. Trace:
I20260709 12:18:28.056687 28286 rpcz_store.cc:276] 0709 12:18:24.667503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.762276 (+1094773us) service_pool.cc:224] Handling call
0709 12:18:25.848443 (+ 86167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.815724 (+967281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.974392 (+1158668us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54003d9fd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:27.974649 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222919424}
W20260709 12:18:27.976624 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.819473 28289 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:28.001993 28358 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:28.003623 32175 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:28.006377 32179 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:27.977555 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 349743us
W20260709 12:18:28.053573 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 116719us
I20260709 12:18:28.062351 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:28.010149 28439 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
W20260709 12:18:28.012428 28284 consensus_meta.cc:287] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.203s	user 0.010s	sys 0.000s
I20260709 12:18:28.014051 32145 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:18:28.017208 31977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.011s	sys 0.002s
I20260709 12:18:28.041658 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.038013 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.048877 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.027129 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.046890 28283 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:28.070251 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.070927 28283 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:28.053164 31988 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.065696 32046 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.063529 28439 pending_rounds.cc:85] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 7
I20260709 12:18:28.063834 28284 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:28.064257 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:28.054541 31992 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.065092 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.049999 32194 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.076464 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 1998) took 1296 ms. Trace:
I20260709 12:18:28.084868 28284 rpcz_store.cc:276] 0709 12:18:26.779616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.779769 (+   153us) service_pool.cc:224] Handling call
0709 12:18:28.076394 (+1296625us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:28.075289 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.075758 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.085865 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.075847 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.087023 28283 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:28.087208 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 1976) took 1182 ms. Trace:
I20260709 12:18:28.087809 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.088511 28284 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:28.076201 28439 pending_rounds.cc:107] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:18:28.090073 28439 raft_consensus.cc:2889] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:28.089570 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.091085 28284 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:28.093258 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.078938 28285 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:28.079643 32209 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.093073 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.086649 28286 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:28.077992 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:28.097055 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:28.106145 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.094671 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.095276 28289 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:18:28.074683 31977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.013s	sys 0.003s
I20260709 12:18:28.098660 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:28.104851 28358 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:28.087646 28362 rpcz_store.cc:276] 0709 12:18:26.904912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.165521 (+260609us) service_pool.cc:224] Handling call
0709 12:18:27.168040 (+  2519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.544932 (+376892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.544962 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.545508 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.641576 (+ 96068us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:27.641619 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.641694 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.086977 (+445283us) raft_consensus.cc:1685] finished
0709 12:18:28.086999 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.087145 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":369561,"raft.queue_time_us":369679,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":95614,"threads_started":1,"wal-append.queue_time_us":498471}
I20260709 12:18:28.094533 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.107421 28285 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:28.108162 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:28.068219 31960 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:28.118184 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.119652 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.105796 32194 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:18:28.121578 32000 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:28.080312 31992 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:28.125571 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 98574.4us
I20260709 12:18:28.127446 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.095679 32174 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:28.123042 32120 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.130321 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.130807 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.073699 31988 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.131268 28284 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:28.097198 32179 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:28.107600 31823 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:28.139050 28291 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:28.133852 28283 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:28.135142 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:28.140872 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:28.142333 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.143065 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:28.130326 31992 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.163520 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.131987 32046 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.144454 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.150687 31934 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.153110 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:28.154261 31823 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:28.155089 32174 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:28.145320 31988 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:28.166524 31934 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.156057 32029 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:28.132983 32129 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:18:28.133566 32120 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.157200 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.137571 32179 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.177336 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:28.236771 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 59515.5us
I20260709 12:18:28.141485 31970 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.237890 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.240826 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:28.242609 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2020) took 1042 ms. Trace:
I20260709 12:18:28.243860 28286 rpcz_store.cc:276] 0709 12:18:27.199910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.180026 (+980116us) service_pool.cc:224] Handling call
0709 12:18:28.242516 (+ 62490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:28.180639 32046 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:28.232592 32133 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.246455 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:28.236796 32120 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.233925 31934 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:28.235561 32174 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:28.254463 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.239545 32179 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.255934 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.257431 28291 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.243754 32024 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.256256 28286 raft_consensus.cc:2410] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:28.239427 31970 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.251655 32194 consensus_queue.cc:1243] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:28.239923 31988 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:28.245575 32045 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.248860 32046 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:28.247344 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 127789us
I20260709 12:18:28.248353 32000 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.249119 32133 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.252223 32026 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:28.266773 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.257154 31934 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.262045 31970 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.284830 28286 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:28.285297 28330 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.285619 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2007) took 1435 ms. Trace:
I20260709 12:18:28.287184 28286 rpcz_store.cc:276] 0709 12:18:26.850550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.264564 (+1414014us) service_pool.cc:224] Handling call
0709 12:18:28.280826 (+ 16262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:28.285302 (+  4476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.285548 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:28.288601 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 161242us
I20260709 12:18:28.290158 28255 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.290329 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.292680 28291 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:28.276731 32194 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.264722 32045 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:28.273591 32133 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.268992 32000 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.268769 32046 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:28.295711 32206 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:18:28.303207 28286 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:28.304725 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:28.305860 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 53948.8us
I20260709 12:18:28.299719 32045 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:28.303018 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 194945us
I20260709 12:18:28.315883 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2009) took 1464 ms. Trace:
I20260709 12:18:28.264403 32024 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.297454 32194 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.286672 32029 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:18:28.321990 28286 rpcz_store.cc:276] 0709 12:18:26.851090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.288104 (+1437014us) service_pool.cc:224] Handling call
0709 12:18:28.303001 (+ 14897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:28.315546 (+ 12545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.315788 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:28.322319 28332 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:28.314522 32129 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.303642 32000 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:28.337194 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 85900us
I20260709 12:18:28.326730 32194 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.314793 32026 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:28.323284 32213 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:28.321911 31960 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:28.315464 31823 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:28.350186 31977 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.320717 32145 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 12:18:28.346524 32210 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:18:28.340816 32129 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.369874 31977 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.344384 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.392349 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2001) took 1087 ms. Trace:
I20260709 12:18:28.394325 28440 rpcz_store.cc:276] 0709 12:18:27.304628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.056286 (+751658us) service_pool.cc:224] Handling call
0709 12:18:28.059524 (+  3238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:28.077942 (+ 18418us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:28.258082 (+180140us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.391936 (+133854us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:28.391982 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:28.392036 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.392105 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.392258 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34560,"prepare.queue_time_us":71813,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"thread_start_us":235934,"threads_started":2}
I20260709 12:18:28.351874 32213 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:28.399273 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 97223.6us
W20260709 12:18:28.394523 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 72292us
I20260709 12:18:28.401484 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.392768 32129 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.403474 32228 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:28.344661 32218 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:18:28.404083 32228 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:28.405052 32228 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.397436 32216 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:28.398000 32221 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.406701 32221 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:28.407194 32216 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:28.407620 32216 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:28.407150 32221 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.414114 32220 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.414834 32220 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.393221 32211 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.416890 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 569) took 1271 ms. Trace:
I20260709 12:18:28.327250 31988 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.417320 28438 rpcz_store.cc:276] 0709 12:18:27.145244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.825228 (+679984us) service_pool.cc:224] Handling call
0709 12:18:27.825571 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:27.825993 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.826020 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:27.826643 (+   623us) log.cc:844] Serialized 52 byte log entry
0709 12:18:27.838516 (+ 11873us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:27.838561 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.838627 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.416626 (+577999us) raft_consensus.cc:1685] finished
0709 12:18:28.416649 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.416819 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4222,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"thread_start_us":11426,"threads_started":1,"wal-append.queue_time_us":249676}
I20260709 12:18:28.414273 32229 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.399334 32177 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:18:28.419687 32229 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.420181 32177 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:28.420548 32177 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:28.359583 32026 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.416922 32211 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:28.422308 32211 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.376192 32153 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:18:28.320964 32174 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.414701 31960 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.401345 28332 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:28.466105 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 64950.6us
W20260709 12:18:28.449844 32212 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.114s	user 0.002s	sys 0.000s
I20260709 12:18:28.449265 32153 consensus_queue.cc:1243] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:18:28.451501 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49227.4us
I20260709 12:18:28.463619 31860 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.421576 32177 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.490895 32026 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:28.494088 32024 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.495230 32220 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:28.495188 32194 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.422475 32213 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:28.500772 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.486789 31977 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:28.540328 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:28.450923 32217 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:18:28.539376 32026 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:28.541676 32213 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.542470 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 511) took 1899 ms. Trace:
I20260709 12:18:28.520689 31860 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:28.560050 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.556082 28290 rpcz_store.cc:276] 0709 12:18:26.643506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.643637 (+   131us) service_pool.cc:224] Handling call
0709 12:18:26.666603 (+ 22966us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.698446 (+ 31843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.698482 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:26.699063 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:18:28.081316 (+1382253us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b1800149860. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:18:28.252840 (+171524us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:28.369693 (+116853us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.375233 (+  5540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.377104 (+  1871us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.542112 (+165008us) raft_consensus.cc:1685] finished
0709 12:18:28.542233 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.542400 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":208493,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":110716,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":12321,"spinlock_wait_cycles":7747968,"thread_start_us":264759,"threads_started":2,"wal-append.queue_time_us":1552325}
W20260709 12:18:28.562268 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 246478us
I20260709 12:18:28.565243 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:28.539747 32229 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:28.575193 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 231679us
I20260709 12:18:28.576283 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:28.576433 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:28.556865 32026 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.578608 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:28.560036 32177 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.560483 31860 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:28.579952 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 100966us
I20260709 12:18:28.585559 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:28.408432 32216 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.587384 32225 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.588127 32216 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.589910 32024 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:28.580133 32223 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.591863 32223 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:28.592293 32223 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.588387 32220 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.595958 32000 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.598762 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2000) took 1294 ms. Trace:
I20260709 12:18:28.599164 28436 rpcz_store.cc:276] 0709 12:18:27.304275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.057058 (+752783us) service_pool.cc:224] Handling call
0709 12:18:28.069351 (+ 12293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:28.069849 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:28.112699 (+ 42850us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.322804 (+210105us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:28.338280 (+ 15476us) log.cc:844] Serialized 26 byte log entry
0709 12:18:28.339524 (+  1244us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:28.339564 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.339638 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.598522 (+258884us) raft_consensus.cc:1685] finished
0709 12:18:28.598543 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.598693 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":974,"prepare.queue_time_us":55377,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":3022,"raft.queue_time_us":37940,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":3045,"thread_start_us":182387,"threads_started":2,"wal-append.queue_time_us":198411}
I20260709 12:18:28.603121 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 520) took 1431 ms. Trace:
I20260709 12:18:28.603528 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.575889 32232 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:28.605679 31907 consensus_meta.cc:287] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.048s	user 0.019s	sys 0.010s
I20260709 12:18:28.580890 32145 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:28.592103 32045 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.608119 32194 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.608829 32045 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:28.582314 32026 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:28.606213 31907 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.613065 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.615302 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.616482 28361 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:28.615680 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.617888 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.618491 28365 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:28.619861 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.619961 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.620519 28365 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:28.621887 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.622550 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.603675 28286 rpcz_store.cc:276] 0709 12:18:27.172041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.326854 (+1154813us) service_pool.cc:224] Handling call
0709 12:18:28.343172 (+ 16318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:28.470000 (+126828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:28.470031 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:28.474052 (+  4021us) log.cc:844] Serialized 26 byte log entry
0709 12:18:28.586582 (+112530us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:28.586616 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.602851 (+ 16235us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.602898 (+    47us) raft_consensus.cc:1685] finished
0709 12:18:28.602912 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.603056 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7050,"raft.queue_time_us":103505,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":2898,"thread_start_us":212476,"threads_started":2,"wal-append.queue_time_us":112547}
I20260709 12:18:28.623827 28361 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:28.624855 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.635995 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.635283 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.638164 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.650003 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:28.616106 32194 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.654196 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.655936 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:28.666267 28986 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:28.666759 28985 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:28.617256 32226 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.657339 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.657609 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:28.652405 28292 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:28.662206 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.657907 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:28.673355 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.677238 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.692444 32229 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:28.692162 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45821.7us
I20260709 12:18:28.694082 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.588195 32225 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:18:28.694101 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.635025 32045 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:28.702368 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:28.691452 28290 raft_consensus.cc:2376] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11. Re-sending same reply.
I20260709 12:18:28.643586 31977 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:28.710354 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:28.656802 31907 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.694977 32229 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:28.651329 32220 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:28.667205 28388 debug-util.cc:398] Leaking SignalData structure 0x7b080016a4c0 after lost signal to thread 27119
I20260709 12:18:28.619419 32179 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.716002 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:28.718299 28983 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:28.651917 32239 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.723466 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.669168 32000 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:28.655378 32237 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:28.729192 28365 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:28.732156 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.737358 28290 raft_consensus.cc:2376] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9. Re-sending same reply.
I20260709 12:18:28.721298 32024 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:28.718020 31977 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:28.702726 32194 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.758143 28367 raft_consensus.cc:2410] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:28.757087 32174 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Committing config change with OpId 9.7: config changed from index 2 to 7, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:28.746449 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:28.746704 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.759852 32230 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.748250 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.748822 32000 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.730177 32220 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.739334 32237 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:28.761108 32237 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.739234 32239 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.763170 32239 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:28.763778 32239 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:28.765918 32000 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.767102 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.768287 32000 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:28.751448 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.768875 32024 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:28.769369 32024 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.769752 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:28.771075 32118 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 2.846s	user 0.005s	sys 0.004s
I20260709 12:18:28.771708 32220 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:28.760635 31934 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.776957 31934 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.760172 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.755328 32223 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.721665 32045 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.779127 32211 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.780254 32045 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:28.756938 31977 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.779189 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.785144 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:28.785411 28366 raft_consensus.cc:2376] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8. Re-sending same reply.
W20260709 12:18:28.785040 28388 debug-util.cc:398] Leaking SignalData structure 0x7b0800060880 after lost signal to thread 28188
I20260709 12:18:28.786458 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.786688 32226 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:28.787002 28388 debug-util.cc:398] Leaking SignalData structure 0x7b08000ec360 after lost signal to thread 28224
I20260709 12:18:28.787680 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.788466 28366 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:28.789249 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.789098 32129 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.729678 31992 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.660625 32046 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.803328 31992 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.803628 28986 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9.
I20260709 12:18:28.792052 31823 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:28.804287 31823 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:28.805377 31823 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.806371 31823 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:28.807227 28983 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:28.803341 32046 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.755986 32213 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:28.808390 32213 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.808631 28358 raft_consensus.cc:2410] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:28.810182 28987 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:18:28.811743 32194 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.817013 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:28.770992 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.793329 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.818420 28285 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.819653 32129 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.795233 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:28.820955 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.821125 28434 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
W20260709 12:18:28.794302 28287 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 510) took 2151 ms (client timeout 1608 ms). Trace:
I20260709 12:18:28.795768 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 1997) took 3821 ms. Trace:
W20260709 12:18:28.821992 28287 rpcz_store.cc:269] 0709 12:18:26.643181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.648994 (+  5813us) service_pool.cc:224] Handling call
0709 12:18:28.794018 (+2145024us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b540009e7d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:28.794177 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948818176}
I20260709 12:18:28.822211 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:28.823262 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.818104 28290 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:28.823832 28287 raft_consensus.cc:2376] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11. Re-sending same reply.
I20260709 12:18:28.824769 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.795591 31970 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.828147 31970 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.829057 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:28.781494 32211 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.723213 32120 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.831801 32120 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:28.817566 28360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.958s	user 0.000s	sys 0.011s
W20260709 12:18:28.832608 28360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.977s	user 0.001s	sys 0.011s
I20260709 12:18:28.818091 32226 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:28.796672 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:28.833988 32226 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:28.834414 28437 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:28.835556 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:28.836930 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:28.829002 28362 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:28.837571 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:28.823264 28434 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:28.822525 28288 rpcz_store.cc:276] 0709 12:18:24.974710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.874159 (+899449us) service_pool.cc:224] Handling call
0709 12:18:26.129920 (+255761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.251406 (+121486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.973679 (+1722273us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54003f9b54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:28.795543 (+821864us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54003f9b50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:28.795702 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836621312}
I20260709 12:18:28.840503 28332 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:28.768415 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.799930 32221 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.821583 28283 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.855742 28285 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:28.843367 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.760778 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.844129 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:28.857118 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 219029us
I20260709 12:18:28.846674 28330 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.792395 31977 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.849519 28366 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:28.800732 32133 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.842644 28363 raft_consensus.cc:2376] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12. Re-sending same reply.
I20260709 12:18:28.843245 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.864722 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.864935 28287 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:28.843159 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.857069 28367 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.856520 28289 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:28.871119 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:28.874608 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:28.875358 28437 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.877521 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.878346 28362 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:28.879526 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.880342 28362 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:28.855181 32213 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.887897 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.888490 28288 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:28.888413 28283 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:28.884317 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:28.889784 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.890399 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:28.891551 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:18:28.863478 32170 consensus_meta.cc:287] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.262s	user 0.015s	sys 0.001s
I20260709 12:18:28.834441 32226 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.844998 32223 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:28.886866 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:28.854620 32230 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:28.858952 32043 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:28.909585 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:28.868125 31970 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:28.910620 28363 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:28.910796 28283 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.913512 28288 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:28.916823 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:28.902644 32129 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.918532 32129 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.918851 32024 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.920886 32129 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:28.920961 32024 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:28.922048 31974 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:28.922477 31974 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:28.922854 31974 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:28.873040 32216 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:28.916823 28441 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.361s	user 0.006s	sys 0.008s
I20260709 12:18:28.924764 28441 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:28.925499 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2249) took 1696 ms. Trace:
I20260709 12:18:28.922933 28987 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:28.925822 28441 rpcz_store.cc:276] 0709 12:18:27.229518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.554120 (+324602us) service_pool.cc:224] Handling call
0709 12:18:28.925429 (+1371309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:28.927083 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:28.927382 28367 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:28.927763 28441 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:28.928324 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:28.929278 28989 consensus_meta.cc:287] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 1.335s	user 0.012s	sys 0.000s
I20260709 12:18:28.929860 28989 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:28.931010 28287 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:28.932780 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.933476 28287 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:28.935117 28989 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 558) took 1358 ms (client timeout 1782 ms). Trace:
W20260709 12:18:28.935806 28989 rpcz_store.cc:269] 0709 12:18:27.572247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.572450 (+   203us) service_pool.cc:224] Handling call
0709 12:18:28.930464 (+1358014us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:28.936926 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.937134 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:28.888397 32194 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:28.921994 32170 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:28.877102 32029 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:18:28.893366 32000 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:28.882807 32228 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:18:28.940433 28367 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:28.882542 31992 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:18:28.907428 32229 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:28.915678 31970 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:28.937860 28288 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:28.877962 32243 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:18:28.939569 31960 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:28.908547 32026 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.955250 28441 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
W20260709 12:18:28.958235 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.958431 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.969094 28288 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:28.967907 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:28.971834 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:28.972855 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:28.963641 28283 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:28.945134 32170 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:28.978042 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:28.986492 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:28.989042 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.992993 28367 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:28.932649 32230 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.953934 31970 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:28.986505 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:29.001906 28441 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:29.002918 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:29.003666 28441 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:28.996685 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:28.942348 32237 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:28.970654 31960 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:28.986107 32024 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:29.004889 28283 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:28.986837 32120 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:28.987357 32170 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.010762 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:29.012527 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 121041us
I20260709 12:18:29.012936 28441 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:18:29.020321 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:29.014654 28288 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:29.022068 28361 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:29.023715 32024 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:29.024326 28283 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:29.024686 32029 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.037362 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 522) took 1864 ms. Trace:
I20260709 12:18:29.037489 32029 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.037789 28283 rpcz_store.cc:276] 0709 12:18:27.172608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.023602 (+1850994us) service_pool.cc:224] Handling call
0709 12:18:29.023950 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.037053 (+ 13103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.037282 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.038391 32024 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.004149 31477 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:29.039793 32029 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:29.043313 28364 raft_consensus.cc:2410] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:29.029282 28287 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:29.040198 31477 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:29.025563 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2005) took 1686 ms. Trace:
I20260709 12:18:29.039579 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 524) took 1711 ms. Trace:
I20260709 12:18:29.044071 28441 rpcz_store.cc:276] 0709 12:18:27.334580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.011447 (+1676867us) service_pool.cc:224] Handling call
0709 12:18:29.011819 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:29.020544 (+  8725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.020779 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.044154 28283 rpcz_store.cc:276] 0709 12:18:27.327937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.038695 (+1710758us) service_pool.cc:224] Handling call
0709 12:18:29.039038 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.039220 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.039250 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.039295 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:29.039339 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.039392 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.039491 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.045419 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 523) took 1717 ms. Trace:
I20260709 12:18:29.046519 28287 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:29.046659 28288 rpcz_store.cc:276] 0709 12:18:27.327621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.044584 (+1716963us) service_pool.cc:224] Handling call
0709 12:18:29.044919 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.045103 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.045127 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.045168 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:29.045197 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.045250 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.045359 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.047349 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1974) took 1225 ms. Trace:
I20260709 12:18:29.048103 28287 rpcz_store.cc:276] 0709 12:18:27.821890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.045971 (+1224081us) service_pool.cc:224] Handling call
0709 12:18:29.046321 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.047059 (+   738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.047284 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.050634 28287 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:29.051111 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 525) took 1558 ms. Trace:
I20260709 12:18:29.053119 28441 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:29.053138 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:29.052630 28283 rpcz_store.cc:276] 0709 12:18:27.492609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.050289 (+1557680us) service_pool.cc:224] Handling call
0709 12:18:29.050570 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.050762 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.050789 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.050833 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:29.050861 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.050916 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.051032 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.030282 32120 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:29.055789 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2009) took 1719 ms. Trace:
I20260709 12:18:29.055764 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:29.056856 31477 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.057366 31960 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.058413 32120 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.055182 32133 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:29.055040 31977 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:29.067196 32203 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.996s	user 0.004s	sys 0.000s
I20260709 12:18:29.063449 32248 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:29.068509 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 459) took 2060 ms. Trace:
I20260709 12:18:29.069061 32026 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:29.067072 32220 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.071501 32220 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:29.068501 32237 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:29.052420 32221 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:29.075423 31970 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.076665 31860 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:28.830528 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:29.078364 31970 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:29.080013 28437 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:29.056257 28441 rpcz_store.cc:276] 0709 12:18:27.336256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.052604 (+1716348us) service_pool.cc:224] Handling call
0709 12:18:29.052932 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.054240 (+  1308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.055725 (+  1485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.080160 28434 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:29.068944 28359 rpcz_store.cc:276] 0709 12:18:27.008578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.544215 (+535637us) service_pool.cc:224] Handling call
0709 12:18:27.544588 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.591125 (+ 46537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.591156 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.604685 (+ 13529us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.644386 (+ 39701us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:27.644435 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.644538 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:29.066631 (+1422093us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:29.068178 (+  1547us) raft_consensus.cc:1685] finished
0709 12:18:29.068202 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.068434 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41285,"raft.queue_time_us":40370,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":2421,"spinlock_wait_cycles":13412224,"thread_start_us":39320,"threads_started":1,"wal-append.queue_time_us":449359}
I20260709 12:18:29.084903 28441 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:29.090077 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 1998) took 2353 ms. Trace:
I20260709 12:18:29.091387 28359 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:29.094058 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1975) took 1271 ms. Trace:
I20260709 12:18:29.087141 32214 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:29.078222 28188 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 8 to 9, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 9 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:29.090550 28360 rpcz_store.cc:276] 0709 12:18:26.733189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:26.828073 (+ 94884us) service_pool.cc:224] Handling call
0709 12:18:26.841659 (+ 13586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:28.833028 (+1991369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:28.833065 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:28.833610 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:18:29.020307 (+186697us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:29.086020 (+ 65713us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.086275 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.086355 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:29.086398 (+    43us) raft_consensus.cc:1685] finished
0709 12:18:29.086413 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.086527 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13434,"raft.queue_time_us":44645,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"thread_start_us":2203283,"threads_started":3,"wal-append.queue_time_us":211467}
I20260709 12:18:29.088275 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:29.097339 28288 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:29.110118 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1976) took 1287 ms. Trace:
I20260709 12:18:29.110249 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:29.099555 28287 rpcz_store.cc:276] 0709 12:18:27.822206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.049971 (+1227765us) service_pool.cc:224] Handling call
0709 12:18:29.050457 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.093798 (+ 43341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.093994 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.110541 28288 rpcz_store.cc:276] 0709 12:18:27.822453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.055960 (+1233507us) service_pool.cc:224] Handling call
0709 12:18:29.097147 (+ 41187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.109850 (+ 12703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.110056 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.111727 28287 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:29.112597 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 1979) took 1289 ms. Trace:
I20260709 12:18:29.112984 28287 rpcz_store.cc:276] 0709 12:18:27.823205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.111249 (+1288044us) service_pool.cc:224] Handling call
0709 12:18:29.111562 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.112225 (+   663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.112541 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.115579 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:29.082376 32029 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:29.116760 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:29.116108 32133 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:29.090160 32220 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:29.051756 32242 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:18:29.133471 32017 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.142223 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:29.143193 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:29.145298 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:29.146593 32230 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:29.147679 28438 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:29.148473 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:29.156865 28438 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:29.148722 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:29.120683 32248 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:29.159574 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:29.155268 28289 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:29.153012 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:29.127507 32029 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.164740 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:29.125574 32245 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:29.129073 32243 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:29.176179 28438 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:29.176551 28330 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:29.183246 28983 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:18:29.142356 32209 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.205322 28287 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:29.210006 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:29.211129 28289 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.165459 32248 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.179775 32029 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.179724 32245 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.165969 32017 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.179041 32230 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.223335 28438 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:29.175544 31970 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:29.166189 32133 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:29.184604 31992 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:29.276166 28438 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:29.218752 28388 debug-util.cc:398] Leaking SignalData structure 0x7b08000f9860 after lost signal to thread 28316
W20260709 12:18:29.278455 28388 debug-util.cc:398] Leaking SignalData structure 0x7b08000f92c0 after lost signal to thread 28317
W20260709 12:18:29.279932 28388 debug-util.cc:398] Leaking SignalData structure 0x7b08002b40c0 after lost signal to thread 28391
W20260709 12:18:29.280282 28388 debug-util.cc:398] Leaking SignalData structure 0x7b0800280a60 after lost signal to thread 28392
W20260709 12:18:29.214787 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47538.1us
I20260709 12:18:29.282663 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:29.284257 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:29.179183 32237 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:29.211614 32209 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.212236 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2023) took 1350 ms. Trace:
I20260709 12:18:29.228852 28289 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:29.290963 28287 rpcz_store.cc:276] 0709 12:18:27.862177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.113855 (+1251678us) service_pool.cc:224] Handling call
0709 12:18:29.114203 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.211746 (+ 97543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.212170 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.291863 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:29.292289 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:29.293069 28287 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.239758 28989 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:18:29.245368 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:29.293395 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:29.211062 32243 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.213143 32221 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:29.281630 28388 debug-util.cc:398] Leaking SignalData structure 0x7b0800341040 after lost signal to thread 28465
W20260709 12:18:29.336103 28388 debug-util.cc:398] Leaking SignalData structure 0x7b080038c5e0 after lost signal to thread 28466
I20260709 12:18:29.294378 28289 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.295101 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:29.277710 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:29.338696 28388 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5360 after lost signal to thread 29015
I20260709 12:18:29.331015 32220 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:29.364629 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 199825us
I20260709 12:18:29.297789 31823 raft_consensus.cc:2764] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:29.223740 32245 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:29.213387 32248 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.332218 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 574) took 1738 ms. Trace:
W20260709 12:18:29.333575 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 88317.1us
I20260709 12:18:29.330490 32243 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:29.392235 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.300029 32129 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:29.330631 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.335047 31992 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:29.300700 31970 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:29.401458 28985 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:29.391894 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:29.395985 28289 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:29.394778 28287 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:29.335793 31860 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:29.423126 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 531) took 1243 ms. Trace:
I20260709 12:18:29.409540 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:29.417249 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:29.566857 28287 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:18:29.437937 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:29.571520 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 162959us
I20260709 12:18:29.566913 28286 rpcz_store.cc:276] 0709 12:18:28.179519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.421921 (+1242402us) service_pool.cc:224] Handling call
0709 12:18:29.422525 (+   604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.422776 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.422802 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.422844 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:29.422873 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.422927 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.423050 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.410588 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:29.420573 28292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1978) took 1597 ms (client timeout 1786 ms). Trace:
W20260709 12:18:29.575484 28292 rpcz_store.cc:269] 0709 12:18:27.822969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.838941 (+ 15972us) service_pool.cc:224] Handling call
0709 12:18:29.420309 (+1581368us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54003ac354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:29.420495 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1391523584}
I20260709 12:18:29.577378 28292 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:18:29.567655 28433 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:29.568447 28289 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:18:29.587460 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:29.588238 28366 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:29.411731 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:29.419636 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.296403 32237 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.376783 31974 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.401767 31907 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.180s
W20260709 12:18:29.583297 28388 debug-util.cc:398] Leaking SignalData structure 0x7b0800379be0 after lost signal to thread 32130
I20260709 12:18:29.598730 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:29.599089 28388 debug-util.cc:398] Leaking SignalData structure 0x7b0800303760 after lost signal to thread 32134
I20260709 12:18:29.599577 28366 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:29.599924 28388 debug-util.cc:398] Leaking SignalData structure 0x7b08003bc1a0 after lost signal to thread 32165
I20260709 12:18:29.599944 28286 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
W20260709 12:18:29.600468 28388 debug-util.cc:398] Leaking SignalData structure 0x7b08001ff540 after lost signal to thread 32176
I20260709 12:18:29.600519 28286 pending_rounds.cc:85] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 6
I20260709 12:18:29.600898 28286 pending_rounds.cc:107] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:18:29.601136 28286 raft_consensus.cc:2889] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:29.583734 28287 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:29.603812 28287 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:29.340978 31977 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:29.558214 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:29.584239 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:29.605855 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47741.8us
W20260709 12:18:29.590420 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 198645us
W20260709 12:18:29.591969 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 358483us
I20260709 12:18:29.606801 28358 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.546747 32221 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:29.400815 32229 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:29.622774 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:29.623524 28366 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.556460 28437 rpcz_store.cc:276] 0709 12:18:27.593573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.090685 (+1497112us) service_pool.cc:224] Handling call
0709 12:18:29.121569 (+ 30884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.123610 (+  2041us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:29.180991 (+ 57381us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.331789 (+150798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.331856 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:29.331905 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.331974 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.332111 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4092,"prepare.queue_time_us":108150,"raft.queue_time_us":34256,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":6157,"spinlock_wait_cycles":15500160,"thread_start_us":180484,"threads_started":3,"wal-append.queue_time_us":150926}
I20260709 12:18:29.592514 28432 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:29.607059 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:29.624994 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:29.607417 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:29.609902 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 269403us
I20260709 12:18:29.617101 28287 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:29.628352 28255 leader_election.cc:400] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9.
I20260709 12:18:29.628720 28255 leader_election.cc:403] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:29.634347 28438 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 8: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:29.635329 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:29.633270 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:29.595202 32045 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.403465 32226 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.639298 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:29.406132 32220 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.404210 32213 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.612800 32026 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:29.612458 32000 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.681308 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:29.685963 28433 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:29.598146 31938 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 12:18:29.688104 31938 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:29.688869 31974 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:29.688623 31938 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.689591 31974 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:29.690121 31974 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:29.689997 32221 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.691133 31974 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.594012 32177 consensus_queue.cc:1059] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:18:29.450201 32133 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.696082 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:29.633123 32229 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:29.637993 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:29.639163 32239 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:29.613752 31860 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:18:29.702199 28287 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:18:29.702517 32253 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
W20260709 12:18:29.702824 32251 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:18:29.719677 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:29.703490 32270 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:29.702311 32259 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:18:29.620748 32046 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:18:29.683131 31907 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:29.710067 28284 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:29.711342 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:29.703115 32252 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
I20260709 12:18:29.703938 32267 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:29.768699 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:29.699466 32228 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:29.722203 32177 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:29.705991 32045 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.700313 32000 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:29.779513 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 151304us
W20260709 12:18:29.784637 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 177321us
I20260709 12:18:29.733258 31907 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:29.694149 31992 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:29.789216 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 190556us
I20260709 12:18:29.791069 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:29.721107 28358 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
W20260709 12:18:29.792224 32258 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.452s	user 0.000s	sys 0.001s
I20260709 12:18:29.835494 28433 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:18:29.708309 31970 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.838740 28437 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:18:29.733104 32270 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.840344 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 473) took 1196 ms. Trace:
W20260709 12:18:29.842140 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44616.3us
I20260709 12:18:29.840785 28363 rpcz_store.cc:276] 0709 12:18:28.643906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.922274 (+278368us) service_pool.cc:224] Handling call
0709 12:18:28.922884 (+   610us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:28.923483 (+   599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:28.923515 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:28.969228 (+ 45713us) log.cc:844] Serialized 608 byte log entry
0709 12:18:29.026408 (+ 57180us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:29.026450 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.026518 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:29.840075 (+813557us) raft_consensus.cc:1685] finished
0709 12:18:29.840098 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.840273 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76,"raft.queue_time_us":891,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":8258,"thread_start_us":16615,"threads_started":1,"wal-append.queue_time_us":45611}
I20260709 12:18:29.801273 32026 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:29.847636 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 275025us
I20260709 12:18:29.856055 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:29.856011 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 245231us
W20260709 12:18:29.839008 32261 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.448s	user 0.001s	sys 0.000s
W20260709 12:18:29.844048 32266 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 12:18:29.863112 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:29.866142 28363 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:29.836410 31977 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:29.861618 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:29.866911 28984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 435) took 1231 ms. Trace:
I20260709 12:18:29.867650 28433 raft_consensus.cc:2410] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:29.769960 32267 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.867412 28984 rpcz_store.cc:276] 0709 12:18:28.635613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.635756 (+   143us) service_pool.cc:224] Handling call
0709 12:18:28.656332 (+ 20576us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:28.938708 (+282376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:28.938742 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:28.945290 (+  6548us) log.cc:844] Serialized 52 byte log entry
0709 12:18:29.056183 (+110893us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:29.056219 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.056288 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:29.866621 (+810333us) raft_consensus.cc:1685] finished
0709 12:18:29.866650 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.866838 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37326,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":375402,"threads_started":2,"wal-append.queue_time_us":172219}
I20260709 12:18:29.871397 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 527) took 2031 ms. Trace:
I20260709 12:18:29.875702 28288 rpcz_store.cc:276] 0709 12:18:27.840338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.111294 (+1270956us) service_pool.cc:224] Handling call
0709 12:18:29.128991 (+ 17697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.129250 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:29.388603 (+259353us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.707054 (+318451us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.870975 (+163921us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.871041 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:29.871093 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.871172 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.871324 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":949,"prepare.queue_time_us":574428,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":7001,"spinlock_wait_cycles":16300416,"thread_start_us":707094,"threads_started":3,"wal-append.queue_time_us":382461}
I20260709 12:18:29.847718 32274 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9.
I20260709 12:18:29.883225 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:29.854728 31938 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:29.895643 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:29.883147 32170 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.851s	user 0.012s	sys 0.000s
I20260709 12:18:29.896504 32170 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.895246 31907 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.901331 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:29.839192 32244 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:18:29.904186 28363 raft_consensus.cc:2368] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:29.908401 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:29.909341 28363 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:18:29.794715 32229 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.870069 31977 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:29.871572 32237 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:29.896533 32214 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:29.895346 31938 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:29.923698 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:29.865494 32026 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:29.895679 32226 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:29.905077 32278 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:29.926313 28363 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:29.905419 32024 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.924682 31938 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:29.919494 32281 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:29.917070 32213 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:29.981079 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2000) took 1354 ms. Trace:
W20260709 12:18:29.925690 32265 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
W20260709 12:18:29.945724 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.968060 32024 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:29.981797 28285 rpcz_store.cc:276] 0709 12:18:28.626585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.430434 (+803849us) service_pool.cc:224] Handling call
0709 12:18:29.430885 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.431347 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:29.637043 (+205696us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.980418 (+343375us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.980486 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:29.980535 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.980616 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.980999 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4919,"prepare.queue_time_us":273904,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":3181,"raft.queue_time_us":161156,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":16867,"thread_start_us":461550,"threads_started":3,"wal-append.queue_time_us":209932}
I20260709 12:18:29.940884 32177 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:29.988762 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 355574us
I20260709 12:18:29.990561 28366 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:29.991392 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:29.969564 31907 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:29.954002 32228 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:29.990182 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 131432us
I20260709 12:18:29.940948 32278 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:29.996042 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:29.968461 32281 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:29.988168 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 157937us
I20260709 12:18:29.998865 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 534) took 1300 ms. Trace:
I20260709 12:18:29.928052 32214 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.018293 28257 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:30.006992 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:29.999435 31974 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:29.997884 32179 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:30.005057 32264 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:18:29.985360 32289 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:18:30.023773 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:29.993180 32226 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:18:30.024230 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:30.006502 32045 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:30.035606 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:30.038790 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:30.038959 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:30.039453 32024 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:30.028074 32290 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:30.041775 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2021) took 1346 ms. Trace:
I20260709 12:18:30.040689 28984 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:30.031522 32177 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.042614 32278 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.042778 31974 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:30.027045 32286 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:30.042207 28360 rpcz_store.cc:276] 0709 12:18:28.695670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.108743 (+413073us) service_pool.cc:224] Handling call
0709 12:18:29.109137 (+   394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:29.124283 (+ 15146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.124322 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:29.154246 (+ 29924us) log.cc:844] Serialized 52 byte log entry
0709 12:18:29.390780 (+236534us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:29.390828 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.390906 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.041501 (+650595us) raft_consensus.cc:1685] finished
0709 12:18:30.041524 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.041680 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14789,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":7388,"thread_start_us":163023,"threads_started":1,"wal-append.queue_time_us":515883}
I20260709 12:18:30.061453 32026 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:30.062739 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:30.062927 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:30.101539 28440 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:30.063444 32282 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:30.063915 32285 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:18:30.028085 32297 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:18:30.066605 28987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 438) took 1250 ms. Trace:
I20260709 12:18:30.067854 32214 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:30.043198 32045 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:30.070094 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 519) took 2898 ms. Trace:
I20260709 12:18:30.116349 28291 rpcz_store.cc:276] 0709 12:18:27.171681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.293393 (+1121712us) service_pool.cc:224] Handling call
0709 12:18:28.293705 (+   312us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:28.391461 (+ 97756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:28.391499 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:28.392189 (+   690us) log.cc:844] Serialized 52 byte log entry
0709 12:18:28.588913 (+196724us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b4800080358. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c37b0aa1 00007f85c37a7b37 00007f85c37a79ca 00007f85c389f26f 00007f85c389ef37 00007f85c38b355c 00007f85c38b3508 00007f85c38b3488 00007f85c38b3443 00007f85c38b26ec 00007f85c27dca96
0709 12:18:29.176307 (+587394us) spinlock_profiling.cc:243] Waited 58.3 ms on lock 0x7b18000f05e0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:18:29.290774 (+114467us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:29.290820 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.290901 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.069825 (+778924us) raft_consensus.cc:1685] finished
0709 12:18:30.069849 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.070020 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49540,"raft.queue_time_us":118618,"raft.run_cpu_time_us":2467,"raft.run_wall_time_us":140874,"spinlock_wait_cycles":138335872,"thread_start_us":153791,"threads_started":2,"wal-append.queue_time_us":1195530}
I20260709 12:18:30.070830 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:30.054296 32228 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:30.071645 28284 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 533) took 1588 ms (client timeout 2105 ms). Trace:
I20260709 12:18:30.071484 31977 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.118686 28986 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:30.072551 32267 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:30.118901 28284 rpcz_store.cc:269] 0709 12:18:28.482798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.825662 (+342864us) service_pool.cc:224] Handling call
0709 12:18:30.071430 (+1245768us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b54001af254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:30.071576 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927769344}
I20260709 12:18:30.030529 32213 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:18:30.121676 32213 consensus_queue.cc:1243] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer f526369c0d0743b192be6e205540a059 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:18:30.073323 32288 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:30.034535 32179 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:18:30.122835 32267 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:30.123395 32267 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:30.076455 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:30.071208 32281 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:30.061580 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:30.100627 28988 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:30.102025 28440 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:30.066134 32029 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.799s	user 0.002s	sys 0.011s
I20260709 12:18:30.107221 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 475) took 1407 ms. Trace:
I20260709 12:18:30.126125 28360 rpcz_store.cc:276] 0709 12:18:28.699345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.106352 (+1407007us) service_pool.cc:224] Handling call
0709 12:18:30.106697 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.106888 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.106913 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.106955 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:30.106985 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.107037 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.107154 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.124392 32267 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.109050 28987 rpcz_store.cc:276] 0709 12:18:28.816094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.816249 (+   155us) service_pool.cc:224] Handling call
0709 12:18:28.816596 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:28.933534 (+116938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.066336 (+1132802us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540046e954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:30.066540 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244068224}
W20260709 12:18:30.109795 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 250505us
I20260709 12:18:30.133616 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:30.112227 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 53604.4us
I20260709 12:18:30.111044 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:30.138325 32267 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.104168 31974 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:30.113611 32279 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:18:30.151459 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 472) took 1527 ms. Trace:
I20260709 12:18:30.166754 28986 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:30.152390 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:30.108649 32270 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:30.146350 32287 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:18:30.132898 32278 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.168942 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:30.149521 32245 consensus_meta.cc:287] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.001s
W20260709 12:18:30.149997 31960 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.010s	sys 0.041s
W20260709 12:18:30.192345 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45130.9us
W20260709 12:18:30.179335 28255 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 1.566s (SENT)
I20260709 12:18:30.186128 32229 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:30.193666 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:30.187295 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:30.154024 32221 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:30.113113 32223 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:18:30.153469 32297 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.203621 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:30.191208 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:30.195441 28987 raft_consensus.cc:2376] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14. Re-sending same reply.
I20260709 12:18:30.167157 32288 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:30.132529 32026 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:18:30.219815 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:30.155085 32213 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:18:30.236757 28359 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:18:30.220022 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:30.151813 32214 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:18:30.272038 28285 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 9: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:30.255481 28284 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:30.194309 32270 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:30.225399 32300 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:18:30.273970 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:30.151274 32029 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:30.279189 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 99951.5us
I20260709 12:18:30.222404 28179 maintenance_manager.cc:419] P b5f297f7df3d4c2da68fabf56f1654bd: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033369
I20260709 12:18:30.153570 32293 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 12:18:30.222761 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:30.295948 28288 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:30.281805 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2274) took 1045 ms. Trace:
I20260709 12:18:30.283355 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:30.189946 32298 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 12:18:30.297164 28285 raft_consensus.cc:2410] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:30.283166 28435 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:30.284031 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:30.275247 32282 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:30.290355 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:30.255755 32288 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.322325 28287 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:30.323048 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:30.323578 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:30.324833 28288 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:18:30.217344 32277 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 12:18:30.326692 28983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 481) took 1185 ms. Trace:
I20260709 12:18:30.292510 32229 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:30.327167 28983 rpcz_store.cc:276] 0709 12:18:29.141680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.145262 (+  3582us) service_pool.cc:224] Handling call
0709 12:18:29.154055 (+  8793us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:29.209413 (+ 55358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.325258 (+1115845us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b540064f354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:30.326478 (+  1220us) spinlock_profiling.cc:243] Waited 8.35 ms on lock 0x7b540064f350. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:30.326612 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85552768}
I20260709 12:18:30.288214 31907 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:30.329797 32248 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.760s	user 0.004s	sys 0.008s
I20260709 12:18:30.330075 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2004) took 1668 ms. Trace:
W20260709 12:18:30.292529 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 101438us
W20260709 12:18:30.283087 32279 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:18:30.299448 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:30.215554 32245 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.278903 32286 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:30.290630 32270 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:30.329659 31907 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.286605 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:30.296679 28438 rpcz_store.cc:276] 0709 12:18:29.236703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.277393 (+ 40690us) service_pool.cc:224] Handling call
0709 12:18:30.281512 (+1004119us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b5400085f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:30.281717 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679789952}
I20260709 12:18:30.292994 32220 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:30.321847 32297 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.338716 28292 rpcz_store.cc:276] 0709 12:18:28.661214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.585586 (+924372us) service_pool.cc:224] Handling call
0709 12:18:29.585964 (+   378us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:30.031032 (+445068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.031065 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:30.036745 (+  5680us) log.cc:844] Serialized 52 byte log entry
0709 12:18:30.235954 (+199209us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:30.235996 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.236076 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.329830 (+ 93754us) raft_consensus.cc:1685] finished
0709 12:18:30.329856 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.330008 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36300,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":6509,"thread_start_us":568461,"threads_started":2,"wal-append.queue_time_us":279101}
I20260709 12:18:30.291172 32226 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.299084 32237 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:30.302625 32282 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:30.322153 32288 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.351079 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:30.365890 28438 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:30.331261 31960 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.330358 32248 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.342602 32274 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:30.374732 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:30.360394 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:30.334652 32228 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:18:30.342103 31860 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.351174 32298 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.295393 31974 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.361581 28437 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:18:30.377614 32299 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:18:30.377799 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:30.377995 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:30.345645 31970 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:30.380270 28286 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:30.376144 28439 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:30.379310 31907 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:30.375021 32274 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:30.375944 32220 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:30.386624 31970 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.387104 32220 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.388860 32029 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:30.382477 31860 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.342504 32229 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:30.398633 32243 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.807s	user 0.013s	sys 0.000s
I20260709 12:18:30.398859 31860 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:30.398970 32229 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:30.390952 32270 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.400753 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:30.401471 28438 raft_consensus.cc:2376] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10. Re-sending same reply.
I20260709 12:18:30.379576 31960 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.363848 32297 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.392704 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2254) took 1921 ms. Trace:
I20260709 12:18:30.370203 32223 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.384344 32298 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:30.396339 32209 consensus_meta.cc:287] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.065s	user 0.010s	sys 0.003s
W20260709 12:18:30.401001 32230 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.161s	user 0.009s	sys 0.004s
I20260709 12:18:30.362672 32226 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.376080 32315 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:30.365638 32237 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.388979 31970 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:30.427835 32305 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 12:18:30.441546 32307 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:18:30.404739 32243 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:30.442740 32308 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:18:30.423178 32230 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.378975 32282 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.460297 28440 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.456846 32243 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:30.459280 32248 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:30.427999 32317 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:18:30.458494 31960 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:30.443696 32310 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:18:30.410697 32223 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.437958 32315 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:30.422024 32209 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.460354 32245 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:30.386781 32274 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.473630 32297 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:30.478786 32315 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.480878 32209 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:30.479580 32282 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.562463 28440 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.626959 32288 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.486994 32274 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.481518 32245 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:30.486682 32298 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:30.485699 32226 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:30.487725 32237 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:30.496320 32223 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:30.701815 32045 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.555s	user 0.007s	sys 0.006s
I20260709 12:18:30.702626 32045 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:30.687182 32245 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:30.709350 28440 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:18:30.709851 28440 pending_rounds.cc:85] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 8
I20260709 12:18:30.710042 28440 pending_rounds.cc:107] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:18:30.710246 28440 raft_consensus.cc:2889] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:30.711388 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:30.712013 28440 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:30.713981 32216 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:30.714499 32216 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:30.714861 32267 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.000s
I20260709 12:18:30.715333 32288 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:30.712342 32245 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:30.717048 28358 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 1997) took 1550 ms (client timeout 1570 ms). Trace:
W20260709 12:18:30.717373 28358 rpcz_store.cc:269] 0709 12:18:29.166529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.416870 (+250341us) service_pool.cc:224] Handling call
0709 12:18:30.716886 (+1300016us) spinlock_profiling.cc:243] Waited 39.4 ms on lock 0x7b54000a7154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:30.716989 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90694016}
I20260709 12:18:30.715338 32216 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.720448 32216 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:30.723592 28440 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
W20260709 12:18:30.724156 32275 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.887s	user 0.000s	sys 0.001s
I20260709 12:18:30.724924 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:30.726035 28440 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:30.726920 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:30.728401 28440 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:30.730130 28440 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:30.731531 28440 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:18:30.733647 28440 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:30.734112 28440 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:30.735071 28440 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.736905 28440 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.738271 32295 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:30.738739 32295 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:30.739593 32295 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.740840 32295 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.737066 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2018) took 1098 ms. Trace:
I20260709 12:18:30.745325 28437 rpcz_store.cc:276] 0709 12:18:29.638751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.840372 (+201621us) service_pool.cc:224] Handling call
0709 12:18:29.840780 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:30.382614 (+541834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.736761 (+354147us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400085554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:30.736994 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197147520}
I20260709 12:18:30.746348 28440 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:18:30.746798 28440 pending_rounds.cc:85] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 7
I20260709 12:18:30.747043 28440 pending_rounds.cc:107] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:18:30.747364 28440 raft_consensus.cc:2889] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:30.749011 28440 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:30.750372 28440 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:30.755384 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 479) took 1614 ms. Trace:
I20260709 12:18:30.755954 28989 rpcz_store.cc:276] 0709 12:18:29.140845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.141907 (+  1062us) service_pool.cc:224] Handling call
0709 12:18:30.755127 (+1613220us) spinlock_profiling.cc:243] Waited 84.4 ms on lock 0x7b5400660154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:30.755297 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194219392}
W20260709 12:18:30.766873 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 360242us
I20260709 12:18:30.772395 32209 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.773015 32209 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.775444 32209 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:30.776520 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.776660 32170 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:30.776921 32209 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:30.779160 32209 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.782280 32209 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:30.811581 28434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2258) took 2331 ms (client timeout 2579 ms). Trace:
W20260709 12:18:30.812115 28434 rpcz_store.cc:269] 0709 12:18:28.480158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.916501 (+436343us) service_pool.cc:224] Handling call
0709 12:18:30.811295 (+1894794us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54002e31d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:30.811491 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373583616}
I20260709 12:18:30.823163 32000 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:30.823635 32239 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.011s	sys 0.002s
W20260709 12:18:30.824214 32239 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.092s	user 0.011s	sys 0.002s
I20260709 12:18:30.824615 32239 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:30.826398 32239 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.833141 28188 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:30.838824 28363 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:30.839632 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2026) took 2105 ms. Trace:
I20260709 12:18:30.839954 28363 rpcz_store.cc:276] 0709 12:18:28.733718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.838222 (+2104504us) service_pool.cc:224] Handling call
0709 12:18:30.838615 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.839348 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.839559 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.841317 28363 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:30.841957 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 476) took 1986 ms. Trace:
I20260709 12:18:30.847322 28363 rpcz_store.cc:276] 0709 12:18:28.855141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.840792 (+1985651us) service_pool.cc:224] Handling call
0709 12:18:30.841125 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.841660 (+   535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.841894 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.835083 28188 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 2 to 4, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 4 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:30.849215 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 477) took 1993 ms. Trace:
I20260709 12:18:30.849769 28363 rpcz_store.cc:276] 0709 12:18:28.855441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.848270 (+1992829us) service_pool.cc:224] Handling call
0709 12:18:30.848617 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.848832 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.848859 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.848906 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:30.848937 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.848991 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.849140 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.851799 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 478) took 1996 ms. Trace:
I20260709 12:18:30.852200 28363 rpcz_store.cc:276] 0709 12:18:28.855703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.850895 (+1995192us) service_pool.cc:224] Handling call
0709 12:18:30.851250 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.851451 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.851474 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.851517 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:30.851547 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.851603 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.851728 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:30.857430 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:30.857996 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:30.858349 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:30.863257 28364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 2002) took 2250 ms (client timeout 2864 ms). Trace:
W20260709 12:18:30.863850 28364 rpcz_store.cc:269] 0709 12:18:28.612556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.612764 (+   208us) service_pool.cc:224] Handling call
0709 12:18:30.862932 (+2250168us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b5400125554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:30.863168 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986909568}
I20260709 12:18:30.867573 31938 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:30.869470 31938 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.865695 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2029) took 1715 ms. Trace:
I20260709 12:18:30.875916 28364 rpcz_store.cc:276] 0709 12:18:29.150533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.864765 (+1714232us) service_pool.cc:224] Handling call
0709 12:18:30.865144 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.865357 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.865383 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.865426 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:30.865462 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.865513 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.865633 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.877979 28364 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:30.878784 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2002) took 1711 ms. Trace:
I20260709 12:18:30.879172 28364 rpcz_store.cc:276] 0709 12:18:29.167829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.877181 (+1709352us) service_pool.cc:224] Handling call
0709 12:18:30.877599 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:30.878470 (+   871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.878718 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.880693 28364 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:30.881444 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2003) took 1713 ms. Trace:
I20260709 12:18:30.885526 28364 rpcz_store.cc:276] 0709 12:18:29.168070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.880031 (+1711961us) service_pool.cc:224] Handling call
0709 12:18:30.880427 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:30.881164 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.881386 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.888083 28364 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:30.888598 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2006) took 1719 ms. Trace:
I20260709 12:18:30.888849 28364 rpcz_store.cc:276] 0709 12:18:29.168835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.887614 (+1718779us) service_pool.cc:224] Handling call
0709 12:18:30.887909 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.888405 (+   496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.888551 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.900034 28364 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:30.900627 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2008) took 1692 ms. Trace:
I20260709 12:18:30.900921 28364 rpcz_store.cc:276] 0709 12:18:29.208679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.899533 (+1690854us) service_pool.cc:224] Handling call
0709 12:18:30.899847 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.900422 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.900576 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.910308 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:30.911093 28989 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:30.918257 28188 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 1 to 9, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 9 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:30.920183 28188 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:30.926955 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 474) took 2228 ms. Trace:
I20260709 12:18:30.927414 28366 rpcz_store.cc:276] 0709 12:18:28.698331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.991871 (+1293540us) service_pool.cc:224] Handling call
0709 12:18:29.994671 (+  2800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.995182 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:30.367378 (+372196us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.926495 (+559117us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.926568 (+    73us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:30.926628 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.926706 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.926878 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":230261,"prepare.queue_time_us":291946,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"thread_start_us":145973,"threads_started":2,"wal-append.queue_time_us":320164}
I20260709 12:18:30.929874 31860 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:30.930472 31860 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:30.930902 31860 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:30.936604 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2025) took 2203 ms. Trace:
I20260709 12:18:30.937170 28358 rpcz_store.cc:276] 0709 12:18:28.733430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.718264 (+1984834us) service_pool.cc:224] Handling call
0709 12:18:30.718619 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.718856 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:30.735908 (+ 17052us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.936111 (+200203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.936186 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:30.936242 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.936329 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.936507 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124668,"prepare.queue_time_us":16859,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"thread_start_us":75016,"threads_started":1,"wal-append.queue_time_us":51639}
I20260709 12:18:30.943318 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:30.945194 32213 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:30.946267 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:30.946003 32213 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:30.948596 32213 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:30.945360 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 480) took 1847 ms. Trace:
I20260709 12:18:30.969233 28363 rpcz_store.cc:276] 0709 12:18:29.098409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.853209 (+1754800us) service_pool.cc:224] Handling call
0709 12:18:30.853566 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.853829 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:18:30.854133 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.944946 (+ 90813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.945013 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:30.945062 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.945139 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.945282 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78790,"prepare.queue_time_us":122,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":83479,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":145,"thread_start_us":11567,"threads_started":1,"wal-append.queue_time_us":6350}
I20260709 12:18:30.971633 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:30.973047 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:30.974473 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:30.975273 28364 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:30.973300 28363 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:18:30.982424 28363 pending_rounds.cc:85] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 11
I20260709 12:18:30.984637 28363 pending_rounds.cc:107] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:18:30.984987 28363 raft_consensus.cc:2889] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:31.251309 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.323151 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 190776us
I20260709 12:18:31.550202 32230 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:31.581880 32230 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:31.582788 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.584328 32230 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:31.585318 32230 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.569185 28290 raft_consensus.cc:2368] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
I20260709 12:18:31.585843 32230 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:31.588667 32230 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:30.402740 28441 rpcz_store.cc:276] 0709 12:18:28.470866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.094592 (+623726us) service_pool.cc:224] Handling call
0709 12:18:29.097955 (+  3363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:29.098695 (+   740us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:29.217458 (+118763us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b180041ffe0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85c227f3c7 00007f85be8919a7 00007f85be8a10bf 00007f85be8a1038
0709 12:18:29.454042 (+236584us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.881911 (+427869us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:29.882561 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:18:29.882715 (+   154us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:29.882752 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.882823 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.392314 (+509491us) raft_consensus.cc:1685] finished
0709 12:18:30.392473 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.392634 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":120171,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":51174,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":17187,"spinlock_wait_cycles":3628928,"thread_start_us":503483,"threads_started":3,"wal-append.queue_time_us":252845}
I20260709 12:18:31.635114 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:31.636122 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2027) took 1306 ms. Trace:
I20260709 12:18:31.636458 28438 rpcz_store.cc:276] 0709 12:18:30.329228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.403833 (+ 74605us) service_pool.cc:224] Handling call
0709 12:18:31.636045 (+1232212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:31.658720 28290 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 1997) took 3057 ms (client timeout 1866 ms). Trace:
W20260709 12:18:31.659248 28290 rpcz_store.cc:269] 0709 12:18:28.601363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.824534 (+223171us) service_pool.cc:224] Handling call
0709 12:18:30.076268 (+1251734us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5800000098. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762e9c 00007f85bb762a5b 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:18:31.658438 (+1582170us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54003f9b54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:31.658625 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3367590400}
W20260709 12:18:31.661113 28367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56626 (request call id 479) took 2738 ms (client timeout 1817 ms). Trace:
W20260709 12:18:31.661580 28367 rpcz_store.cc:269] 0709 12:18:28.922994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.936878 (+ 13884us) service_pool.cc:224] Handling call
0709 12:18:31.660898 (+2724020us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b54003570d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:31.661027 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5986048}
I20260709 12:18:31.667812 32230 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.668668 28365 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 12:18:31.668411 32230 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:31.671196 32230 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:31.671749 32230 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:31.672219 32230 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:31.675424 32230 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:31.676028 32230 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.676501 32230 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:31.679097 32230 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:31.679668 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.679623 32230 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:31.680290 32230 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:31.682473 32230 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:31.685803 31921 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:31.686260 31921 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:31.687150 31921 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:31.688400 31921 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:31.680348 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 575) took 2080 ms. Trace:
I20260709 12:18:31.689674 28435 rpcz_store.cc:276] 0709 12:18:29.599874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.600082 (+   208us) service_pool.cc:224] Handling call
0709 12:18:30.282974 (+682892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.297631 (+ 14657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.680138 (+1382507us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b5400085f50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:31.680281 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569671936}
W20260709 12:18:31.692030 28404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.360s	user 0.023s	sys 0.017s
W20260709 12:18:31.693054 28404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.361s	user 0.023s	sys 0.017s
W20260709 12:18:31.693399 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 1.36117e+06us
I20260709 12:18:31.694842 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:31.696290 28404 leader_election.cc:400] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
I20260709 12:18:31.696550 28404 leader_election.cc:403] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:18:31.702189 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:31.703065 28290 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:31.704147 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:31.704819 28284 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:31.705829 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:31.711678 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:31.712406 28291 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:31.712523 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:31.713304 28284 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:31.714138 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:31.715710 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:31.714367 28433 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 12:18:31.716512 28290 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:18:31.717219 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1727319760000228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 12:18:31.717720 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1601319760000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
W20260709 12:18:31.718050 28404 connection.cc:256] RPC call timeout handler was delayed by -1.147531975999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 12:18:31.718282 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1358319759999631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
W20260709 12:18:31.718504 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1098917619999611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
W20260709 12:18:31.718714 28404 connection.cc:256] RPC call timeout handler was delayed by -1.034131976000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
I20260709 12:18:31.719542 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:31.719628 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:31.720237 28290 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:31.720489 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:31.713317 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:31.721715 28291 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.721901 32343 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:31.729605 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:31.734028 28284 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:31.736073 28404 leader_election.cc:400] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:31.736428 28404 leader_election.cc:403] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 12:18:31.732479 28330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.559s	user 0.030s	sys 0.036s
W20260709 12:18:31.739672 28330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.567s	user 0.031s	sys 0.037s
W20260709 12:18:31.740002 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 1.57488e+06us
I20260709 12:18:31.740880 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:31.742219 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:31.744489 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:31.699936 32230 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.745770 28985 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:31.734303 32343 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:31.733378 28283 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:31.748524 32315 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:31.749006 28983 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:31.751637 28291 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:31.752991 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:31.754622 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:31.751744 32345 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:31.762742 32345 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:31.763290 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2053) took 1478 ms. Trace:
I20260709 12:18:31.763686 28286 rpcz_store.cc:276] 0709 12:18:30.284725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.287161 (+  2436us) service_pool.cc:224] Handling call
0709 12:18:31.763096 (+1475935us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b5400059bd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:31.763227 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89337216}
I20260709 12:18:31.752874 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:31.765337 32345 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:31.765676 28283 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:31.765897 32345 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:31.766417 32345 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:31.768317 28284 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:31.771826 32317 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:31.772250 32317 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:31.772603 32317 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:31.768406 32345 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:31.775092 32345 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:31.775660 32345 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:31.775965 32345 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:31.756964 32248 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:31.778440 32248 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:31.746310 32230 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:31.781369 32230 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:31.749653 32315 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:31.790517 32315 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:31.791224 32315 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:31.791849 32315 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:31.760937 28290 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:31.784292 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:31.794195 28435 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
W20260709 12:18:31.785418 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.785719 28438 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:31.764816 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:31.799170 32343 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.787981 32248 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:31.786339 28439 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:18:31.806357 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:31.928881 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.943336 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:31.944275 28441 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:31.946343 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:31.947304 28441 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
W20260709 12:18:31.985034 28462 debug-util.cc:398] Leaking SignalData structure 0x7b080017d060 after lost signal to thread 27119
I20260709 12:18:32.059333 28441 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:32.062374 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:32.077126 28439 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:32.075986 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.776710 32345 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.008910 32248 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:32.015857 32343 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:31.736810 32344 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
W20260709 12:18:32.091488 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 337044us
W20260709 12:18:32.087440 32315 consensus_queue.cc:1175] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6634fa3fbd594726a5ec3c1686900284" responder_term: 10 status { last_received { term: 10 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:32.094864 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:18:32.096225 32254 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.908s	user 0.006s	sys 0.000s
I20260709 12:18:32.107050 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:32.107527 32346 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.108196 32248 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.108249 32346 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.108661 32170 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:32.108901 32248 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.110138 32170 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:32.110544 32170 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.110898 32170 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.093251 32344 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:32.114210 32344 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.115286 32344 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.116629 32344 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.117092 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.118022 28290 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:32.120792 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.121395 28290 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:32.123116 32170 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.123854 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.124732 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.127947 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.128630 28290 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:32.091593 32345 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.166985 28362 rpcz_store.cc:276] 0709 12:18:28.624006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.885514 (+261508us) service_pool.cc:224] Handling call
0709 12:18:28.885795 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:28.885974 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:29.032242 (+146268us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.150478 (+1118236us) spinlock_profiling.cc:243] Waited 7.84 ms on lock 0x7b180043c540. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85c227f3c7 00007f85be8919a7 00007f85be8a10bf 00007f85be8a1038
0709 12:18:30.150717 (+   239us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.150930 (+   213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.150987 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:30.151042 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.151135 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.151362 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18239,"prepare.queue_time_us":272358,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":5099,"raft.queue_time_us":34663,"spinlock_wait_cycles":49565952,"thread_start_us":225328,"threads_started":2,"wal-append.queue_time_us":356897}
W20260709 12:18:32.131152 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.131657 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.117072 32346 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.138861 32248 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.141750 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:32.125475 32170 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.142935 32346 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:32.143362 32346 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:32.143769 32346 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:32.147254 28255 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 1.677s (SENT) [suppressed 23 similar messages]
I20260709 12:18:32.154955 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.157194 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.158036 28292 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:32.152933 32346 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.159511 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.170202 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:32.163683 28432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 581) took 2560 ms (client timeout 2105 ms). Trace:
W20260709 12:18:32.171036 28432 rpcz_store.cc:269] 0709 12:18:29.603545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.680220 (+ 76675us) service_pool.cc:224] Handling call
0709 12:18:32.163414 (+2483194us) spinlock_profiling.cc:243] Waited 2.46 s on lock 0x7b5400347454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:32.163584 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5659498240}
I20260709 12:18:32.166769 32303 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:32.172272 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.172242 32303 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.173147 32209 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:32.173687 32209 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.174734 32303 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.175252 32303 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.175690 32303 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.177685 32303 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:32.159478 32170 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:32.177493 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:32.178462 28284 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:32.179948 32303 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.180096 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:32.179883 32346 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.180264 32226 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:32.181213 32303 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.181411 32226 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.182068 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:32.180485 32153 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.177395 32298 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.182768 32153 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.182874 32298 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.183997 28292 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:18:32.179872 28283 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:32.184954 32153 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:32.185074 32298 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:32.185570 32153 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:32.185712 32298 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:32.186594 32153 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:32.187073 32153 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.187372 32226 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.187548 32153 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.187979 32226 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:32.187834 32298 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:32.180727 32209 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.190445 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.191203 32248 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:32.191937 32000 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:32.192462 32000 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.194355 32248 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.195412 32248 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.197088 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.195767 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.200140 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.202562 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.201619 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.203655 28291 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:32.191879 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2023) took 3491 ms. Trace:
I20260709 12:18:32.193509 32000 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.211725 32267 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:32.204008 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.207046 28360 rpcz_store.cc:276] 0709 12:18:28.700561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.127107 (+1426546us) service_pool.cc:224] Handling call
0709 12:18:30.127510 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:30.127968 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.127995 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:30.128607 (+   612us) log.cc:844] Serialized 52 byte log entry
0709 12:18:30.331437 (+202830us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:30.331483 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.331555 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:32.191619 (+1860064us) raft_consensus.cc:1685] finished
0709 12:18:32.191646 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:32.191813 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":173,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":138344,"threads_started":1,"wal-append.queue_time_us":229456,"wal-append.run_cpu_time_us":11391,"wal-append.run_wall_time_us":1339982}
I20260709 12:18:32.219600 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.178472 32170 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:32.220606 32170 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:32.221030 32153 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.224615 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.225816 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:32.226461 28985 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.227941 32346 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:32.228470 32346 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.228907 32346 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:32.230581 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.235181 28291 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:32.237397 28291 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:32.239573 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.240275 28292 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:32.240309 28291 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:32.242130 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:32.243708 28283 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:32.244470 28283 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:32.245638 28286 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:32.246628 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:32.222702 32209 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:32.247274 32000 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:32.188357 32226 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:32.246452 28283 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.224318 32345 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:32.249485 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.223292 32267 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.170279 32317 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.259773 28283 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:32.261435 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:32.230020 32344 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:18:32.259385 32000 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.269408 32344 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:32.270251 32000 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.270895 32000 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:32.272290 32000 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:32.272722 32000 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.271920 32343 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.273042 32267 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.274749 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.261220 32345 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:32.274709 32317 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.278611 32317 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:32.272955 28291 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 12:18:32.267427 28985 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:32.274816 32343 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.286960 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:32.287521 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 78898us
I20260709 12:18:32.291002 32046 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:32.291240 32120 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:32.292572 32046 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:32.289186 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:32.293134 32046 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:32.294117 32344 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:32.294088 32046 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.294771 32344 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.295835 32344 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.289216 32295 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:32.282290 32345 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.297578 32295 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.300053 32295 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.273082 32000 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.309448 28283 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:32.311477 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:32.312131 28439 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.312577 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:18:32.316176 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.316368 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.317090 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.318684 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:32.318800 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:32.319489 28291 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.323378 28432 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:32.324554 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.325460 28432 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:32.326052 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.326419 28283 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:32.327344 28436 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:18:32.331453 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.332223 28434 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:32.333238 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.334154 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.335035 28440 raft_consensus.cc:2410] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:32.337391 28432 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:32.339017 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.347276 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.348187 28436 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:32.318339 32295 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:32.318419 32343 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.352557 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.320673 32209 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:32.354513 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.355393 28437 raft_consensus.cc:2410] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:32.306707 32280 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:18:32.360100 28291 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:32.362360 28438 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:32.352201 32298 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.371198 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2057) took 2085 ms. Trace:
I20260709 12:18:32.371698 28287 rpcz_store.cc:276] 0709 12:18:30.285723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.324513 (+ 38790us) service_pool.cc:224] Handling call
0709 12:18:30.324865 (+   352us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:30.375347 (+ 50482us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:30.744756 (+369409us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b180003e020. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:30.918914 (+174158us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b180003e020. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85c227f3c7 00007f85be8919a7 00007f85be8a10bf 00007f85be8a1038
0709 12:18:30.986148 (+ 67234us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.998112 (+ 11964us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:30.998886 (+   774us) log.cc:844] Serialized 78 byte log entry
0709 12:18:30.999089 (+   203us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:30.999361 (+   272us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.000194 (+   833us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.000472 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.370805 (+1370333us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b540009e2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:32.370864 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:32.370917 (+    53us) raft_consensus.cc:1685] finished
0709 12:18:32.370930 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:32.371120 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10909,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":123,"raft.queue_time_us":1761289,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":3313,"spinlock_wait_cycles":123459456,"thread_start_us":77747,"threads_started":2,"wal-append.queue_time_us":4169,"wal-append.run_cpu_time_us":3878,"wal-append.run_wall_time_us":1096558}
I20260709 12:18:32.380615 32280 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:32.359587 32343 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.384052 32343 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.388662 28439 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:32.399046 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.398623 28188 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 7 to 8, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 8 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:32.399748 28438 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:32.401046 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.400718 28188 catalog_manager.cc:5697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 3 to 6, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 6 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:32.402504 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.403220 28437 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:31.673724 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:32.407133 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:18:32.409477 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.409684 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.322136 32226 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.411438 28287 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:32.352138 32295 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.418057 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:32.417016 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.419332 28439 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:32.418062 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.394259 32353 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:32.394759 32230 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.423851 32226 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.422729 32209 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:32.424499 32346 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.421361 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.392733 32320 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:18:32.423208 32353 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:32.438454 28435 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:32.426709 32230 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.448319 32209 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:32.463588 32209 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:32.464087 32209 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:32.473076 32353 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.448865 32345 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.486752 28432 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:32.489406 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:32.494987 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2022) took 3799 ms. Trace:
I20260709 12:18:32.496026 32346 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.488158 32345 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.497895 28288 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:32.495486 28365 rpcz_store.cc:276] 0709 12:18:28.696009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.144288 (+448279us) service_pool.cc:224] Handling call
0709 12:18:31.668378 (+2524090us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:31.669058 (+   680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.494623 (+825565us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b54003570d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:32.494892 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5587776384}
W20260709 12:18:32.499560 32358 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 12:18:32.504577 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 142382us
I20260709 12:18:32.505172 32320 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:32.511596 28954 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:32.485334 32209 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.515849 31921 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:32.513962 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2020) took 2815 ms. Trace:
I20260709 12:18:32.514067 32232 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:32.517520 32320 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:32.517707 32232 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:32.518059 32209 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.518792 32209 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:32.518646 32320 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.516853 28433 rpcz_store.cc:276] 0709 12:18:29.697570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.897695 (+200125us) service_pool.cc:224] Handling call
0709 12:18:29.913273 (+ 15578us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:31.716377 (+1803104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.512346 (+795969us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54003cb254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:32.512608 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4141557504}
I20260709 12:18:32.519232 32209 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:32.521378 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 100097us
I20260709 12:18:32.520665 32209 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.522426 32368 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:32.522862 32368 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:32.503350 32170 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.524971 32368 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.529300 32368 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.532531 28283 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:32.533638 28283 raft_consensus.cc:606] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:32.534195 28283 raft_consensus.cc:614] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:32.499276 32362 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:18:32.528111 32295 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.542282 32170 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.543102 32362 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.543661 32362 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:32.547969 28462 debug-util.cc:398] Leaking SignalData structure 0x7b08000d68a0 after lost signal to thread 28224
W20260709 12:18:32.549324 31860 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.618s	user 0.026s	sys 0.008s
I20260709 12:18:32.550016 31860 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:32.553996 28956 connection.cc:256] RPC call timeout handler was delayed by -1.2408298789998753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21050000000000002s for a response
I20260709 12:18:32.563234 32361 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:32.523010 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.565478 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.559701 32017 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:32.560796 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2020) took 3891 ms. Trace:
I20260709 12:18:32.566493 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.566232 28441 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:32.567459 28359 rpcz_store.cc:276] 0709 12:18:28.695276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.093447 (+398171us) service_pool.cc:224] Handling call
0709 12:18:29.093855 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:30.271668 (+1177813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.560486 (+2288818us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400370ad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:32.560721 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2627639040}
I20260709 12:18:32.569685 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:32.570353 28283 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.570410 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:32.572115 32368 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:18:32.574259 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:32.574786 32368 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:32.575259 32368 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:32.575953 32230 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:32.574853 28287 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:32.572120 32301 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:18:32.575263 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:32.572301 32226 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:32.576418 32368 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.577175 32226 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.577821 32368 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:32.577224 28291 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.577626 32226 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:32.573549 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 62052.6us
I20260709 12:18:32.591403 28283 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:32.587846 28291 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:32.592064 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:32.590948 28287 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:32.596443 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.631409 28291 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:32.613574 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.634297 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:32.634971 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.635341 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.637066 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.638104 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:32.639263 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:32.606550 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.593473 32301 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:32.643481 32301 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.636427 32170 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 12:18:32.618927 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.642300 28286 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:32.641250 28439 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:32.647078 28286 raft_consensus.cc:606] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:32.571715 32274 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:32.597296 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:32.651578 28286 raft_consensus.cc:614] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:32.647387 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.645718 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.654289 28440 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:32.657146 28291 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:32.657521 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.641511 31921 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:18:32.643991 32301 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.659567 32344 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:32.652961 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.633618 32361 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:18:32.665717 28439 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:32.662458 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:32.655339 32274 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.634536 32000 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.669382 28440 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:32.656495 32346 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:32.642017 32270 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:18:32.644207 32295 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:18:32.647276 32345 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:18:32.663991 32344 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.664485 31921 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:18:32.664891 32367 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:32.676692 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.673949 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.664733 31860 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.679980 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:32.681056 32366 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:32.655200 32209 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.687798 28291 raft_consensus.cc:606] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:32.671452 32129 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:18:32.694947 32000 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:32.686791 32367 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.706302 32301 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.706979 32367 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.708676 32301 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:32.679566 32344 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.671303 32274 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:32.703339 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 58447.5us
I20260709 12:18:32.703922 32346 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.704479 32170 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:32.705153 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:32.706132 32343 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:32.712906 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:32.713032 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.718513 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:32.719262 28284 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:32.718988 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.721293 28286 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 12:18:32.706525 32290 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:18:32.709913 32248 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:32.726984 32170 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.713512 32366 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:32.722071 32295 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:32.719470 32000 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:32.717372 32343 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.734215 28284 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:32.742182 32320 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:32.710316 32226 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.748714 31921 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:32.753582 28986 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:32.754891 32369 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:32.757094 32274 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.757826 32295 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.758210 32343 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.759300 32153 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:32.761293 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.762434 32370 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 12:18:32.763648 32170 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:32.780913 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.781793 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.782392 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 217859us
I20260709 12:18:32.782958 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:32.783608 28439 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.786212 28956 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:32.788762 32295 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.782354 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:32.775696 32369 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:32.770676 32000 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.770838 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:32.769783 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 56020.3us
I20260709 12:18:32.772481 32209 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:32.774473 32226 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:32.736500 32248 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:32.776485 28284 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:32.789656 28440 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.853077 32046 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.769472 32346 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.792475 32290 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.825935 28290 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:32.882782 28290 raft_consensus.cc:606] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:32.806515 32366 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.838519 32000 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.853123 32209 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.850849 32274 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:32.852499 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:32.904734 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 183922us
I20260709 12:18:32.912798 32232 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:32.875890 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.915716 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:32.917328 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.874150 32367 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:32.917985 31960 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:32.918543 31960 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.919462 32367 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:32.921223 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.922217 28362 raft_consensus.cc:2410] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:32.922613 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.925232 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.926074 28367 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:32.928442 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 1.18754e+06us
I20260709 12:18:32.929492 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:32.930397 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:32.931275 28989 consensus_meta.cc:287] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 2.020s	user 0.017s	sys 0.000s
I20260709 12:18:32.894768 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.907269 32353 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:32.930625 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.940011 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:32.940428 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.922128 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.930121 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:32.942116 28359 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.944468 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:32.945333 28441 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:32.946313 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:32.947647 28441 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:32.947889 28332 connection.cc:256] RPC call timeout handler was delayed by -1.1118595810000897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
W20260709 12:18:32.948767 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.949162 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.933251 28989 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:32.950836 28983 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:32.951555 28989 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 440) took 2041 ms (client timeout 2382 ms). Trace:
W20260709 12:18:32.952051 28989 rpcz_store.cc:269] 0709 12:18:30.909811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.909967 (+   156us) service_pool.cc:224] Handling call
0709 12:18:32.951477 (+2041510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:32.935268 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.952894 28360 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:32.892522 28439 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:32.936393 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.937134 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.937166 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.937927 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.956805 28358 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:32.934146 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.958536 28367 raft_consensus.cc:2410] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:32.954420 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.959126 28363 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:32.955865 28364 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:32.962059 28366 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:32.768292 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:32.964490 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 216559us
I20260709 12:18:32.830694 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:32.953835 31477 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:32.974249 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.977710 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:32.965937 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:32.932592 32237 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.932668 32312 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.983062 28437 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.978469 28438 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.938457 32297 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:32.954488 32301 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:32.985574 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 55253.2us
I20260709 12:18:32.929991 32209 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.987555 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.988318 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:32.989079 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:32.954141 32293 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:32.986315 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:32.989919 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:33.063827 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.091243 28363 raft_consensus.cc:2376] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12. Re-sending same reply.
I20260709 12:18:33.082805 28441 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:33.090327 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.092123 28365 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.086076 28358 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:33.094357 28360 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.980195 32232 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:33.097572 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 109342us
I20260709 12:18:32.954737 32344 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.091305 28364 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:32.976312 31960 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:33.082342 32281 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:33.121752 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.096779 32289 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.990579 32174 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:33.106921 31477 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:32.931878 32290 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:33.137241 28438 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:33.127585 28436 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:33.096586 31860 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:33.097002 32228 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:33.122948 28359 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:33.126466 28440 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:32.985028 32237 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.177376 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.195961 28290 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:33.113286 32295 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:33.088027 32297 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.119776 32293 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:33.199388 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:32.964694 32369 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:32.989892 32312 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.192746 28434 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:18:32.989670 32248 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:33.136636 32289 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.138365 32174 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.189141 28365 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:33.484380 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.531833 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:33.238538 28437 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:33.254630 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:33.434525 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:33.489832 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.537078 28988 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
W20260709 12:18:33.537395 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.522929 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 593) took 1123 ms. Trace:
I20260709 12:18:33.336267 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.203150 28367 raft_consensus.cc:2376] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11. Re-sending same reply.
I20260709 12:18:33.503954 28440 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:33.538585 28284 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
W20260709 12:18:33.262121 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 153350us
I20260709 12:18:33.551560 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:33.293704 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.244838 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:33.551559 28331 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.005s	sys 0.009s
W20260709 12:18:33.554571 28331 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.005s	sys 0.009s
W20260709 12:18:33.448316 32387 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:18:33.201705 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:33.529659 31823 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.556792 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.560289 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.561208 28366 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:33.339138 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:33.197911 32281 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:33.553872 32315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.000s	sys 0.014s
I20260709 12:18:33.564894 28284 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:33.567917 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:33.533401 32209 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:33.533658 32303 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:33.538134 28432 rpcz_store.cc:276] 0709 12:18:32.399059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.405026 (+  5967us) service_pool.cc:224] Handling call
0709 12:18:32.405322 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.490077 (+ 84755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.522554 (+1032477us) spinlock_profiling.cc:243] Waited 81.2 ms on lock 0x7b5400348854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:33.522829 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186746752}
I20260709 12:18:33.575109 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:33.539657 32344 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:33.246243 32295 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:33.569519 28990 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:33.322955 31977 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.577121 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:33.318549 32267 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:33.199792 31860 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.580106 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.244721 32379 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.476950 32288 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.622581 32288 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:33.568032 32230 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:33.626160 32288 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:33.245635 32383 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:18:33.574743 32315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.000s	sys 0.014s
I20260709 12:18:33.626525 32293 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:33.201400 31477 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.414s
I20260709 12:18:33.631629 32395 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:33.631388 32295 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:33.633261 32232 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
I20260709 12:18:33.632855 32281 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:33.635021 32174 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.600006 32209 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.637042 32209 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.638157 31860 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.639322 32209 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.568480 31823 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.643019 31823 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.202613 31960 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.396s
I20260709 12:18:33.644054 31960 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:33.644450 31960 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:33.645364 31960 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.198761 32228 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.618232 32267 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.619050 32289 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.620081 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:33.652109 28462 debug-util.cc:398] Leaking SignalData structure 0x7b0800134da0 after lost signal to thread 28316
W20260709 12:18:33.652522 28462 debug-util.cc:398] Leaking SignalData structure 0x7b0800134cc0 after lost signal to thread 28317
I20260709 12:18:33.652666 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.653927 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.654521 28284 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:33.655267 28284 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:33.620868 32379 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:33.631784 31977 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.656530 31977 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.657673 32379 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:33.659053 31977 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.659935 28984 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:33.614178 32237 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.661455 28284 raft_consensus.cc:606] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:33.662066 28284 raft_consensus.cc:614] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:33.663424 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:33.663709 32267 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:33.665128 28360 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.570s	user 0.010s	sys 0.002s
I20260709 12:18:33.665517 28360 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:33.651999 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2029) took 1542 ms. Trace:
W20260709 12:18:33.665613 28956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.011s	sys 0.049s
W20260709 12:18:33.666556 28956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.011s	sys 0.049s
I20260709 12:18:33.666144 28285 rpcz_store.cc:276] 0709 12:18:32.109820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.114073 (+  4253us) service_pool.cc:224] Handling call
0709 12:18:32.114466 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:32.114716 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:32.114742 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:32.115222 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:18:32.115398 (+   176us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:32.115434 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.115511 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:33.651740 (+1536229us) raft_consensus.cc:1685] finished
0709 12:18:33.651765 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.651929 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:33.668526 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.668875 28285 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:18:33.666082 28404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.014s	sys 0.046s
W20260709 12:18:33.670146 28404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.014s	sys 0.046s
W20260709 12:18:33.671231 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.673892 28285 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:33.675909 28285 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:33.685978 32170 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:33.690536 28330 connection.cc:256] RPC call timeout handler was delayed by -1.5451393759999519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 12:18:33.690613 31977 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:33.692595 28330 connection.cc:256] RPC call timeout handler was delayed by -1.3201393760000428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260709 12:18:33.692883 31977 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.695427 31977 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.690917 32237 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.696718 32237 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.699199 32237 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:33.700814 28330 connection.cc:256] RPC call timeout handler was delayed by -1.094239375999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
W20260709 12:18:33.703118 28330 connection.cc:256] RPC call timeout handler was delayed by -1.0321393760000319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 12:18:33.704573 32170 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.705201 32170 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.705566 32177 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:33.706132 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:33.707242 32177 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:33.707475 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:33.707865 32170 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:33.708894 32177 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:33.708702 31977 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:18:33.708019 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.712842 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.714669 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.716375 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:33.724898 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:33.648051 32288 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.672719 32267 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.773922 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.725057 31977 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.754895 28434 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:33.773612 32312 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.773895 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:33.771063 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 46243.8us
I20260709 12:18:33.775940 28360 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:33.777933 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.778934 28367 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:33.739828 32297 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.740322 32289 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:33.784055 28284 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:18:33.784298 28330 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:33.779539 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 227590us
I20260709 12:18:33.768976 31960 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:33.775207 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 224581us
I20260709 12:18:33.788228 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:33.790530 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.791428 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.792814 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.793553 28363 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:33.795593 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:33.796803 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.798065 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.798930 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.799587 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.808707 28364 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.802449 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.774946 32403 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:33.809650 28367 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:33.810367 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.810631 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.807953 28363 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:33.812301 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.811367 28367 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:33.806907 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.809376 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:33.769728 32288 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:33.813650 28284 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:33.813854 28285 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:33.814420 28359 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:33.818817 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.819591 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.821965 28365 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:33.822171 28367 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:33.799178 31823 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:33.823060 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.825193 28367 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:33.826381 28367 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:33.824651 32399 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:18:33.780241 32267 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.795686 32297 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:33.830472 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:33.832034 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.832432 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:33.833398 28364 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:33.835237 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.837360 28359 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:33.839896 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.840920 28365 raft_consensus.cc:2410] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:33.838692 28285 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:33.843727 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.844682 28285 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:33.842041 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:33.847882 28255 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:33.849591 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:33.850533 28284 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.852314 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.853058 28365 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:33.854899 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:33.853034 28285 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:33.854000 32232 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:33.856040 32230 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:33.838342 31823 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.835038 32223 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.860227 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:33.862308 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.872730 28363 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:33.859119 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:33.868124 32397 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:18:33.846328 32410 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:33.857692 32303 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.838109 32209 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:33.857779 32228 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:33.870944 28359 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:33.874269 28285 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:33.873876 32223 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:33.878942 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.881003 28284 raft_consensus.cc:2410] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:33.881788 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.882447 28364 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:33.883538 28285 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:33.871258 32230 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.889595 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.890508 28363 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:33.891582 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:33.892186 28363 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:33.895332 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.896759 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.897509 28283 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:33.899225 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2059) took 1157 ms. Trace:
I20260709 12:18:33.899724 28291 rpcz_store.cc:276] 0709 12:18:32.742283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.937247 (+194964us) service_pool.cc:224] Handling call
0709 12:18:32.937601 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:32.963800 (+ 26199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:32.963834 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:32.981158 (+ 17324us) log.cc:844] Serialized 52 byte log entry
0709 12:18:33.334735 (+353577us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:33.334781 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.334856 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:33.898940 (+564084us) raft_consensus.cc:1685] finished
0709 12:18:33.898968 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.899150 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19045,"raft.queue_time_us":24113,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8860,"thread_start_us":214479,"threads_started":1,"wal-append.queue_time_us":548315}
I20260709 12:18:33.902053 32303 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:33.902451 28435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2033) took 1588 ms (client timeout 2009 ms). Trace:
I20260709 12:18:33.902649 32303 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:33.903141 32303 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:33.871069 32232 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:33.906904 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.908843 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2054) took 1189 ms. Trace:
I20260709 12:18:33.909303 28288 rpcz_store.cc:276] 0709 12:18:32.716482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.716687 (+   205us) service_pool.cc:224] Handling call
0709 12:18:32.717009 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.717227 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:32.728464 (+ 11237us) log.cc:844] Serialized 14 byte log entry
0709 12:18:33.905216 (+1176752us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b1800348100. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287
0709 12:18:33.905343 (+   127us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:33.905407 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:33.905457 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.905540 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.905681 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7056,"prepare.queue_time_us":13059,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":27401,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":48,"spinlock_wait_cycles":9162112,"thread_start_us":7381,"threads_started":1}
I20260709 12:18:33.877755 32410 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:33.916128 28359 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:33.902981 28435 rpcz_store.cc:269] 0709 12:18:32.314034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.314152 (+   118us) service_pool.cc:224] Handling call
0709 12:18:33.902192 (+1588040us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54002e27d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:33.902375 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249493248}
I20260709 12:18:33.948962 32223 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:33.949788 32411 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:18:33.919109 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:34.016641 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:33.984261 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.017799 28283 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:34.018688 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.908782 28291 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:33.909411 32232 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:33.907819 32230 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:34.020637 28364 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:34.018750 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:34.022934 28365 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
W20260709 12:18:33.949040 32415 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:18:34.023993 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.008652 32274 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.007447 28289 rpcz_store.cc:276] 0709 12:18:28.698762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.588192 (+889430us) service_pool.cc:224] Handling call
0709 12:18:29.588496 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.588718 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:29.883699 (+294981us) log.cc:844] Serialized 14 byte log entry
0709 12:18:29.998497 (+114798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.998551 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:29.998600 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.998677 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.998799 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3471,"prepare.queue_time_us":179903,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":4629,"raft.queue_time_us":909,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5382,"thread_start_us":375332,"threads_started":2}
I20260709 12:18:33.983604 32209 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.033332 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.025293 28363 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
W20260709 12:18:34.002097 32353 consensus_meta.cc:287] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 1.039s	user 0.013s	sys 0.001s
I20260709 12:18:33.983433 32288 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:34.034747 28284 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:34.040186 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.050028 28365 raft_consensus.cc:2376] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9. Re-sending same reply.
I20260709 12:18:34.050444 28360 raft_consensus.cc:2410] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:34.051323 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.052204 28365 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:34.053248 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.956496 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:34.014828 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.061916 32303 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.063913 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.064544 28359 raft_consensus.cc:2376] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11. Re-sending same reply.
I20260709 12:18:34.061941 32274 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.065507 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.066656 28359 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:34.066956 32274 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:34.067574 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.996743 31823 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.044405 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.068847 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:33.955075 32410 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.049196 28284 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 12:18:34.028067 32280 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.069474 28359 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:34.045071 32267 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:34.027076 28283 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:34.060348 28435 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:34.076333 28283 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:34.067616 32303 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:34.054315 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:34.073645 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.071568 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.067833 32216 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:34.079032 28364 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:34.076397 32362 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:34.078802 32216 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:34.082561 32216 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.083261 32000 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:34.083844 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.084076 32000 consensus_queue.cc:1243] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:34.084607 32405 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:34.085115 32405 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:34.085983 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.086130 32405 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.087239 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.084723 32216 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.087678 28287 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:34.088901 32344 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:18:34.092463 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:34.093511 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:34.094436 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:34.094851 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 551) took 1379 ms. Trace:
I20260709 12:18:34.095233 28286 rpcz_store.cc:276] 0709 12:18:32.715532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.715688 (+   156us) service_pool.cc:224] Handling call
0709 12:18:32.716059 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:32.721830 (+  5771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.094567 (+1372737us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b54000f2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:34.094781 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11172608}
I20260709 12:18:34.096129 32362 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:34.096725 32362 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:34.096864 32423 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:34.097483 32423 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:34.099197 32290 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:34.072669 32280 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.073123 28284 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:34.062098 32209 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.100941 32267 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.115958 28284 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:18:34.063030 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:34.116331 32280 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:34.116709 28288 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:34.117304 28284 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:34.118052 28288 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:34.119867 28291 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:34.026983 32223 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.103699 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.120867 32223 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.121649 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.122983 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.123178 32223 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:34.124234 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.124918 28359 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:34.096517 28364 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.130504 31823 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:34.131311 31823 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:34.103902 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:34.132355 31823 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.104318 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.097769 32362 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.135164 28363 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:34.136895 28363 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:34.138099 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.144330 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:34.104625 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 244606us
I20260709 12:18:34.106956 28283 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:34.086321 32385 raft_consensus.cc:993] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:34.116338 32209 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:34.072443 28365 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:34.146160 32223 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:34.104489 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:34.141402 28364 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:34.165187 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:34.146456 28359 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
W20260709 12:18:34.165459 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.148856 28358 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:34.166754 28359 pending_rounds.cc:85] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 4
I20260709 12:18:34.167193 28359 pending_rounds.cc:107] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:18:34.167579 28359 raft_consensus.cc:2889] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:34.087463 32405 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.174280 28364 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:18:34.175983 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 71588.6us
I20260709 12:18:34.176867 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:34.185703 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:34.189388 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.193490 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:34.196348 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:34.197059 28283 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:18:34.201570 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.152442 32408 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.289s	user 0.002s	sys 0.000s
I20260709 12:18:34.203045 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:34.200193 32026 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:34.204639 28364 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:34.208988 28364 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:34.210222 28462 debug-util.cc:398] Leaking SignalData structure 0x7b080034dd40 after lost signal to thread 28391
W20260709 12:18:34.210623 28462 debug-util.cc:398] Leaking SignalData structure 0x7b080039dfa0 after lost signal to thread 28392
W20260709 12:18:34.211934 28462 debug-util.cc:398] Leaking SignalData structure 0x7b080035ae80 after lost signal to thread 28465
I20260709 12:18:34.206646 28283 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
W20260709 12:18:34.213174 28462 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8060 after lost signal to thread 28466
I20260709 12:18:34.213778 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:34.214917 28462 debug-util.cc:398] Leaking SignalData structure 0x7b08003fcda0 after lost signal to thread 29015
W20260709 12:18:34.215370 28462 debug-util.cc:398] Leaking SignalData structure 0x7b080039f120 after lost signal to thread 29016
I20260709 12:18:34.215174 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.218883 28436 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:34.221349 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.222136 28432 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
W20260709 12:18:34.223901 28361 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 2007) took 5587 ms (client timeout 1566 ms). Trace:
W20260709 12:18:34.224360 28361 rpcz_store.cc:269] 0709 12:18:28.622241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.649353 (+ 27112us) service_pool.cc:224] Handling call
0709 12:18:34.209719 (+5560366us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b5400078d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:34.209892 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":761516800}
W20260709 12:18:34.200366 32419 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.310s	user 0.000s	sys 0.003s
I20260709 12:18:34.199025 32209 raft_consensus.cc:1081] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:34.227753 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:34.227418 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.232328 28432 raft_consensus.cc:2410] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:34.205586 32290 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.171224 32274 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 12:18:34.233052 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.200497 32361 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:34.201761 32315 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.205390 32368 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:34.212980 32026 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.256084 32026 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.207749 32303 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:18:34.257599 32423 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.239603 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.264103 28435 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:34.241549 32365 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:34.265858 32365 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:34.266261 32365 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:34.263317 32374 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:34.267531 32374 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:34.255769 32361 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:34.269311 32361 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.270743 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2050) took 1282 ms. Trace:
W20260709 12:18:34.272836 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 485142us
I20260709 12:18:34.274406 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:34.255795 32232 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.280386 32232 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:34.280530 32405 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.281173 32405 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.282235 28363 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:18:34.282641 28363 pending_rounds.cc:85] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 6
I20260709 12:18:34.282866 28363 pending_rounds.cc:107] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:18:34.283133 28363 raft_consensus.cc:2889] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:34.283768 32405 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:34.264746 32369 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:34.242700 28989 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:18:34.228464 32223 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:18:34.234241 28364 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.275934 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:34.246917 32344 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:34.248612 32290 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:34.230583 32366 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:34.235945 32426 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:18:34.253865 31921 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:34.254527 32368 raft_consensus.cc:993] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:34.194774 32248 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:18:34.195006 32267 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:18:34.205032 32393 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:18:34.267454 28983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 490) took 2029 ms. Trace:
I20260709 12:18:34.307108 28983 rpcz_store.cc:276] 0709 12:18:32.238254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.238434 (+   180us) service_pool.cc:224] Handling call
0709 12:18:32.950585 (+712151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.951223 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.267212 (+1315989us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b54004c18d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:34.267382 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1637027712}
I20260709 12:18:34.300330 28364 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:34.298343 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 574) took 1103 ms. Trace:
I20260709 12:18:34.323352 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:34.324520 28367 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:34.324259 28286 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:34.323810 28989 rpcz_store.cc:276] 0709 12:18:33.194609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.194756 (+   147us) service_pool.cc:224] Handling call
0709 12:18:33.195124 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:34.242444 (+1047320us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540016ae20. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7fedc0 00007f85be8891c0 00007f85be88a231 00007f85be886d8f 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c
0709 12:18:34.297989 (+ 55545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.298263 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326069632}
I20260709 12:18:34.326880 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:34.298214 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.328521 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.271210 28362 rpcz_store.cc:276] 0709 12:18:32.988490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.083809 (+ 95319us) service_pool.cc:224] Handling call
0709 12:18:33.084076 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.084297 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:33.501722 (+417425us) log.cc:844] Serialized 14 byte log entry
0709 12:18:33.795104 (+293382us) log.cc:844] Serialized 14 byte log entry
0709 12:18:34.151737 (+356633us) spinlock_profiling.cc:243] Waited 9.85 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:34.270087 (+118350us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b1800348be0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:18:34.270305 (+   218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:34.270370 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:34.270426 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.270506 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.270668 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":469682,"prepare.queue_time_us":475527,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":4335,"raft.queue_time_us":499235,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":21913,"spinlock_wait_cycles":36266752,"thread_start_us":645697,"threads_started":2,"wal-append.queue_time_us":356373}
I20260709 12:18:34.323895 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 2047) took 1358 ms. Trace:
I20260709 12:18:34.335064 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.338184 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.289685 31977 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:34.352381 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47779.1us
I20260709 12:18:34.352329 31977 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.353214 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:34.301827 32366 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:34.354588 32366 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.322549 32239 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:34.355276 32239 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:34.354919 32400 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:18:34.356415 32239 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.355399 32395 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:34.358182 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.343966 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:34.359350 28359 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:34.343972 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.358541 32395 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.303540 32368 raft_consensus.cc:1081] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:34.360261 28358 raft_consensus.cc:2376] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12. Re-sending same reply.
I20260709 12:18:34.360499 32395 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:34.361373 32410 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.333191 28366 rpcz_store.cc:276] 0709 12:18:32.965448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.967769 (+  2321us) service_pool.cc:224] Handling call
0709 12:18:34.323660 (+1355891us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400134f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:34.323818 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080386304}
I20260709 12:18:34.347034 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:34.297676 32407 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.440s	user 0.003s	sys 0.002s
I20260709 12:18:34.365310 28361 raft_consensus.cc:2410] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:34.289721 32230 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.301620 32290 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:34.299252 32221 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.302767 32426 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:34.350361 32365 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.299922 32223 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:18:34.300468 32281 consensus_queue.cc:1175] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 12 status { last_received { term: 12 index: 7 } last_committed_idx: 6 last_received_current_leader { term: 12 index: 7 } }
I20260709 12:18:34.303551 31921 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:34.328882 32177 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:34.353394 31977 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.366566 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2063) took 1242 ms. Trace:
I20260709 12:18:34.344748 28360 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:34.385677 28332 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:34.435032 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49443.4us
I20260709 12:18:34.435810 28332 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:34.416249 28285 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:34.438321 32214 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.442018 32216 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:34.442665 32216 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:34.443595 32216 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.446049 28441 rpcz_store.cc:276] 0709 12:18:33.124660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.125109 (+   449us) service_pool.cc:224] Handling call
0709 12:18:33.125470 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.125705 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:33.496935 (+371230us) log.cc:844] Serialized 14 byte log entry
0709 12:18:33.497504 (+   569us) log.cc:844] Serialized 14 byte log entry
0709 12:18:34.292721 (+795217us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b180011a400. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb7682a6 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1
0709 12:18:34.365826 (+ 73105us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b180011a400. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85be43dd70 00007f85be431293 00007f85be426cea 00007f85be42723a 00007f85be891a2a 00007f85be8a10bf 00007f85be8a1038
0709 12:18:34.366125 (+   299us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:34.366195 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:34.366245 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.366332 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.366491 (+   159us) inbound_call.cc:177] Queueing success response
0709 12:18:34.365725 (+  -766us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b180011a400. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
Metrics: {"mutex_wait_us":27113,"prepare.queue_time_us":15437,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":45983,"raft.queue_time_us":198,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":12862,"spinlock_wait_cycles":79962112,"thread_start_us":67034,"threads_started":1}
I20260709 12:18:34.394016 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:34.449162 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 55242.3us
I20260709 12:18:34.451557 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:34.452845 32228 raft_consensus.cc:2764] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:34.453724 32228 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:34.454166 32228 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:34.454567 32228 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:34.417526 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.467425 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:34.468417 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.468556 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.471273 28434 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:34.474509 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.475596 28432 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:34.477315 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.479743 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.481905 28436 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:34.490402 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.412058 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:34.496167 28432 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:34.497973 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:34.499305 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47830.9us
I20260709 12:18:34.502279 32414 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:34.505139 32414 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:34.507583 32346 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.508352 32346 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.511399 32346 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:34.398061 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.428643 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.519161 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.521986 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:34.522791 28984 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:34.386605 32120 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.384747 32000 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:34.522311 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.524686 28990 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:18:34.527166 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 308671us
I20260709 12:18:34.527325 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.527972 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:34.391185 32426 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:34.415584 31921 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:34.415871 32223 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:34.384902 32239 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.540969 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.529696 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.554093 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:34.419811 32177 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.559199 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:34.568694 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:34.572078 28441 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:34.573992 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.576049 28436 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:34.584170 28432 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:18:34.586542 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 363061us
I20260709 12:18:34.584170 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:34.391518 32225 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.138s
I20260709 12:18:34.418668 32278 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.139s
I20260709 12:18:34.418831 32400 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.128s
I20260709 12:18:34.616751 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 579) took 1220 ms. Trace:
I20260709 12:18:34.602842 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.416833 32221 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.587765 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:34.617307 28986 rpcz_store.cc:276] 0709 12:18:33.396131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.396302 (+   171us) service_pool.cc:224] Handling call
0709 12:18:33.396753 (+   451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:33.397234 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.397262 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:33.397917 (+   655us) log.cc:844] Serialized 52 byte log entry
0709 12:18:33.691439 (+293522us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:33.691484 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.691569 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:34.296835 (+605266us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:34.604662 (+307827us) raft_consensus.cc:1685] finished
0709 12:18:34.604688 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.616638 (+ 11950us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1205,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":10810,"spinlock_wait_cycles":27201408,"thread_start_us":234946,"threads_started":1,"wal-append.queue_time_us":386294}
I20260709 12:18:34.606398 28441 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
W20260709 12:18:34.646763 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 58973us
W20260709 12:18:34.642355 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 414637us
I20260709 12:18:34.583101 32248 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:34.582682 32177 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.563411 32365 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.650230 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:34.738911 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 88810.2us
W20260709 12:18:34.654320 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 70253.9us
I20260709 12:18:34.603937 32410 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:34.684274 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.741519 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:34.602682 32315 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:34.740594 28255 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:34.745330 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.785728 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.529295 32120 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.799348 28289 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:34.631246 31477 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:34.805167 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 413493us
I20260709 12:18:34.807593 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.657238 32423 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:34.894023 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47967.2us
I20260709 12:18:34.603369 32179 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.189s
W20260709 12:18:34.841372 28362 consensus_meta.cc:287] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.000s
I20260709 12:18:34.658110 32289 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.607797 32312 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.196s
I20260709 12:18:34.743438 32248 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:34.827358 32214 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:34.582275 32414 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.212s
I20260709 12:18:34.792846 32320 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:18:34.904070 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 335412us
I20260709 12:18:34.905133 28362 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:34.812326 32315 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.812520 32346 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:34.658294 32237 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.196s
W20260709 12:18:34.914391 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.894681 32423 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.921607 32404 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.658003 32368 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.194s
I20260709 12:18:34.738396 32362 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.907476 32361 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 12:18:34.962348 32371 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 12:18:34.999215 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.899637 32410 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:18:34.924901 32026 consensus_meta.cc:287] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.005s
I20260709 12:18:34.906710 31477 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.960728 28360 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:34.907951 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:35.042980 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 135111us
W20260709 12:18:35.044625 28954 leader_election.cc:336] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 1.817s (SENT)
I20260709 12:18:35.045125 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:34.920866 32232 raft_consensus.cc:2764] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:18:34.925130 32248 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:34.804399 32426 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.977346 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2028) took 1140 ms. Trace:
I20260709 12:18:34.806840 32433 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.162s
I20260709 12:18:34.801873 32223 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:34.959594 32369 consensus_meta.cc:287] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.667s	user 0.006s	sys 0.010s
W20260709 12:18:34.947345 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.953096 32346 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.633628 32290 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:34.993537 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:34.954219 32315 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.810078 31921 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:34.964756 32423 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:34.967404 32414 consensus_queue.cc:1243] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:35.120252 32216 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:35.120832 32216 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:35.031456 32362 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.123152 32412 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:35.124095 32343 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.124475 32362 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:35.115577 32450 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.126017 32450 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.127657 28364 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:35.128368 32450 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:35.128816 32120 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.129127 32000 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:35.129654 32000 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.130193 32216 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.131294 32221 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.076239 32179 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:35.133029 32179 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.135954 32179 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.072136 32435 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.075562 32405 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:18:34.992774 32404 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.060915 32393 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:18:35.046826 32410 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.114585 32445 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:35.115443 32379 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:35.036372 32403 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:35.138834 32230 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.140859 32394 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:18:35.071449 32432 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 12:18:35.076282 32346 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:35.105516 32436 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:18:35.183120 32391 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.187867 32457 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.064s	user 0.001s	sys 0.003s
W20260709 12:18:35.187542 32353 consensus_queue.cc:1175] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a3de34b6f4cf4606a61944db6e2739c8" responder_term: 11 status { last_received { term: 11 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 11 index: 10 } }
I20260709 12:18:35.239009 32403 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:35.192589 32445 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.238050 32394 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:35.274544 32394 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.276149 32394 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:35.276696 32400 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:35.277244 32400 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:35.278251 32400 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.279198 32410 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.279806 32400 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:35.258281 32438 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.339s	user 0.000s	sys 0.008s
W20260709 12:18:35.258720 32454 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 12:18:35.281570 32439 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.293s	user 0.000s	sys 0.004s
I20260709 12:18:35.284739 32426 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:35.288334 32395 consensus_meta.cc:287] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.927s	user 0.014s	sys 0.000s
I20260709 12:18:35.288939 32395 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:35.259835 32452 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:18:35.291757 32423 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.257478 32445 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.188788 32435 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.296241 32435 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.263942 32403 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.268409 32404 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:35.270704 32442 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.350s	user 0.003s	sys 0.000s
W20260709 12:18:35.354585 28255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.024s	sys 0.019s
W20260709 12:18:35.355533 28255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.025s	sys 0.020s
W20260709 12:18:35.358248 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 617652us
I20260709 12:18:35.361047 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:35.355386 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:35.364486 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:35.365640 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:34.981267 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:35.128397 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2063) took 1088 ms. Trace:
W20260709 12:18:35.373453 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 392097us
I20260709 12:18:35.374182 28364 rpcz_store.cc:276] 0709 12:18:34.040019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.177629 (+137610us) service_pool.cc:224] Handling call
0709 12:18:34.204504 (+ 26875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.127365 (+922861us) spinlock_profiling.cc:243] Waited 84.1 ms on lock 0x7b54000cfe20. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7fedc0 00007f85be889d87 00007f85be88a470 00007f85be886d8f 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c
0709 12:18:35.128092 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.128320 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193433344}
I20260709 12:18:35.375550 28289 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:35.376250 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2061) took 1688 ms. Trace:
I20260709 12:18:35.380733 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:35.383242 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.049623 28367 rpcz_store.cc:276] 0709 12:18:33.837033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.837221 (+   188us) service_pool.cc:224] Handling call
0709 12:18:34.324127 (+486906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.324992 (+   865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.977043 (+652051us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b5400134f50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:34.977245 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118998272}
I20260709 12:18:35.376870 28285 rpcz_store.cc:276] 0709 12:18:33.687602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.889407 (+201805us) service_pool.cc:224] Handling call
0709 12:18:34.020300 (+130893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.437963 (+417663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.375465 (+937502us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54003da4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:35.375921 (+   456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910071168}
I20260709 12:18:35.398865 28332 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:35.402518 28432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:18:35.403788 28432 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:35.404452 28432 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:35.406231 28441 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:18:35.402266 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:35.410357 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.411702 28983 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
W20260709 12:18:35.420611 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:35.412110 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.412190 31823 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:35.423766 31823 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:35.424505 31823 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:35.424115 28983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 496) took 1012 ms. Trace:
I20260709 12:18:35.130620 31477 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:35.425235 28983 rpcz_store.cc:276] 0709 12:18:34.411373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.411691 (+   318us) service_pool.cc:224] Handling call
0709 12:18:35.423602 (+1011911us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b54007413d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:35.423940 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2111532416}
I20260709 12:18:35.426669 31477 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.428012 28989 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:35.427700 31477 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.185230 31921 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.433077 32000 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.434719 28441 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:35.435254 28441 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.427026 31960 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:35.438128 31960 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:35.438587 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 581) took 1054 ms. Trace:
I20260709 12:18:35.439083 28989 rpcz_store.cc:276] 0709 12:18:34.383748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.419416 (+ 35668us) service_pool.cc:224] Handling call
0709 12:18:35.427443 (+1008027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.428656 (+  1213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.438236 (+  9580us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54007413d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:35.438451 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2070681472}
I20260709 12:18:35.443441 31477 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:35.444790 28186 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 218) took 39053 ms (client timeout 15000 ms). Trace:
I20260709 12:18:35.445930 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:35.051230 32026 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.142630 32216 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.465615 32274 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:35.467499 32274 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:18:35.467787 28188 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 115) took 30846 ms (client timeout 15000 ms). Trace:
I20260709 12:18:35.468034 32274 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:35.051934 32228 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.597s	user 0.008s	sys 0.004s
I20260709 12:18:35.470875 32228 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.186954 32223 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.477571 28432 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.480400 28432 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:35.482167 32170 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:35.483212 32170 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:35.449909 28186 rpcz_store.cc:269] 0709 12:17:56.390717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.391027 (+   310us) service_pool.cc:224] Handling call
0709 12:18:35.444423 (+39053396us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:21.742463 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:21.851439 (+108976us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:21.876890 (+ 25451us) write_op.cc:435] Acquired schema lock
0709 12:18:21.883544 (+  6654us) tablet.cc:662] Decoding operations
0709 12:18:22.231724 (+348180us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:22.231827 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:22.231859 (+    32us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:18:22.233170 (+  1311us) tablet.cc:701] Row locks acquired
0709 12:18:22.233215 (+    45us) write_op.cc:260] PREPARE: finished
0709 12:18:22.233435 (+   220us) write_op.cc:270] Start()
0709 12:18:22.233613 (+   178us) write_op.cc:276] Timestamp: P: 1783599502233358 usec, L: 0
0709 12:18:22.233629 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 12:18:22.249967 (+ 16338us) log.cc:844] Serialized 5248 byte log entry
0709 12:18:23.504725 (+1254758us) op_driver.cc:464] REPLICATION: finished
0709 12:18:23.651860 (+147135us) write_op.cc:301] APPLY: starting
0709 12:18:23.668231 (+ 16371us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:23.703654 (+ 35423us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:23.722781 (+ 19127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:23.856254 (+133473us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:23.856546 (+   292us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:18:23.856595 (+    49us) write_op.cc:312] APPLY: finished
0709 12:18:23.856921 (+   326us) log.cc:844] Serialized 95 byte log entry
0709 12:18:23.857118 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:23.857277 (+   159us) write_op.cc:454] Released schema lock
0709 12:18:23.857331 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:23.865560 (+  8229us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:23.875992 (+ 10432us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1897706,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":131784,"apply.run_cpu_time_us":5611,"apply.run_wall_time_us":213489,"num_ops":13,"prepare.queue_time_us":530057,"prepare.run_cpu_time_us":14374,"prepare.run_wall_time_us":634025,"raft.queue_time_us":337944,"raft.run_cpu_time_us":50198,"raft.run_wall_time_us":115014,"replication_time_us":1270918,"thread_start_us":690545,"threads_started":4,"wal-append.queue_time_us":242366,"wal-append.run_cpu_time_us":30943,"wal-append.run_wall_time_us":2429967}]]}
I20260709 12:18:35.486128 32288 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.486989 32288 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.489635 32288 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:35.440117 32000 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:35.197491 32245 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:35.492725 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:35.468308 28188 rpcz_store.cc:269] 0709 12:18:04.621751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.622158 (+   407us) service_pool.cc:224] Handling call
0709 12:18:35.467518 (+30845360us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:32.480136 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:32.480824 (+   688us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:32.480888 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:18:32.480921 (+    33us) tablet.cc:662] Decoding operations
0709 12:18:32.481815 (+   894us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:32.481931 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:32.481964 (+    33us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:18:32.482663 (+   699us) tablet.cc:701] Row locks acquired
0709 12:18:32.482716 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:18:32.482915 (+   199us) write_op.cc:270] Start()
0709 12:18:32.483083 (+   168us) write_op.cc:276] Timestamp: P: 1783599512482839 usec, L: 0
0709 12:18:32.483110 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:18:32.484014 (+   904us) log.cc:844] Serialized 2814 byte log entry
0709 12:18:33.137808 (+653794us) op_driver.cc:464] REPLICATION: finished
0709 12:18:33.807593 (+669785us) write_op.cc:301] APPLY: starting
0709 12:18:33.830041 (+ 22448us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:33.912184 (+ 82143us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:33.912335 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:33.951052 (+ 38717us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:34.002249 (+ 51197us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:18:34.002331 (+    82us) write_op.cc:312] APPLY: finished
0709 12:18:34.002791 (+   460us) log.cc:844] Serialized 59 byte log entry
0709 12:18:34.106602 (+103811us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:34.144476 (+ 37874us) write_op.cc:454] Released schema lock
0709 12:18:34.144567 (+    91us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:34.145135 (+   568us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:34.145163 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7036773,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":599825,"apply.run_cpu_time_us":28098,"apply.run_wall_time_us":357781,"num_ops":7,"prepare.queue_time_us":73818,"prepare.run_cpu_time_us":69944,"prepare.run_wall_time_us":191057,"raft.queue_time_us":300936,"raft.run_cpu_time_us":53422,"raft.run_wall_time_us":606758,"replication_time_us":636146,"thread_start_us":918518,"threads_started":5,"wal-append.queue_time_us":280865,"wal-append.run_cpu_time_us":30726,"wal-append.run_wall_time_us":2300295}]]}
I20260709 12:18:35.297502 32423 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.498757 28441 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:35.583998 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.465566 32026 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:35.607887 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:35.616686 32170 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:35.638036 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:35.625054 32228 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.313046 32445 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:35.644430 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.651770 32393 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.652660 32379 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.652756 32393 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.653450 32446 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:35.656761 32410 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:35.656559 32446 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.657373 32410 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.657903 32410 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:35.653502 32379 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.664652 32320 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:35.665526 32320 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.666383 32435 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:35.666822 32461 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:35.667182 32368 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:35.667691 32230 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.668084 32368 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.668462 32230 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.669265 32379 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.668773 32374 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:35.670065 28437 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.005s	sys 0.010s
I20260709 12:18:35.670444 32446 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.668987 32320 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.670836 32282 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:35.711791 32395 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.712127 32282 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:35.678491 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 232807us
I20260709 12:18:35.679102 32344 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.680544 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:35.713941 28332 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:35.682116 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:35.684486 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:35.246505 32391 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.686148 32458 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:35.690523 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:35.694715 32369 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:35.717043 28366 raft_consensus.cc:2410] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 11 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:18:35.717366 32368 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:35.665804 32303 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:35.722476 32369 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.722713 32303 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.698204 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:35.698429 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:35.699177 32435 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.705098 32274 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:35.712849 28437 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.005s	sys 0.010s
I20260709 12:18:35.717941 28364 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:35.713680 32344 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:35.688318 32403 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.693457 32445 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:35.715015 28367 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:35.720479 32391 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:35.785387 28434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.007s	sys 0.024s
I20260709 12:18:35.714977 32282 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:35.721894 32295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.032s	sys 0.040s
I20260709 12:18:35.696249 28362 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:18:35.716192 31977 consensus_meta.cc:287] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.194s	user 0.006s	sys 0.017s
I20260709 12:18:35.759371 28359 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.643361 32000 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.790676 28434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.008s	sys 0.024s
I20260709 12:18:35.759589 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:35.824137 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.823747 32000 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.791285 32374 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:35.817483 28954 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.010s	sys 0.014s
I20260709 12:18:35.826743 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2318) took 1247 ms. Trace:
W20260709 12:18:35.838805 28954 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.010s	sys 0.014s
I20260709 12:18:35.811057 32395 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.835608 28406 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.013s	sys 0.028s
I20260709 12:18:35.841806 32369 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:35.843333 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 129606us
I20260709 12:18:35.818310 32458 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:35.851020 28406 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.014s	sys 0.028s
I20260709 12:18:35.832777 28437 rpcz_store.cc:276] 0709 12:18:34.578942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.579074 (+   132us) service_pool.cc:224] Handling call
0709 12:18:34.579421 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.579697 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:34.580018 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.790448 (+1210430us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:35.790581 (+   133us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:35.817308 (+ 26727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.817436 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.826641 (+  9205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93200,"prepare.queue_time_us":2924,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":142783,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":15065,"thread_start_us":1169772,"threads_started":2,"wal-append.queue_time_us":495722}
I20260709 12:18:35.856245 28432 raft_consensus.cc:2410] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 12:18:35.856752 28330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.000s	sys 0.009s
W20260709 12:18:35.857161 28330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.000s	sys 0.010s
W20260709 12:18:35.838577 32385 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.031s	sys 0.013s
W20260709 12:18:35.865325 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 820693us
I20260709 12:18:35.834249 32303 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:35.865626 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:35.883972 32344 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:35.886616 32274 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.888712 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2317) took 1310 ms. Trace:
I20260709 12:18:35.818878 32461 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.891674 32000 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:35.892822 32461 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.899621 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:35.900846 28437 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:35.821275 31977 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:35.903208 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:35.903931 31977 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.885457 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:35.886018 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:35.904106 28437 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.913098 28360 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:18:35.915457 28441 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:35.856458 32435 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.865361 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:35.871001 28404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.004s	sys 0.020s
I20260709 12:18:35.917341 28359 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
W20260709 12:18:35.920953 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46468.2us
W20260709 12:18:35.923151 28404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.004s	sys 0.020s
I20260709 12:18:35.924676 28367 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:35.925393 28366 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
W20260709 12:18:35.938318 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 288291us
W20260709 12:18:35.938820 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:35.921762 32385 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.032s	sys 0.014s
I20260709 12:18:35.889256 28434 rpcz_store.cc:276] 0709 12:18:34.578478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.578622 (+   144us) service_pool.cc:224] Handling call
0709 12:18:34.632486 (+ 53864us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.632750 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:35.040268 (+407518us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.887911 (+847643us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.888199 (+   288us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:35.888278 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:35.888353 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.888450 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.888622 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12350,"prepare.queue_time_us":174041,"prepare.run_cpu_time_us":7138,"prepare.run_wall_time_us":54948,"raft.queue_time_us":17868,"thread_start_us":1081623,"threads_started":2,"wal-append.queue_time_us":825595}
I20260709 12:18:35.948859 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:35.978857 28362 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:35.980001 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:35.951996 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 1.02052e+06us
I20260709 12:18:35.919783 32393 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:35.986163 32046 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:18:35.983816 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 1.24232e+06us
I20260709 12:18:35.946074 31977 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:18:35.994555 32230 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:36.004089 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 95136us
I20260709 12:18:35.956506 32395 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.014456 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.016259 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:35.993778 32274 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.021121 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:35.998214 32458 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:36.027760 28441 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:36.028138 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.032261 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 92737.5us
I20260709 12:18:36.016923 32289 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.026485 28434 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
W20260709 12:18:36.043804 32239 consensus_meta.cc:287] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.473s	user 0.000s	sys 0.020s
W20260709 12:18:36.049672 31921 consensus_meta.cc:287] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.593s	user 0.018s	sys 0.012s
I20260709 12:18:36.054550 32274 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.057411 32274 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.060256 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:18:36.061892 32216 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.005s
I20260709 12:18:36.067116 28363 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:36.064528 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:36.044727 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:36.070509 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47433.3us
I20260709 12:18:36.035863 32297 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:36.072140 32297 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.073976 32216 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:36.075381 32297 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:36.075843 28332 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:36.076097 32297 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:36.076740 32458 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:36.062837 32395 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:36.063165 28440 raft_consensus.cc:2410] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:18:36.036298 32394 consensus_meta.cc:287] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.712s	user 0.020s	sys 0.000s
W20260709 12:18:36.064900 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.078620 32026 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.036185 32290 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.028584 32046 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.064599 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 497) took 1637 ms. Trace:
I20260709 12:18:36.079988 32290 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.079180 32026 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.082187 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.083703 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.079818 28985 rpcz_store.cc:276] 0709 12:18:34.427080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.427274 (+   194us) service_pool.cc:224] Handling call
0709 12:18:34.427696 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:35.433144 (+1005448us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b18003d8c40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85be87f060 00007f85be87fbfb 00007f85be88a64b
0709 12:18:35.433527 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.433556 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:35.447262 (+ 13706us) log.cc:844] Serialized 26 byte log entry
0709 12:18:35.914792 (+467530us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:35.915111 (+   319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.938283 (+ 23172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:36.064049 (+125766us) raft_consensus.cc:1685] finished
0709 12:18:36.064080 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.064368 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":182284,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":29063,"spinlock_wait_cycles":6464768,"thread_start_us":689947,"threads_started":2,"wal-append.queue_time_us":510050}
I20260709 12:18:36.084625 28438 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:36.088310 32216 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:18:36.070013 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.092923 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:36.035244 31823 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.610s	user 0.019s	sys 0.014s
I20260709 12:18:36.085688 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.098711 28437 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:36.100895 28362 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:36.106757 32289 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.110311 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.111045 28437 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:36.108770 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.101919 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:36.113422 28434 raft_consensus.cc:2410] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:36.113996 28432 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
W20260709 12:18:36.118233 32248 consensus_meta.cc:287] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.045s	user 0.021s	sys 0.000s
I20260709 12:18:36.118933 32248 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.121435 32248 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:36.082571 32290 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:36.118530 32365 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 1.316s	user 0.016s	sys 0.000s
W20260709 12:18:36.136502 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63854.6us
I20260709 12:18:36.138338 32289 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.138584 32365 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:36.139081 32426 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.141012 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:36.142020 28438 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 7.
W20260709 12:18:36.148241 32343 consensus_meta.cc:287] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.010s	user 0.014s	sys 0.000s
I20260709 12:18:36.149631 28433 raft_consensus.cc:2410] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:36.150369 32270 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.150956 32270 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.152580 32470 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:36.153265 32470 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.153726 32470 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:36.158983 32177 consensus_meta.cc:287] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.344s	user 0.011s	sys 0.008s
I20260709 12:18:36.159663 32177 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:36.169582 32245 consensus_meta.cc:287] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
I20260709 12:18:36.170476 32345 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.125077 32239 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:18:36.171171 32345 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.173334 28439 raft_consensus.cc:2410] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:18:36.174841 32315 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.013s	user 0.012s	sys 0.002s
I20260709 12:18:36.175042 32177 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.183023 28360 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:36.184691 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:36.169431 32400 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.889s	user 0.007s	sys 0.009s
I20260709 12:18:36.186990 28441 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:36.141373 32365 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:36.186475 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44824us
W20260709 12:18:36.188572 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 48201.7us
I20260709 12:18:36.110008 31823 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.190487 32216 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:36.195541 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 151919us
I20260709 12:18:36.195823 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.198256 28432 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:36.195258 32365 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.205482 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.207453 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.208168 28432 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:36.179484 32426 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.212157 28441 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:36.177150 32315 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.216269 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:36.217420 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 104565us
W20260709 12:18:36.212121 32366 consensus_meta.cc:287] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.840s	user 0.017s	sys 0.006s
I20260709 12:18:36.222088 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.225113 28436 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:18:36.224932 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.225927 28432 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:36.233536 28441 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:18:36.230247 32346 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.624s	user 0.016s	sys 0.000s
I20260709 12:18:36.223798 32248 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:18:36.237231 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.231936 32216 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:18:36.227276 32366 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.241356 32366 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:36.242229 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2068) took 2029 ms. Trace:
I20260709 12:18:36.244724 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:36.246290 28332 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:36.245100 28435 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:18:36.247473 28985 raft_consensus.cc:2410] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:36.247905 28435 pending_rounds.cc:85] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
W20260709 12:18:36.236045 32410 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.017s
I20260709 12:18:36.246867 28439 rpcz_store.cc:276] 0709 12:18:34.212998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.214925 (+  1927us) service_pool.cc:224] Handling call
0709 12:18:36.241960 (+2027035us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b5400346f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:36.242134 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4500113664}
W20260709 12:18:36.252213 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.247151 32481 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:18:36.250537 28435 pending_rounds.cc:107] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:36.257961 28435 raft_consensus.cc:2889] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:36.252291 32403 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:36.253082 32451 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.264906 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:36.267473 28439 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:36.270143 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:36.271586 28983 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:36.248608 32346 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.268328 32451 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.273566 28986 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:36.274963 31823 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:36.262553 32315 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:36.274606 28360 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:36.276767 28441 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:36.276211 28360 pending_rounds.cc:85] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 5
I20260709 12:18:36.254083 32410 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.278069 28360 pending_rounds.cc:107] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:36.278450 28360 raft_consensus.cc:2889] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:36.278749 32216 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:36.279281 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:36.272045 32374 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.281126 28439 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:36.286238 28990 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:36.293761 28439 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:36.297834 28983 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:36.294111 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:36.303503 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:36.300930 32413 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.306730 32413 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.291337 32403 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.315790 28990 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:36.307118 32404 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.316847 32404 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.317536 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:36.319322 32404 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.306620 32374 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.322361 32374 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.313911 32177 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:36.309120 32413 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.329396 28439 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:36.332166 32483 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:18:36.334829 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:36.336697 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.327481 31823 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.338847 28441 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:36.338272 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.341145 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:36.340441 28439 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:36.342737 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.343359 32400 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:36.341707 28990 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:36.344530 28360 raft_consensus.cc:2410] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:36.337580 32403 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.356669 28439 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 12. Ops: []
I20260709 12:18:36.360934 32400 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.364008 32400 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:36.364737 28287 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:36.366628 28439 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:36.338325 32470 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.358804 32413 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.371619 32413 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.372426 32470 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:36.338512 32177 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.383553 32177 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:36.386044 32216 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.358426 31823 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:36.353768 32374 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:18:36.378026 32346 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:36.398626 32413 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.381311 32410 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:36.386730 32216 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.395629 31823 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:36.404856 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 380559us
I20260709 12:18:36.386715 32365 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.388224 32400 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:18:36.412678 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:36.417078 32445 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:36.418224 31823 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:36.420534 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.421970 28367 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:36.422914 32445 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:36.422930 28362 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:36.425190 31823 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.417431 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:36.432986 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.435711 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 562) took 1063 ms. Trace:
I20260709 12:18:36.441426 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:36.442239 28362 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:36.442832 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:36.443423 28367 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.439994 32410 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:18:36.445781 31823 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:36.442480 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:36.441604 32365 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.439723 28287 rpcz_store.cc:276] 0709 12:18:35.372692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.373128 (+   436us) service_pool.cc:224] Handling call
0709 12:18:35.374038 (+   910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.365260 (+991222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.435371 (+ 70111us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b540009f1d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:36.435620 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2277604224}
I20260709 12:18:36.451258 28441 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:36.452798 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.441450 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.424401 32445 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.456668 32445 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.413416 32216 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.462904 32216 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:18:36.463868 28440 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2300) took 1997 ms (client timeout 2658 ms). Trace:
W20260709 12:18:36.464437 28440 rpcz_store.cc:269] 0709 12:18:34.466955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.467115 (+   160us) service_pool.cc:224] Handling call
0709 12:18:36.463522 (+1996407us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b540026c0d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:36.463778 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3666790656}
I20260709 12:18:36.439560 28358 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:36.455305 32403 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:36.455050 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2080) took 1071 ms. Trace:
I20260709 12:18:36.453656 28441 raft_consensus.cc:2368] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14.
I20260709 12:18:36.471303 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.479594 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:36.454249 32410 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:36.482199 28440 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:36.480518 28432 rpcz_store.cc:276] 0709 12:18:35.383571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.228865 (+845294us) service_pool.cc:224] Handling call
0709 12:18:36.232360 (+  3495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.251982 (+ 19622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.252017 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.265248 (+ 13231us) log.cc:844] Serialized 26 byte log entry
0709 12:18:36.324266 (+ 59018us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:36.324316 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.324386 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:36.454742 (+130356us) raft_consensus.cc:1685] finished
0709 12:18:36.454768 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.454949 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13138,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":6979,"thread_start_us":41883,"threads_started":1,"wal-append.queue_time_us":71433}
I20260709 12:18:36.485786 28362 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:36.482254 28439 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:36.486388 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.491240 32270 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:36.484315 32410 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.492936 32410 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:36.494282 32451 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:36.503610 28404 connection.cc:256] RPC call timeout handler was delayed by -1.8583550090002063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 12:18:36.504267 28404 connection.cc:256] RPC call timeout handler was delayed by -1.8336745100000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20650000000000002s for a response
I20260709 12:18:36.506112 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:36.506974 28440 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:18:36.509893 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.511765 28367 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:36.498560 28437 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:36.499356 28989 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:18:36.499567 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:36.499838 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:36.514117 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.513025 28989 pending_rounds.cc:85] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 7
I20260709 12:18:36.516112 28989 pending_rounds.cc:107] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:18:36.516417 28989 raft_consensus.cc:2889] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:36.519661 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.522015 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:36.522468 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:36.522871 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:36.524246 28362 raft_consensus.cc:2368] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:36.524098 28985 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
W20260709 12:18:36.538610 28433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2056) took 3772 ms (client timeout 2652 ms). Trace:
I20260709 12:18:36.543635 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2090) took 1744 ms. Trace:
I20260709 12:18:36.549393 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2303) took 2081 ms. Trace:
I20260709 12:18:36.552398 32046 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.562099 28441 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:36.564859 28292 rpcz_store.cc:276] 0709 12:18:34.798783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.799813 (+  1030us) service_pool.cc:224] Handling call
0709 12:18:34.800110 (+   297us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:35.038297 (+238187us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:35.070058 (+ 31761us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.886432 (+816374us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.899384 (+ 12952us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:35.900289 (+   905us) log.cc:844] Serialized 52 byte log entry
0709 12:18:35.900442 (+   153us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:35.900719 (+   277us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.900935 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.901040 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:36.381842 (+480802us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b1800042b80. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:36.543232 (+161390us) raft_consensus.cc:1685] finished
0709 12:18:36.543260 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.543461 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69258,"prepare.queue_time_us":67257,"prepare.run_cpu_time_us":5835,"prepare.run_wall_time_us":67268,"raft.queue_time_us":272133,"raft.run_cpu_time_us":589,"raft.run_wall_time_us":43798,"spinlock_wait_cycles":69095168,"thread_start_us":442600,"threads_started":2,"wal-append.queue_time_us":344936}
I20260709 12:18:36.570842 28435 rpcz_store.cc:276] 0709 12:18:34.469644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.473594 (+  3950us) service_pool.cc:224] Handling call
0709 12:18:36.244489 (+1770895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.260846 (+ 16357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.549163 (+288317us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400346f50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:36.549327 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4071524224}
I20260709 12:18:36.573474 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:36.574419 28435 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.580277 32245 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:36.580870 32245 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:36.557080 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:36.547983 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.583541 32245 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:36.984544 32245 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:36.519203 32320 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.594703 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:36.654532 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:36.567114 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:36.987465 32320 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.692121 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 2063) took 1082 ms. Trace:
I20260709 12:18:36.699077 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2316) took 2121 ms. Trace:
I20260709 12:18:36.712446 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2075) took 2296 ms. Trace:
I20260709 12:18:36.716820 28440 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:36.990381 28364 rpcz_store.cc:276] 0709 12:18:35.609806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.613348 (+  3542us) service_pool.cc:224] Handling call
0709 12:18:36.691823 (+1078475us) spinlock_profiling.cc:243] Waited 75.1 ms on lock 0x7b5400126e54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:36.691993 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172716416}
I20260709 12:18:36.993227 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.003885 28440 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:36.728740 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:36.733417 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.740187 28178 maintenance_manager.cc:643] P b5f297f7df3d4c2da68fabf56f1654bd: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 6.380s	user 0.173s	sys 0.068s Metrics: {"bytes_written":29900,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":3539,"dirs.run_cpu_time_us":571,"dirs.run_wall_time_us":1531,"drs_written":1,"lbm_read_time_us":434,"lbm_reads_lt_1ms":4,"lbm_write_time_us":99081,"lbm_writes_1-10_ms":4,"lbm_writes_10-100_ms":3,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":22168,"threads_started":2,"update_count":7,"wal-append.queue_time_us":17624}
I20260709 12:18:37.006100 28362 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.764168 32345 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:36.783002 32366 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.006306 28363 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:37.006872 32366 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.008527 28179 maintenance_manager.cc:419] P b5f297f7df3d4c2da68fabf56f1654bd: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8054 bytes on disk
I20260709 12:18:37.009369 32366 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.804880 28439 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:37.019198 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.020390 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.021004 28439 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:36.816982 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.022742 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.023624 28437 raft_consensus.cc:2410] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:37.024497 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:37.025025 28437 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:36.823731 31823 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.828217 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:37.027488 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.027931 28990 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.846022 32445 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:36.853593 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.033474 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.035593 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:36.858486 32410 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:36.993248 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:37.038964 28364 raft_consensus.cc:2410] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:36.870303 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:37.039376 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.039850 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.880107 28435 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:37.041198 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.042053 28435 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:37.042874 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.043648 28435 raft_consensus.cc:2410] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:36.892642 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:37.044785 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.911428 28432 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:36.923610 28365 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:36.924083 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.944227 32496 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:36.953161 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:36.961493 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 456516us
W20260709 12:18:37.046098 28404 connection.cc:256] RPC call timeout handler was delayed by -1.8278656640000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
W20260709 12:18:36.676867 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.971235 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.048342 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.717568 32426 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:36.993849 28436 rpcz_store.cc:276] 0709 12:18:34.577475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.577597 (+   122us) service_pool.cc:224] Handling call
0709 12:18:34.578022 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.225678 (+1647656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.698590 (+472912us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540026cfd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:36.698969 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3784621952}
I20260709 12:18:36.784216 32385 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:37.011070 28178 maintenance_manager.cc:643] P b5f297f7df3d4c2da68fabf56f1654bd: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":316,"lbm_reads_lt_1ms":4}
I20260709 12:18:37.015331 28359 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:36.567005 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:37.054296 28358 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:36.990576 28361 rpcz_store.cc:276] 0709 12:18:34.416374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.416520 (+   146us) service_pool.cc:224] Handling call
0709 12:18:34.519302 (+102782us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:35.062637 (+543335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.062680 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:35.073717 (+ 11037us) log.cc:844] Serialized 52 byte log entry
0709 12:18:35.750791 (+677074us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:35.812086 (+ 61295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.711808 (+899722us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b540036c4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:36.711866 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:36.711926 (+    60us) raft_consensus.cc:1685] finished
0709 12:18:36.711942 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.712235 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131371,"raft.queue_time_us":114128,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":8722,"spinlock_wait_cycles":651267456,"thread_start_us":460242,"threads_started":2,"wal-append.queue_time_us":476734,"wal-append.run_cpu_time_us":3779,"wal-append.run_wall_time_us":1015998}
I20260709 12:18:37.040151 28434 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:37.037377 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:36.988541 28441 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:37.046087 28367 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:37.057477 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:18:37.059605 28404 connection.cc:256] RPC call timeout handler was delayed by -1.4828175430000101s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
I20260709 12:18:37.060056 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.041121 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:37.005362 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:36.567971 28433 rpcz_store.cc:269] 0709 12:18:32.766175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.766339 (+   164us) service_pool.cc:224] Handling call
0709 12:18:36.537842 (+3771503us) spinlock_profiling.cc:243] Waited 3.36 s on lock 0x7b5400416754. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:36.538469 (+   627us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7718778240}
I20260709 12:18:37.058925 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.065773 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:37.066095 28434 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:37.067108 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:37.071604 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:37.074178 28360 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:37.064343 28956 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.076582 28435 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:37.080021 32426 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.069919 28366 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:37.081462 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:37.072927 32385 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.082840 28363 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:37.083799 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:37.083920 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:37.084614 28363 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.058406 32445 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:37.039454 32410 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:37.027578 31823 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:37.080535 32426 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.082212 28366 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:37.081387 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:37.086223 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:37.089062 28956 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.070771 32496 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.091493 32216 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.092008 32216 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.093083 32496 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.093658 32403 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:37.086902 32366 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.062491 32413 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:18:37.095155 32413 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.094923 32366 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.095671 32413 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.086926 32410 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:37.087416 31823 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:37.098994 31823 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:37.099387 31823 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.087607 32426 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.094221 32403 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.087761 32345 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:37.106703 32451 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.107194 32451 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.089591 28360 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:37.112573 32366 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:37.115549 28358 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.107563 32451 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.085727 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.117113 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:37.120319 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:37.120290 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 69143.8us
I20260709 12:18:37.122670 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.123764 32346 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.124233 32346 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.124591 32346 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.098536 32469 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.087416 32320 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:37.127477 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:37.136229 28360 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:37.135517 28983 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:37.136410 28990 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:37.139592 28362 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:37.111868 32410 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:37.139804 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.149338 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:37.141338 28363 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:37.113065 32413 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.145524 28437 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:37.146131 28367 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:37.147019 28359 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:37.160916 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:37.161926 28990 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:37.163982 28990 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:37.166132 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:37.166944 28990 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:37.173367 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:37.126777 32469 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.173265 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:37.172987 32413 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:37.179840 28985 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:37.182066 28365 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:37.181959 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.183159 28287 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:37.179217 32046 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:37.184425 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 589795us
I20260709 12:18:37.185319 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:37.192277 28289 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:37.178678 32216 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.219743 32216 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:37.181704 32469 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.203322 32320 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:37.183373 32413 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:37.187695 32410 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.215868 32046 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:37.219058 32026 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:37.233279 28366 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:18:37.239714 28285 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:37.197824 32385 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:37.240317 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 113311us
I20260709 12:18:37.245429 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.246250 28292 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:37.247442 28364 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:37.242833 32046 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:37.254549 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.234457 32289 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.269605 28289 raft_consensus.cc:2368] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10.
W20260709 12:18:37.269153 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 145340us
W20260709 12:18:37.271219 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 85992.1us
I20260709 12:18:37.254354 32345 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.275208 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:37.274631 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.278244 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.279676 28332 leader_election.cc:400] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14.
I20260709 12:18:37.281787 28332 leader_election.cc:403] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:37.278743 28285 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:37.285189 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.285514 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:37.287041 28289 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:37.288203 32345 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:37.288769 32345 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.289399 32413 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.290997 32413 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:37.272604 32403 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:37.294224 32505 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:37.294716 32505 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:37.295333 31823 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.295795 32505 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.296959 32505 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.297996 32289 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.298532 32289 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.298872 32410 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.300693 32289 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.303153 28292 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:37.303970 28190 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:37.306037 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.305693 28190 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 6 to 9, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 9 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:37.307291 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.307533 32046 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.307899 28190 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 6 to 9, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 9 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:37.309074 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.297509 31823 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:37.310191 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 64421.9us
I20260709 12:18:37.310672 32245 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:37.310909 32403 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14.
I20260709 12:18:37.311009 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:37.311280 32506 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.311765 32506 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.312081 32506 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:37.281980 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 149674us
I20260709 12:18:37.313321 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:37.315153 32245 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:37.315663 28358 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:37.306216 32346 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.322245 32500 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.323124 32403 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:37.323966 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 51949.2us
I20260709 12:18:37.268215 32426 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.326117 28285 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:37.326928 32046 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.344213 32403 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.336685 32245 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:37.306723 28287 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:37.322116 32451 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.330753 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:37.335777 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 681265us
W20260709 12:18:37.349278 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 241796us
I20260709 12:18:37.342377 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.341157 28190 catalog_manager.cc:5697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:37.346372 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:37.309713 28286 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:37.334265 32289 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:18:37.353169 28983 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:18:37.354264 32046 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:37.334132 32507 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:18:37.360925 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.363952 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.347213 32403 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:37.364810 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 84351.3us
I20260709 12:18:37.364805 28441 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:37.366442 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:37.367326 28433 raft_consensus.cc:2410] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:37.368388 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.369189 28435 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:37.328495 32506 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.370126 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.370790 28432 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:37.372161 32506 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:37.373770 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.374049 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.375031 28285 raft_consensus.cc:2410] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:37.375178 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:37.376425 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 258231us
I20260709 12:18:37.376933 28437 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.383677 32507 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:37.384253 32507 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:37.384951 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:37.385583 28433 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.385229 32507 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.385913 32508 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:37.386355 32508 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:37.386436 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.386715 32507 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.356726 32501 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.388855 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.388993 32509 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:37.390110 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.383864 28432 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:37.384421 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:37.393083 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49246.3us
I20260709 12:18:37.393533 28435 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:37.394309 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:37.387310 32508 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.397447 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:37.402366 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 60080.5us
I20260709 12:18:37.403887 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:37.391472 28190 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 6 to 9, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 9 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:37.407024 28435 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:18:37.408488 28435 pending_rounds.cc:85] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
I20260709 12:18:37.408735 28435 pending_rounds.cc:107] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:37.408959 28435 raft_consensus.cc:2889] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:37.409263 28435 pending_rounds.cc:107] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:18:37.409986 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.409494 28435 raft_consensus.cc:2889] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:37.410696 28440 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
W20260709 12:18:37.409272 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.416343 28437 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:37.426009 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.414316 32506 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:18:37.442662 28292 raft_consensus.cc:2410] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:37.415585 31823 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:37.444056 28332 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:37.444626 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:37.416548 32508 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.410111 32510 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:37.448666 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.419337 32505 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.449847 28440 raft_consensus.cc:2368] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:18:37.450465 32426 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.451236 32346 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:37.451391 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.388633 32501 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.452819 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:37.453575 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:37.454039 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:37.461440 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 93700.5us
I20260709 12:18:37.472971 28433 raft_consensus.cc:2410] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:37.453619 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.390080 32469 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.482142 32513 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.482759 32513 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.483158 32513 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.396931 32500 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:18:37.483736 32345 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.484978 32506 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:37.475512 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:37.454453 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.487244 28954 leader_election.cc:400] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10.
I20260709 12:18:37.487605 28954 leader_election.cc:403] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:37.486545 28190 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 5 to 7, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 7 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:37.489053 28284 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:18:37.489110 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 100345us
W20260709 12:18:37.490890 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46413us
I20260709 12:18:37.475982 28288 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:37.492830 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.496357 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:37.485428 32506 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.497279 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.496814 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.507438 28286 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:37.498358 28283 raft_consensus.cc:2368] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14.
I20260709 12:18:37.498823 32426 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.488703 32413 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.500286 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:37.502125 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:37.502365 32510 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:37.502700 28289 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.503129 32509 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:37.491667 32445 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:37.505236 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:37.501993 32516 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:37.512714 28291 raft_consensus.cc:2410] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:37.484094 32469 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:37.511967 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 138351us
I20260709 12:18:37.514369 28190 catalog_manager.cc:5697] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:37.506213 28287 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.514358 28433 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:37.516382 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.516111 32413 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.516992 32520 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10.
I20260709 12:18:37.514115 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:37.475514 32505 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.529846 32480 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:37.514678 32426 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.485656 32515 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.554270 32216 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.553663 28292 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
W20260709 12:18:37.557930 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 113963us
I20260709 12:18:37.563061 28289 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:37.565371 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:37.574028 28330 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:37.595710 28332 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:37.596524 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:37.513986 32445 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.509301 32506 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:37.579363 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.514520 32509 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.601349 32509 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.587496 28190 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 9 to 12, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 12 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:37.602437 32426 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:37.603055 28313 debug-util.cc:398] Leaking SignalData structure 0x7b0800338ce0 after lost signal to thread 27119
W20260709 12:18:37.604264 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8c20 after lost signal to thread 28224
I20260709 12:18:37.604877 32518 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:18:37.591384 32517 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:18:37.607550 32517 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:37.607959 32345 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:37.508759 32290 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:37.554452 32046 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.592675 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.601068 28284 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.608954 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:37.609616 28313 debug-util.cc:398] Leaking SignalData structure 0x7b080032be00 after lost signal to thread 28316
I20260709 12:18:37.604430 28288 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:37.602483 28190 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 9, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 9 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:37.588400 32516 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:37.621206 28190 catalog_manager.cc:5697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 5 to 7, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 7 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:37.619211 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.623869 28190 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
W20260709 12:18:37.624637 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 70643.7us
I20260709 12:18:37.619777 28289 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
W20260709 12:18:37.619727 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08001e3700 after lost signal to thread 28317
I20260709 12:18:37.625763 28255 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:37.626812 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08003157a0 after lost signal to thread 28391
W20260709 12:18:37.627177 28313 debug-util.cc:398] Leaking SignalData structure 0x7b0800336ae0 after lost signal to thread 28392
W20260709 12:18:37.628582 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08003031c0 after lost signal to thread 28465
W20260709 12:18:37.630189 28313 debug-util.cc:398] Leaking SignalData structure 0x7b0800248360 after lost signal to thread 29015
W20260709 12:18:37.630482 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08002e43e0 after lost signal to thread 29016
W20260709 12:18:37.631021 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08003d5fc0 after lost signal to thread 32278
W20260709 12:18:37.631886 28313 debug-util.cc:398] Leaking SignalData structure 0x7b08003e7fe0 after lost signal to thread 32409
W20260709 12:18:37.632551 28313 debug-util.cc:398] Leaking SignalData structure 0x7b0800057a40 after lost signal to thread 32436
I20260709 12:18:37.592597 32515 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.600080 32445 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.608139 32501 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.645977 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.647362 28291 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:37.608222 32517 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.633185 28283 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:37.649156 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:37.651126 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.609227 32480 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:37.661196 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.661017 28190 catalog_manager.cc:5697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 8, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 8 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
W20260709 12:18:37.662657 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 67057.4us
I20260709 12:18:37.622756 32520 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.620684 32290 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:37.618503 32519 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:37.666846 28289 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:37.671952 28288 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:37.672312 28291 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:37.673090 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:37.671850 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 71368.4us
I20260709 12:18:37.667464 28190 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:37.675122 32525 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:37.646111 32515 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.608628 32216 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.662375 32499 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:37.678293 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 178081us
I20260709 12:18:37.665720 32520 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.666728 32497 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:37.682718 28190 catalog_manager.cc:5697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:37.661638 32522 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:37.688345 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:37.688706 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.671733 32290 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.689380 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:37.693706 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:37.696440 28440 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.693612 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:37.696538 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 209448us
I20260709 12:18:37.698844 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:37.699348 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:37.695916 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.681201 32519 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:37.697548 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.702430 28283 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:37.689093 32499 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.692804 32501 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:37.691347 32520 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.727983 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:37.729372 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:37.724541 28190 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 7, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 7 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:18:37.730214 28441 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 13: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:37.730985 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:37.734191 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:37.739326 28440 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:37.741768 28954 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:37.721508 32499 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.727453 32345 raft_consensus.cc:2764] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:18:37.727916 32469 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:37.736158 32514 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:18:37.749840 28286 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:18:37.750859 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 74499.2us
I20260709 12:18:37.713486 32320 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:18:37.766515 32320 consensus_queue.cc:1243] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:37.752189 32508 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:18:37.755568 32245 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:37.762580 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 72999.1us
I20260709 12:18:37.771085 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:37.773161 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:37.701983 32516 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.827910 28287 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:37.748952 32498 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:37.830914 32498 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.831378 32498 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.684440 32510 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.835033 32513 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.720404 32529 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
W20260709 12:18:37.756415 32524 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:18:37.839613 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 96868.1us
I20260709 12:18:37.753439 32426 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:37.774535 32413 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.745615 32501 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:37.840442 28330 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.595s (SENT) [suppressed 26 similar messages]
I20260709 12:18:37.880811 28190 catalog_manager.cc:5697] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 2 to 9. New cstate: current_term: 9 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:37.880350 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:37.887068 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 113977us
I20260709 12:18:37.888495 32413 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:37.837059 32512 raft_consensus.cc:2764] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:37.837441 32518 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:37.837276 32523 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:37.839499 32497 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:37.895866 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:37.891345 28190 catalog_manager.cc:5697] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 6 to 8, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 8 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:37.878873 32426 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.889477 32413 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.889786 32469 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:37.906788 28289 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.906970 32426 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:37.912048 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 213293us
I20260709 12:18:37.913049 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:37.889878 32531 raft_consensus.cc:3055] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:37.919032 32541 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:37.928838 28190 catalog_manager.cc:5697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:37.930140 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:37.912858 32497 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:37.943670 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 114444us
I20260709 12:18:37.895975 32525 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:37.961171 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:37.751312 32480 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.960860 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:37.970614 28285 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:37.908329 32469 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:37.942317 32046 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:37.957451 32507 consensus_meta.cc:287] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.570s	user 0.013s	sys 0.000s
I20260709 12:18:37.971067 28285 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:37.947788 32541 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:37.960268 32525 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:37.966599 32530 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.945130 32531 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:37.969385 32216 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:37.945905 32413 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.981552 28190 catalog_manager.cc:5697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:37.985008 28289 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:37.980772 28287 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:37.972070 32523 raft_consensus.cc:2764] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:18:37.981613 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 239939us
I20260709 12:18:37.988770 28284 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:37.937994 32522 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:18:37.991704 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 84196.1us
I20260709 12:18:37.993477 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:37.994167 28289 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.995187 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:37.996112 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.996876 28284 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:37.994632 28285 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:37.987854 32245 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:38.010171 32451 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.012338 28285 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.012941 32445 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.899854 32519 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.014325 32531 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.018328 32505 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.004510 32538 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:18:38.019282 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:38.018330 32403 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.622s	user 0.005s	sys 0.007s
I20260709 12:18:38.049299 32403 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.051904 32403 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:38.054275 32531 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:38.053969 32541 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.055413 32541 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.049438 32506 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:37.998561 32530 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.057981 32469 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.059341 32469 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:38.059917 32530 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:38.060532 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.042936 28283 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:38.021230 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.062681 28286 raft_consensus.cc:2368] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
W20260709 12:18:38.063777 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.063620 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.064412 32525 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.066090 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.066751 28286 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:38.067626 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:38.006585 32501 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.002465 32545 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:38.028849 28289 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:38.070647 32538 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:38.071065 32538 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:38.071606 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.071986 32538 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.073369 32538 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:38.075824 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 235394us
I20260709 12:18:38.077520 28440 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 13: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:38.080391 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:38.081976 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.086658 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:38.087387 28290 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:18:38.090368 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.092164 32498 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.093107 32413 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:38.093428 28288 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:38.068137 28286 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:18:38.094627 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 71030.1us
W20260709 12:18:38.036414 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 64674.7us
W20260709 12:18:38.038512 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 125564us
I20260709 12:18:37.992519 32510 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:38.042692 32537 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:38.069694 32513 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.061956 32506 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:38.101356 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.116494 28440 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
W20260709 12:18:38.118358 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.119438 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.094417 28286 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:38.095422 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:38.102811 28285 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:38.100124 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:38.104427 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.104905 32537 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:38.110745 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:38.109789 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:38.110637 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 86096.8us
I20260709 12:18:38.115029 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:38.092201 32533 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.327s	user 0.000s	sys 0.002s
I20260709 12:18:38.125757 28441 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:38.130347 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.131356 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.136961 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.145255 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.147018 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:38.105180 32517 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.147768 28438 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
W20260709 12:18:38.156025 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55964.4us
I20260709 12:18:38.151048 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.150794 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:38.151271 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:38.157356 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.143666 28284 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:38.148061 28984 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:38.162269 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:38.149935 28286 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.172365 28286 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:18:38.172544 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.173758 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.149158 32403 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.175343 32403 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.176692 32426 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.177819 32403 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.183465 32499 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.163810 31823 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:38.185957 31823 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.186472 32537 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.188025 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:38.188222 32537 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.189860 32510 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:38.190099 28291 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:38.190884 32523 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.143649 32500 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:38.186379 31823 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.143293 32505 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:38.197922 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.194970 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:38.202396 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:38.203663 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:38.147574 32555 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:38.217886 32550 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:38.218127 32506 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:38.218497 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 49903.5us
I20260709 12:18:38.221244 31823 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.228276 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:38.224938 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.190908 32510 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.092743 32515 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.191447 32500 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:38.262437 28364 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:38.263238 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:38.231370 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:38.264686 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.265501 28435 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:38.204061 32320 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:38.209601 32469 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:38.274653 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:38.307826 28332 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:38.272418 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44256.8us
I20260709 12:18:38.309337 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.224233 32480 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.226727 32545 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:38.311604 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:38.227144 32451 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:38.226243 32413 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.226297 32550 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:38.260834 32290 consensus_meta.cc:287] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.531s	user 0.012s	sys 0.000s
I20260709 12:18:38.312729 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:38.272593 32385 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:38.319465 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:38.323545 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:38.333361 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:38.257865 32046 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.209239 32410 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 12:18:38.333807 28360 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:38.326452 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63804.2us
I20260709 12:18:38.274106 32516 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.210758 32505 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.318756 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:38.276031 32500 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.273911 32320 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:38.313611 32469 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:38.348775 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:38.318246 32289 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:38.355679 28365 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.353835 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:38.261030 32506 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.318712 32545 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.356659 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:38.316215 32498 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:38.431067 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.311560 32517 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:38.362730 28362 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:38.330374 32550 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:38.398586 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.367167 28290 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:38.332876 32245 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.340153 32544 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:38.439596 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 79507.9us
I20260709 12:18:38.353957 32216 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:18:38.443948 32509 consensus_meta.cc:287] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.816s	user 0.013s	sys 0.000s
I20260709 12:18:38.366011 32289 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:38.445710 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:38.446291 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 214839us
I20260709 12:18:38.361549 32522 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:38.360054 32320 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:38.366747 28954 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:38.452050 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 95464.8us
I20260709 12:18:38.467866 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.436838 32480 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:38.337649 32290 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.433770 32517 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.363372 32469 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.434448 32545 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.354182 32451 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:38.331836 32413 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.440431 32535 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.359290 32516 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.457890 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:38.447888 32520 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.704s	user 0.012s	sys 0.000s
I20260709 12:18:38.455703 32544 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.499091 32544 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.459348 32501 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:38.462031 32509 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.459113 32445 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.441233 32539 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.198s
I20260709 12:18:38.502105 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:38.503947 32509 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.503770 32520 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.505983 28435 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:38.507810 32520 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.509580 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.511219 32515 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:38.512704 32506 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:38.486668 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.515883 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:38.517242 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:38.518049 32544 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.522295 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.524065 31823 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.485102 32426 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.527761 32499 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.472899 32525 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.441916 32527 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:18:38.478852 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:38.479475 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:38.463588 32289 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.484879 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:38.550022 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63475.1us
I20260709 12:18:38.550540 28358 raft_consensus.cc:2410] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:18:38.549721 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 79532.2us
I20260709 12:18:38.464556 32522 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.360509 32046 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.490999 28363 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:38.491671 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:38.553426 28360 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:38.533352 28983 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:18:38.555280 28437 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:38.503533 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:38.468984 32480 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:38.538187 28286 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:38.539719 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.540017 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.476405 32517 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.476471 32565 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:38.560578 32553 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:18:38.560969 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:38.562964 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.563566 28434 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.558440 28433 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:38.544373 28364 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:38.520292 32544 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.566517 28439 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:38.548662 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.566883 32565 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:38.568190 32565 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:38.570256 32564 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 12:18:38.563269 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:38.574690 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:38.575352 28364 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.497331 32546 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:38.485673 32554 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:38.558702 28440 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.552078 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:38.584693 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.548566 32385 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:38.542510 32501 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:38.601204 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:38.602103 28360 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.603619 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:38.559227 32522 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.560333 32480 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:38.557624 32499 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:38.573895 32498 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:18:38.619796 28363 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [7.5-11.7]
I20260709 12:18:38.621482 28365 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
W20260709 12:18:38.622546 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 70537.7us
I20260709 12:18:38.624372 28985 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:38.596040 32554 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:38.628124 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 261471us
I20260709 12:18:38.629987 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:38.597847 32403 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:38.530494 32535 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.642133 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:38.643349 32566 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:38.643939 32566 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:38.644855 32566 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.645460 28440 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:38.648166 28257 leader_election.cc:400] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:18:38.648566 28257 leader_election.cc:403] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:38.649925 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.651933 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:38.595858 32245 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:18:38.620543 32497 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.657253 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:38.657163 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:38.658771 28989 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:38.657099 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:38.667338 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 53478.8us
I20260709 12:18:38.668499 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:38.672752 28367 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:38.672465 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.522398 32451 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.679061 28360 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:38.633644 32554 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:38.675189 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:38.693498 28358 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 12. Ops: []
I20260709 12:18:38.679317 32506 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:18:38.690478 32385 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.695683 28440 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:38.676671 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.698335 28363 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:38.694317 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:38.704398 32216 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:38.705076 32216 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:38.705261 32515 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:38.706419 32216 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.716778 28990 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:38.660301 32505 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.639739 32410 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:18:38.725291 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:38.730343 28360 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.677095 32573 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:38.732484 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55899.7us
I20260709 12:18:38.733465 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.670387 32539 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:18:38.741134 28367 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:38.655874 32403 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.692490 32499 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:38.798323 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:38.735525 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 83653.9us
I20260709 12:18:38.697788 32544 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:38.731341 32575 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:18:38.796029 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 120924us
W20260709 12:18:38.745147 32541 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.675s	user 0.014s	sys 0.000s
I20260709 12:18:38.807425 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.812991 28434 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
W20260709 12:18:38.730450 32562 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 12:18:38.806933 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:38.724560 32519 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.809820 28954 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.814736 28332 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:38.793919 32505 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.816071 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.840902 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.841411 28440 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:38.842015 28285 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:38.798066 32499 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:38.742406 32573 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.820650 28360 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:38.806937 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:38.720934 32509 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
W20260709 12:18:38.825187 32538 consensus_meta.cc:287] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.729s	user 0.010s	sys 0.000s
I20260709 12:18:38.826048 31823 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.825623 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.859705 28286 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:38.826798 32403 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:38.826166 32569 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 12:18:38.834236 28366 raft_consensus.cc:2410] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:18:38.835139 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.674363 32497 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.848893 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.849069 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:38.802193 32290 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:38.852694 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.735445 32523 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 12:18:38.852489 32216 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:38.855778 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.856447 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.810570 32413 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.858421 28982 raft_consensus.cc:2368] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:18:38.860601 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.807049 32579 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:38.886739 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:38.869437 32289 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.816998 32544 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.871465 32578 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.875159 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.875698 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:38.696604 32501 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.854650 32499 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.884012 32519 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:18:38.894586 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 161217us
W20260709 12:18:38.871098 32572 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 12:18:38.903524 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.904521 28440 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:18:38.943030 28440 pending_rounds.cc:85] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 6
I20260709 12:18:38.943315 28440 pending_rounds.cc:107] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:18:38.853829 32573 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.903604 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.906170 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.944011 28438 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.874238 32245 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.807420 32580 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 12:18:38.916564 28292 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:38.917963 28434 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [7.5-11.7]
W20260709 12:18:38.918857 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 125472us
I20260709 12:18:38.919895 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.922250 28441 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:38.905411 32546 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:38.904304 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:38.884223 28284 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:38.928923 28362 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:38.931740 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 56133.7us
I20260709 12:18:38.932176 32289 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.928431 32445 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:38.933038 32413 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:38.936270 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:38.960127 28287 raft_consensus.cc:2376] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12. Re-sending same reply.
I20260709 12:18:38.854417 32505 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.942309 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:38.943552 28440 raft_consensus.cc:2889] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:38.949680 28432 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:38.950011 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.896214 32579 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.956350 28288 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:38.956276 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.968717 28290 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:38.905236 32516 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.903743 32216 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:38.971486 28363 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:38.974258 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:38.969924 32525 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.975279 32525 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:38.976624 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:38.977535 32525 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:38.978153 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:38.979064 32586 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:38.970080 28285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "114fa146124747358c997ac1008e1b93"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:38.984467 28285 raft_consensus.cc:606] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:38.984880 28285 raft_consensus.cc:614] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:38.988493 32586 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.963488 32413 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.989663 32413 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.990900 32497 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.994382 32046 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:38.987629 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:38.997270 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:38.998334 32216 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:38.998474 32586 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:39.000452 32585 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:38.970716 32579 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.014617 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:39.015904 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:39.016757 28292 raft_consensus.cc:2410] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:39.017370 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:39.018788 32527 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.019006 28288 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:39.020054 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:39.020727 28288 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:39.021507 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:39.022771 28288 raft_consensus.cc:2376] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12. Re-sending same reply.
I20260709 12:18:39.023258 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:39.025095 28292 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 8.
I20260709 12:18:39.024441 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:39.026610 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:38.975608 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:39.027274 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.027737 28366 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
W20260709 12:18:38.958844 32550 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.000s
W20260709 12:18:39.033532 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59964us
I20260709 12:18:39.033857 28364 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:38.973827 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:39.037258 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:38.975492 32570 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:18:39.041880 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:39.047689 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:39.053699 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.042567 28288 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:18:39.058171 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.024461 32469 consensus_meta.cc:287] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.001s
W20260709 12:18:39.060544 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.067822 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 94123.1us
W20260709 12:18:39.037827 32562 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:18:38.971729 32506 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.129s
I20260709 12:18:39.125149 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:39.126907 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:39.299587 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 174583us
I20260709 12:18:39.071427 28436 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:18:39.299942 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:39.198328 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:39.302048 28364 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:39.136334 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:39.070070 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:39.303573 28432 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
W20260709 12:18:39.230871 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 233660us
I20260709 12:18:39.116923 28290 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:39.013500 32320 consensus_meta.cc:287] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.000s
I20260709 12:18:39.306761 28434 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:39.244812 28438 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:39.303126 28284 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:39.305722 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:39.078970 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:39.071488 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:38.973618 32546 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:39.070107 28330 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:39.309747 28287 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:39.312215 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:39.212378 32555 consensus_meta.cc:287] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.951s	user 0.007s	sys 0.005s
W20260709 12:18:39.213985 32531 consensus_meta.cc:287] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 1.159s	user 0.013s	sys 0.008s
W20260709 12:18:39.199582 32545 consensus_meta.cc:287] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.000s
W20260709 12:18:39.329665 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.256955 32537 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.068s	user 0.013s	sys 0.000s
W20260709 12:18:39.113699 32500 consensus_meta.cc:287] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.681s	user 0.013s	sys 0.000s
I20260709 12:18:39.233353 32046 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.111610 32567 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:39.334278 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:39.222061 32245 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:39.336313 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:39.255290 32497 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.338654 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:39.065043 32413 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.339740 28982 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 7.
I20260709 12:18:39.310978 32516 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.313124 32507 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:39.310953 32410 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:39.342921 28287 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:39.344029 32541 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:39.344620 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2094) took 1327 ms. Trace:
W20260709 12:18:39.355013 32522 consensus_meta.cc:287] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.004s
I20260709 12:18:39.350798 32537 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.113184 32544 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.172713 32573 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.196947 32520 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.357177 32507 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.336014 32555 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:39.337147 32385 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:39.347162 32545 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.311147 32498 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:18:39.380379 32480 consensus_meta.cc:287] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.753s	user 0.012s	sys 0.000s
I20260709 12:18:39.038046 32517 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.353281 32290 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:39.398880 32498 consensus_queue.cc:1243] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:39.399889 32385 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:39.400899 32385 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.402087 32535 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.402365 32573 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.387215 32507 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.386994 32520 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.407941 32520 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:39.411273 32517 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:39.356184 32245 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:39.350068 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 280093us
I20260709 12:18:39.358482 32497 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.311573 32469 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.437224 28288 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 7: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:39.368033 32522 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.439420 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.440378 32499 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.441380 32522 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.360529 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:39.069149 32569 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.274s
W20260709 12:18:39.383335 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 72004.9us
I20260709 12:18:39.383971 32537 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.057004 32550 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.313251 32580 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:18:39.351038 32046 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.365571 32410 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:39.349807 32500 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.355430 32578 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:18:39.314664 32320 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.419137 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:39.421056 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:39.351989 32567 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.481980 28363 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.421799 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:39.425663 32290 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:39.414661 32480 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.426826 32413 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.494205 28986 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:39.427597 28286 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:39.429195 32545 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:39.431991 32554 consensus_meta.cc:287] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.702s	user 0.004s	sys 0.006s
I20260709 12:18:39.442790 32499 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:39.442917 32555 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:39.441268 32469 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.361253 28289 rpcz_store.cc:276] 0709 12:18:38.017658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.114427 (+ 96769us) service_pool.cc:224] Handling call
0709 12:18:38.114802 (+   375us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:38.115242 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:38.115274 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:38.131366 (+ 16092us) log.cc:844] Serialized 52 byte log entry
0709 12:18:38.306168 (+174802us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:38.328334 (+ 22166us) log.cc:844] Serialized 14 byte log entry
0709 12:18:38.361782 (+ 33448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.361905 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.868567 (+506662us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b18003d5880. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a522 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:39.343345 (+474778us) raft_consensus.cc:1685] finished
0709 12:18:39.344371 (+  1026us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.344552 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6517,"prepare.queue_time_us":19445,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":5347,"raft.queue_time_us":110328,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":25997,"spinlock_wait_cycles":6857984,"thread_start_us":141823,"threads_started":2,"wal-append.queue_time_us":185787}
I20260709 12:18:39.515245 32500 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:39.497915 32578 consensus_queue.cc:1243] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:39.486148 32567 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.387326 32546 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.358397 32516 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.444429 32245 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:39.413252 32579 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.447523 32505 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.453935 32497 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:39.471846 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 111402us
I20260709 12:18:39.480875 28435 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:39.478332 28364 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:39.542258 28435 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:39.485249 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64312.9us
I20260709 12:18:39.544077 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:39.473654 32589 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:39.498180 28291 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:39.498908 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:39.500005 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.556190 28437 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:39.474813 32535 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.506304 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.506705 28434 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:39.558717 28361 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:18:39.506944 28283 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:39.500557 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:39.508104 32554 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.511026 32320 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:39.511982 32410 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.512702 32046 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:39.508287 32555 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:39.481467 32506 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.569985 32506 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.570302 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.570477 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:39.570915 28360 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:39.567729 28435 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.573619 28435 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.569989 32538 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:18:39.575142 32538 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:39.575510 32538 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:39.576368 32538 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:39.578069 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 77597.3us
I20260709 12:18:39.578682 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:39.579373 28358 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 6.
I20260709 12:18:39.579610 32541 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:39.579946 28359 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:39.581066 32554 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.581293 32290 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.547416 28983 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:39.558444 32578 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:39.548115 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:39.582530 32578 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.583500 28990 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:39.551314 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:39.584447 28364 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
W20260709 12:18:39.553884 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49387.7us
I20260709 12:18:39.585434 28988 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:18:39.586297 32413 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:39.580039 32541 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.497093 32522 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.562819 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:39.587191 32541 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:39.587940 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:39.557328 28287 raft_consensus.cc:606] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:39.588723 32510 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.589529 32523 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:39.589254 32510 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.591146 32480 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.592013 32510 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:39.592290 32245 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:39.595531 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:39.596071 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:39.596776 28364 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:39.567153 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.560246 32505 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.568979 32555 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.494107 32507 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.608392 28360 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:39.612397 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:39.558101 32500 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:39.584812 32578 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.569571 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:39.598860 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:39.566547 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.525517 28363 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:39.578331 28366 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:39.577594 32538 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.570204 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.602283 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:39.603937 28437 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:39.606675 28435 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:39.565626 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:39.591071 32523 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:39.647809 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.619585 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:39.617654 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:39.620430 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:39.623165 32566 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:39.623397 32597 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.060s	user 0.000s	sys 0.002s
W20260709 12:18:39.624222 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 58692.2us
I20260709 12:18:39.661206 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:39.619632 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:39.629536 32592 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:39.602751 32537 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:39.633837 32594 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.235s	user 0.000s	sys 0.002s
I20260709 12:18:39.634353 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 469) took 1001 ms. Trace:
I20260709 12:18:39.566901 32535 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.636664 32579 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.620518 32583 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:39.638881 28291 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:39.614591 32505 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.637498 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:39.641184 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 71693us
I20260709 12:18:39.641721 32554 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:39.642234 32510 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.599493 32589 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:39.646004 32541 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.641621 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 585) took 1317 ms. Trace:
I20260709 12:18:39.657616 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.662263 28437 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:39.664487 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:39.700111 28435 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:39.664656 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:39.637636 32516 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.618221 32506 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:39.693812 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 253668us
I20260709 12:18:39.704294 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:39.697888 28283 rpcz_store.cc:276] 0709 12:18:38.324053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.324207 (+   154us) service_pool.cc:224] Handling call
0709 12:18:38.354775 (+ 30568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:39.564080 (+1209305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.641341 (+ 77261us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000affd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:39.641546 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2649299840}
I20260709 12:18:39.682734 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 2097) took 1193 ms. Trace:
I20260709 12:18:39.694578 28330 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:39.705574 32516 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:39.705999 32516 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.706383 32516 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.707480 32566 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.708819 32566 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:39.709461 32506 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:39.710103 32506 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:39.711095 32506 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.711804 32589 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.713929 32501 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:39.715610 32541 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.677882 32567 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.692771 32592 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:39.705013 28359 rpcz_store.cc:276] 0709 12:18:38.488997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.503261 (+ 14264us) service_pool.cc:224] Handling call
0709 12:18:39.682477 (+1179216us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400125f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:39.682655 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353507584}
I20260709 12:18:39.615449 32520 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:18:39.675800 32565 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:39.675940 32426 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:39.703385 28255 leader_election.cc:400] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:18:39.699193 28439 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.719831 28330 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:39.689833 32517 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.677986 28986 rpcz_store.cc:276] 0709 12:18:38.632836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.632954 (+   118us) service_pool.cc:224] Handling call
0709 12:18:39.634127 (+1001173us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b540046e954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:39.634279 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905700224}
I20260709 12:18:39.709619 32566 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:39.752092 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49012.1us
I20260709 12:18:39.717767 32567 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:39.766319 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 63078.2us
W20260709 12:18:39.766940 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47227.2us
I20260709 12:18:39.666277 32500 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.767943 28362 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:39.784782 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:39.760555 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:39.765794 32046 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:18:39.785594 28985 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 7.
I20260709 12:18:39.691754 32583 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.771456 32517 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.704101 32510 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.790017 32517 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.790525 32510 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.794088 32550 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:39.795468 32507 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.797894 32583 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:39.807209 32469 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.807839 32469 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.808588 32535 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.769482 32565 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:39.809971 32565 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:39.811493 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.811694 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.812018 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.812239 28360 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.812187 28358 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.812633 28438 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.817361 32505 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.769364 32537 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:39.765777 32426 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.819487 32426 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:39.820760 28358 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:39.820899 28360 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:39.762439 32499 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:18:39.772156 32566 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:39.823431 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:39.824631 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:39.768427 32385 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:18:39.790614 32579 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:18:39.676702 32518 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 12:18:39.786020 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:39.697065 32555 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:39.717391 32592 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.779116 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:39.828413 28283 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:39.831255 28439 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
W20260709 12:18:39.829248 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 68772.1us
W20260709 12:18:39.829129 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 46181.6us
I20260709 12:18:39.830178 28360 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:39.826398 32569 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:39.831897 32541 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:39.831615 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 52573.3us
I20260709 12:18:39.837905 28438 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:39.833297 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:39.842175 28956 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:39.842610 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:39.846521 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:39.857111 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 237564us
I20260709 12:18:39.859347 28404 leader_election.cc:400] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:39.830789 32520 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:39.833360 32499 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:39.867329 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.923525 28404 leader_election.cc:403] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:39.925401 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.926363 28439 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:39.927630 28439 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:39.886989 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:39.929397 32289 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:39.929824 32289 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.929946 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:39.930218 32289 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:39.932019 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:39.889868 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.892220 28358 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:39.934242 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.935011 28358 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:39.892422 28363 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:39.893103 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.937249 28365 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:39.937819 32245 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:39.938695 32289 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:39.857020 32537 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:39.942078 32537 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.942605 32537 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.847314 32541 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.946252 32541 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.898285 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.949620 32289 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:39.950263 28361 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:39.871340 32451 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.030s	sys 0.053s
W20260709 12:18:39.951251 32451 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.031s	sys 0.053s
I20260709 12:18:39.901911 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:39.952170 28360 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:39.904338 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.856532 32567 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:18:39.897512 32518 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:39.921144 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:39.954510 28433 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:39.937655 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 78498.3us
I20260709 12:18:39.955065 32510 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.955993 32510 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:39.956154 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:39.956394 32510 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:39.956872 32510 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:39.957990 32523 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:39.901292 32469 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:39.958489 32523 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.958869 32523 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:39.959511 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:39.953661 28438 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:39.895802 32535 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:39.961212 32413 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:39.963341 32245 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:39.965431 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:39.888674 32545 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 12:18:39.973254 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 90907.9us
I20260709 12:18:39.976073 28255 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:39.977290 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:39.973409 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:39.904269 32579 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:39.984911 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:39.903836 32320 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:18:39.906478 32505 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.986945 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:39.996565 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:39.977593 28365 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:39.993300 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.000101 28291 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.003355 28287 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:18:39.999420 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 157353us
I20260709 12:18:40.004796 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:39.974579 32499 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:40.009101 32539 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:18:40.009358 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:39.994100 32289 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.010316 28360 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:40.001965 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.013525 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:40.013352 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:40.015637 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.016191 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:40.016952 28358 raft_consensus.cc:2368] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
I20260709 12:18:40.016292 28287 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:40.018008 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.018626 28289 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:18:40.019515 28285 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:40.021149 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.021914 28285 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:40.022293 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.022616 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.014139 28985 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:40.014269 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:39.984807 32579 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.990407 32569 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:39.993498 32501 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:40.028887 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:40.039373 28363 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:40.031018 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:40.029335 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:40.040874 28365 raft_consensus.cc:2410] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:40.037958 28290 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.032600 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.044607 28283 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.984229 32520 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:39.991374 32497 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:40.032435 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:39.994402 32498 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:39.990608 32538 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:18:39.993891 32518 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:40.074262 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:40.035275 32579 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:40.032943 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.079437 28288 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.010900 32517 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:40.048552 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:40.013752 32413 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:40.050215 28986 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.081338 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:40.082219 28363 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:39.998505 32523 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.028918 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.026005 32289 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:39.993595 32535 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:18:40.068244 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:39.994275 32451 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:18:40.010206 32499 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.011596 32506 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:40.044413 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:40.074263 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:40.092272 28989 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 12:18:40.078109 28404 connection.cc:256] RPC call timeout handler was delayed by -2.4004092719999335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
I20260709 12:18:40.080408 28982 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:40.013056 32578 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:18:40.010581 32505 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.050716 32546 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.048549 32410 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:40.101646 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:40.102634 28360 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:40.087803 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:40.045347 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:40.055491 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.025590 32544 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:18:40.051941 32497 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.033917 32589 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:18:40.085536 32289 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:40.108369 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:40.113166 28361 raft_consensus.cc:2410] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:40.111217 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:40.117148 28989 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:18:40.093370 28404 connection.cc:256] RPC call timeout handler was delayed by -2.3707092719998855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
I20260709 12:18:40.095898 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.099107 28363 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:40.082827 32245 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.103384 28988 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:40.085721 32535 consensus_queue.cc:1243] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:40.079845 32498 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:40.117764 28404 connection.cc:256] RPC call timeout handler was delayed by -2.3212092719998054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
W20260709 12:18:40.135716 28404 connection.cc:256] RPC call timeout handler was delayed by -2.2510092719999193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 12:18:40.136063 28404 connection.cc:256] RPC call timeout handler was delayed by -2.1898092719998203s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:18:40.136351 28404 connection.cc:256] RPC call timeout handler was delayed by -2.0080092719999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260709 12:18:40.136636 28404 connection.cc:256] RPC call timeout handler was delayed by -1.9918092719999549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
W20260709 12:18:40.136901 28404 connection.cc:256] RPC call timeout handler was delayed by -1.9342092719998618s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 12:18:40.137161 28404 connection.cc:256] RPC call timeout handler was delayed by -1.9189092719998371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:18:40.137444 28404 connection.cc:256] RPC call timeout handler was delayed by -1.7803092719998403s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
W20260709 12:18:40.137703 28404 connection.cc:256] RPC call timeout handler was delayed by -1.6219092719998116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2388s for a response
W20260709 12:18:40.134681 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 89493.5us
W20260709 12:18:40.089882 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 231341us
I20260709 12:18:40.106370 32578 consensus_queue.cc:1243] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:40.103399 32506 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:40.118396 28290 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:40.190299 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.108253 32410 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:40.194123 28289 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:40.191917 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:40.110771 32497 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.197013 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:40.110972 32289 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:40.209177 28284 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.211644 28330 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:40.219322 32517 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:40.220528 32621 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:18:40.222658 32537 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.223093 32505 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.224216 32573 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:40.224609 32573 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:40.224817 32537 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.226248 32216 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:40.226621 32216 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:40.227425 32216 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.228658 32216 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:40.228937 32612 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:18:40.231122 32498 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:40.235528 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.235246 32506 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.237679 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:40.240935 28436 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:40.238121 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:40.242290 28434 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:40.237922 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:40.242865 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.243336 28437 raft_consensus.cc:2410] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:40.238317 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:40.244150 28435 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:40.103932 32546 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.238750 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:40.246479 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:40.243495 28286 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:40.215708 28989 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:40.216166 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:40.242494 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.252300 28289 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.247605 28291 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:40.115108 32569 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.114781 32583 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.260069 32583 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:40.260931 28988 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:40.193115 32245 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.264153 28436 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:40.248957 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:40.266726 32538 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:40.267180 32538 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.267544 32538 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:40.268673 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:40.269788 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:40.270465 32523 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:40.270720 32522 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:40.271133 32522 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.271492 32522 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.272826 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2137) took 1227 ms. Trace:
I20260709 12:18:40.273869 32539 raft_consensus.cc:2764] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:18:40.275725 32619 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:18:40.276296 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:40.277190 32544 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:40.277017 32541 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.273196 28292 rpcz_store.cc:276] 0709 12:18:39.016665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.027248 (+ 10583us) service_pool.cc:224] Handling call
0709 12:18:40.244265 (+1217017us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b5400163d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:40.244397 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1619981696}
I20260709 12:18:40.138778 32426 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.282295 32426 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:40.134291 32516 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.284586 32516 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.286856 32579 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.287211 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:40.287863 28358 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.287979 28434 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:40.288232 32544 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:40.225404 32573 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.291373 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.291807 32573 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.292469 32507 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:40.292850 32507 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:40.293726 32507 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.294607 32579 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:40.294368 32410 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.296501 28287 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:18:40.296942 28287 pending_rounds.cc:85] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 5
I20260709 12:18:40.297174 28287 pending_rounds.cc:107] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:40.298198 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.298879 28290 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:40.297397 28287 raft_consensus.cc:2889] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:40.300244 28287 pending_rounds.cc:107] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:18:40.300505 28287 raft_consensus.cc:2889] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:40.306031 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:40.306799 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:40.306970 28433 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:40.306540 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 95653us
I20260709 12:18:40.308506 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:40.197170 32554 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.307358 28291 raft_consensus.cc:606] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:40.339593 28437 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:18:40.332320 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 68097.2us
I20260709 12:18:40.335196 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:40.351541 28984 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 8: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:40.350134 28438 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:40.350108 28433 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:40.357693 28986 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:40.358572 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.360531 28283 raft_consensus.cc:2410] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:40.361150 28288 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:40.287873 32541 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.378456 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.382081 28290 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:18:40.384074 28289 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:40.392414 28358 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:40.389321 28435 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:18:40.289245 32544 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.209818 32518 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.325402 32523 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:40.396996 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:40.391229 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:40.326020 32555 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.334121 32554 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.119330 32520 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:40.395920 32617 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:18:40.419951 28284 raft_consensus.cc:2410] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:40.413645 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.195696 32592 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.154s
I20260709 12:18:40.414745 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:40.359299 32469 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.423696 28359 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:40.424284 28283 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:40.425261 28364 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:18:40.426671 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.416252 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:40.428082 28366 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:40.353436 32522 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.400264 32567 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.354743 32445 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:18:40.399243 32630 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:40.432312 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:40.411655 32581 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:40.411249 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.415949 32554 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.450021 32581 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:40.450307 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 141897us
I20260709 12:18:40.451381 32581 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.440042 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:40.410072 32245 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:40.451686 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:40.420743 32451 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.407512 32583 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:18:40.467304 28404 leader_election.cc:400] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:40.477667 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:40.466087 32581 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.482205 32614 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:40.483927 32614 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:40.484741 32245 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.485530 32245 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:40.485898 28290 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:40.486421 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.474906 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:40.487015 28289 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:40.487705 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:40.489959 32535 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:40.490409 32535 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:40.490795 32535 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:40.493291 32634 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:40.493863 32634 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.494238 32634 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.494575 32537 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:18:40.496547 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.485903 32245 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:40.498286 32245 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.503067 28364 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:40.456229 28366 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:40.407191 32550 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:18:40.502513 28332 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:40.506639 28289 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
W20260709 12:18:40.458324 32510 consensus_meta.cc:287] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.501s	user 0.014s	sys 0.000s
I20260709 12:18:40.420279 32555 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.510090 28366 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 12:18:40.466477 32451 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.511168 32541 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:40.516112 32537 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.505750 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:40.518059 32634 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.542567 32634 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.542771 32535 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.539208 28364 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:18:40.539939 32413 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:40.540447 32546 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.549923 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:40.561219 28435 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.563187 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:40.550781 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.535476 32451 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.545341 32537 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.551398 32565 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:40.467123 32523 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 12:18:40.431507 32469 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.557801 32413 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:40.567979 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:40.582681 28359 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:18:40.578310 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:40.586097 28292 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:40.588025 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:40.587674 28982 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:40.589051 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.590250 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:40.590696 32539 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:40.591394 32539 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:40.594693 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.594781 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.447790 32567 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.595714 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.588049 28435 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:40.596690 28285 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:40.596763 28366 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:40.606005 28985 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:40.606426 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.597685 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:40.607757 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.569609 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:40.608826 28439 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
W20260709 12:18:40.608763 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 195427us
I20260709 12:18:40.610253 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:40.616649 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:40.617619 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.589257 32545 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:40.620321 32445 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:40.598511 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:40.659054 28435 raft_consensus.cc:2368] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11.
I20260709 12:18:40.629068 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.659957 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.630283 28290 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.583180 32537 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.633373 28440 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:40.666848 28440 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:40.668635 28291 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:40.537243 32510 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.458731 32630 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.583226 32565 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:40.662978 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 193424us
I20260709 12:18:40.586506 32498 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:40.605202 32507 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.678013 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:40.620340 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:40.621695 32541 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:40.675627 28359 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 12:18:40.684230 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 64067.2us
I20260709 12:18:40.616571 32546 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.605548 32589 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:18:40.663709 32531 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:40.677301 28440 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.691807 28366 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:40.692189 28287 raft_consensus.cc:2410] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:40.694792 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:40.615576 32578 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:18:40.697914 28440 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.672819 32592 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:40.684063 32445 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:40.715921 32618 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.670s	user 0.000s	sys 0.007s
I20260709 12:18:40.722720 28366 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:40.740460 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:40.723752 28364 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 12:18:40.725873 32566 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:40.730383 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 478757us
I20260709 12:18:40.746048 28366 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:40.731118 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:40.731716 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:40.731861 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:40.748867 28984 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:40.732801 28365 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:40.734040 28983 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:40.734385 32426 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:40.693001 32531 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:40.741160 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 109798us
I20260709 12:18:40.693301 32639 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:40.697213 32627 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.698390 32541 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:40.743546 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 48853.7us
I20260709 12:18:40.753983 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.758396 28364 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:40.761164 28286 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:40.768926 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.769773 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.693720 32546 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.738911 32638 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:40.751117 32531 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.778113 28365 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:40.770419 28286 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.752017 32426 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:40.768294 28440 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:40.740732 32535 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.709432 32592 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.758806 32410 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.770148 32639 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:40.697970 32413 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.765542 32636 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:40.791541 28365 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:18:40.766503 32522 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.776803 32566 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:40.798365 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.800465 28288 raft_consensus.cc:2410] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:40.767395 32627 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.803099 32289 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.827359 32289 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:40.827836 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:40.828506 28983 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:40.827407 32546 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:40.803733 32637 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:18:40.831300 32630 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.831739 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.806142 32516 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.800379 32410 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.832935 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.835498 32410 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.837252 32555 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.840795 32480 raft_consensus.cc:993] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:40.841204 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.841228 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2116) took 1274 ms. Trace:
I20260709 12:18:40.841918 32541 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.842581 28367 rpcz_store.cc:276] 0709 12:18:39.566384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.569282 (+  2898us) service_pool.cc:224] Handling call
0709 12:18:39.569614 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:39.774784 (+205170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:39.774815 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:39.780270 (+  5455us) log.cc:844] Serialized 26 byte log entry
0709 12:18:40.039325 (+259055us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:40.039367 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.840776 (+801409us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54003bb5d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:40.840830 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.840883 (+    53us) raft_consensus.cc:1685] finished
0709 12:18:40.840896 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.841139 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18058,"raft.queue_time_us":23003,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":47189,"spinlock_wait_cycles":418313984,"thread_start_us":191233,"threads_started":1,"wal-append.queue_time_us":364777}
I20260709 12:18:40.845072 28367 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:40.818495 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:40.825033 32627 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.793546 32451 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.872195 28286 raft_consensus.cc:2410] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:40.841439 32480 raft_consensus.cc:1081] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:40.846359 28291 pending_rounds.cc:85] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 5
I20260709 12:18:40.875734 28291 pending_rounds.cc:107] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:40.876101 28291 raft_consensus.cc:2889] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:40.847641 28284 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.848096 28360 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:40.845686 28365 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.816924 28984 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:40.830027 32469 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:40.800627 32216 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.572s	user 0.015s	sys 0.000s
I20260709 12:18:40.884215 28257 leader_election.cc:400] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11.
I20260709 12:18:40.901335 28257 leader_election.cc:403] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:40.873227 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.896112 28365 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:40.912102 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:40.916321 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:40.919013 28286 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.919646 28284 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:40.913646 28330 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:40.765784 32565 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:40.903963 32538 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.000s
I20260709 12:18:40.795032 32638 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 12:18:40.930760 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.891242 32567 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:40.832952 32516 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.937541 28360 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 8.
I20260709 12:18:40.938721 28286 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:40.916885 32634 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.799985 32445 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.890062 32627 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:18:40.922132 32522 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:40.919035 32630 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.939582 32500 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.948985 32451 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:40.917047 32289 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:18:40.942044 32510 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:40.935468 32573 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 12:18:40.960932 32642 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 12:18:40.962591 32554 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.001s
I20260709 12:18:40.963130 32554 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.964510 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:40.966096 28362 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:40.926807 32426 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:40.968094 32589 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.943168 32565 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.968644 32567 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:40.968698 32589 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:40.962870 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:40.979314 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:40.982117 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 68562us
I20260709 12:18:40.983012 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:40.984115 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:40.973493 32645 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:18:40.942828 32641 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:40.994906 28330 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:40.959746 32592 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:40.965606 32554 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:40.999634 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:40.936115 32538 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:41.006173 32652 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:18:41.008006 32522 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:41.009298 32499 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:40.997064 32641 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.012094 32531 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.012897 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:41.013101 32641 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:40.991021 32320 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:41.013823 28287 raft_consensus.cc:2410] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:18:41.014046 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:41.003341 32289 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:40.988358 32573 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:18:41.014827 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:41.015992 28287 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:41.016541 28287 raft_consensus.cc:606] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:41.017026 28287 raft_consensus.cc:614] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:41.017187 28358 raft_consensus.cc:2368] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14.
I20260709 12:18:41.017640 32538 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:40.997262 32567 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:18:41.000456 32510 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:41.018970 28988 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 7. Current term is 9. Ops: []
I20260709 12:18:41.016486 32516 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:41.014866 28283 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.023299 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:40.991616 32480 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:41.024719 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:41.027148 28358 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
W20260709 12:18:41.027724 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:41.028278 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.028404 28362 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:18:41.030815 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:41.031451 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:41.032219 28365 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:40.986665 32500 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:18:41.003926 32592 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:41.034873 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:41.036110 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.037312 28365 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:40.941450 32639 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.037882 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.038546 28436 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:41.036614 28434 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:41.026551 32426 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.043980 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2092) took 1042 ms. Trace:
I20260709 12:18:41.046859 28330 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:41.048486 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:41.048677 28434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:18:41.049394 28434 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:41.049619 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:41.049821 28434 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.050405 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:41.046777 28361 rpcz_store.cc:276] 0709 12:18:40.001329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.013081 (+ 11752us) service_pool.cc:224] Handling call
0709 12:18:41.043792 (+1030711us) spinlock_profiling.cc:243] Waited 98 ms on lock 0x7b5400127d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:41.043920 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225504128}
I20260709 12:18:41.026633 32579 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:18:41.062171 32539 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.062932 32539 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:18:41.035729 32320 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:18:41.063948 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.064316 32290 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:41.064666 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:41.064800 32290 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:41.065241 28987 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.051772 32538 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:41.062830 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.076274 28987 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:41.076958 28432 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:41.066030 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:41.077785 28984 raft_consensus.cc:2368] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:41.078961 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:41.090464 28283 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:41.147814 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:41.081135 28434 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.093040 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:41.036794 32480 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:41.029963 32522 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:18:41.038236 32592 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:41.040195 32510 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:18:41.157299 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 123088us
W20260709 12:18:41.157204 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 78336.1us
I20260709 12:18:41.172631 28330 leader_election.cc:400] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:41.173041 28330 leader_election.cc:403] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:18:41.157647 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:41.161103 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 68201.8us
I20260709 12:18:41.161064 28983 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:41.042932 32289 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:41.165867 28283 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:41.175534 28283 raft_consensus.cc:606] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:41.175994 28283 raft_consensus.cc:614] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:41.170931 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:41.067864 32581 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:41.073628 32535 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:41.101950 32516 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:41.053092 32554 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.143s
I20260709 12:18:41.136744 32541 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:41.199139 28432 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:41.211088 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.212571 28432 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:41.212929 28432 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:41.203887 28434 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:41.065354 32539 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:41.167402 32634 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:41.217175 28285 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:41.216909 28986 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:41.218713 28283 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: [10.6-13.8]
I20260709 12:18:41.218940 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.219694 28434 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:41.227900 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:41.203372 32544 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 7. Current term is 8.
I20260709 12:18:41.252765 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:41.225677 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:41.202733 32581 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:41.217971 32537 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.002s
I20260709 12:18:41.268306 32639 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.268910 32639 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.269384 32639 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:41.269582 32660 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:18:41.271986 32661 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:41.268944 32659 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.259523 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:41.207630 32510 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:41.215181 32554 consensus_queue.cc:1243] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:41.215471 32628 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:41.255612 32550 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:41.226245 32638 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:41.299259 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:41.065717 32290 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.253355 28434 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:41.249656 28432 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.303500 28432 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.217008 32517 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:41.272347 32497 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.947s	user 0.011s	sys 0.001s
W20260709 12:18:41.281028 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 110195us
I20260709 12:18:41.285763 32541 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:41.199888 32641 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 12:18:41.275976 32645 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:18:41.223778 32535 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:41.070410 32469 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Committing config change with OpId 6.6: config changed from index 2 to 6, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:41.276589 32537 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.273030 32659 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.312865 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:41.309859 32638 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.295594 32636 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:41.359599 32516 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:41.366209 32664 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:18:41.366518 32516 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.328848 32497 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.369252 32535 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.369516 32639 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.340372 32517 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.371553 32659 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:41.372155 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 59379.9us
I20260709 12:18:41.373061 28290 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:41.373600 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:41.375767 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:41.376240 32638 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:41.376438 28286 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.342167 28432 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:18:41.378762 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.310395 32628 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:41.339381 32290 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:41.300558 32589 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:41.367841 32622 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.286s	user 0.003s	sys 0.011s
I20260709 12:18:41.387395 32541 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.376538 32537 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:41.378705 28432 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:41.360133 32567 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:18:41.390439 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:41.391108 28432 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.386299 32245 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.295938 32661 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.372403 32517 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:41.303354 32289 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.403326 28286 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:41.409562 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:41.411592 28284 raft_consensus.cc:2410] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 12:18:41.414885 32615 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.006s	sys 0.007s
I20260709 12:18:41.410912 28989 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:41.418883 28286 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:18:41.421882 28286 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:41.422974 28432 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:41.424172 32653 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.303471 32480 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.429003 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:41.423306 28286 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:41.427886 32630 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11.
I20260709 12:18:41.328483 32510 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.429100 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2158) took 1095 ms. Trace:
W20260709 12:18:41.431504 32615 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.007s	sys 0.007s
I20260709 12:18:41.431114 32628 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.433012 28289 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:41.435256 32661 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:41.397801 32426 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:18:41.396775 32671 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:41.383131 32497 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:41.371804 32639 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:41.442934 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 69404.1us
W20260709 12:18:41.420022 32645 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.056s	user 0.002s	sys 0.000s
I20260709 12:18:41.413837 32537 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.444376 28292 rpcz_store.cc:276] 0709 12:18:40.337912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.338284 (+   372us) service_pool.cc:224] Handling call
0709 12:18:41.428905 (+1090621us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540035f2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:41.429038 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521699200}
I20260709 12:18:41.350332 32581 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.458106 32672 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:41.465422 32672 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:41.465551 32581 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.396657 32550 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.418476 32641 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:41.409206 32451 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:18:41.468079 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:41.459657 32046 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:41.465930 32672 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.473855 32550 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.483379 32550 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:41.488657 32672 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.495709 32672 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:41.469064 32641 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.465241 32537 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.496630 32641 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.499403 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:41.500489 32537 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:41.447772 32653 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:41.508024 32643 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.713s	user 0.000s	sys 0.002s
I20260709 12:18:41.508304 28989 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:41.511288 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:41.511731 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:41.514920 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.516969 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2135) took 1212 ms. Trace:
I20260709 12:18:41.476830 32046 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:41.517436 28441 rpcz_store.cc:276] 0709 12:18:40.304457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.304571 (+   114us) service_pool.cc:224] Handling call
0709 12:18:40.350990 (+ 46419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.430031 (+ 79041us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:40.445300 (+ 15269us) log.cc:844] Serialized 14 byte log entry
0709 12:18:40.743086 (+297786us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:40.743141 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:40.743185 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.508800 (+765615us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:41.516712 (+  7912us) spinlock_profiling.cc:243] Waited 78.8 ms on lock 0x7b54002e4ad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:41.516766 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.516897 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62436,"prepare.queue_time_us":15209,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":38357,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":6476,"spinlock_wait_cycles":208403968,"thread_start_us":259107,"threads_started":1,"wal-append.queue_time_us":391649}
I20260709 12:18:41.519557 32046 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.520947 32046 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.521553 32531 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:41.524775 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:41.525301 32499 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:41.525771 32499 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:41.526649 32499 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.527797 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:41.530947 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:41.528463 28292 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:41.532714 28286 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
W20260709 12:18:41.533123 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:41.532222 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.529028 32537 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:41.532729 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:41.462981 32634 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:41.545567 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:41.546761 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:41.550815 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:41.551160 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:41.551745 28292 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:41.556002 32497 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.541988 32290 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.508335 32653 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:41.554201 32640 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.705s	user 0.001s	sys 0.007s
I20260709 12:18:41.581866 32290 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.545181 32531 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:18:41.584326 32290 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:41.556946 32678 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:41.585119 32678 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.585541 32678 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.585812 32634 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.563293 32541 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.590111 32541 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:41.593767 32647 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.789s	user 0.003s	sys 0.000s
I20260709 12:18:41.598275 28988 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 12:18:41.601411 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:41.602210 28289 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.603433 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:41.604135 28360 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:41.606215 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:41.606930 28363 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:41.607520 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:41.607826 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:41.604396 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2108) took 1009 ms. Trace:
I20260709 12:18:41.608548 28366 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:41.612213 32654 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.615s	user 0.001s	sys 0.000s
I20260709 12:18:41.608335 28288 rpcz_store.cc:276] 0709 12:18:40.595149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.617309 (+ 22160us) service_pool.cc:224] Handling call
0709 12:18:41.604182 (+986873us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54000f2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:41.604327 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418068864}
I20260709 12:18:41.614159 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:41.614921 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:41.562165 32497 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:41.615612 28288 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:41.615720 32497 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.555992 32565 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:18:41.616143 28283 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:18:41.616668 28287 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:41.617717 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:41.593853 32644 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.751s	user 0.000s	sys 0.002s
I20260709 12:18:41.618283 32592 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.619385 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 481) took 1427 ms. Trace:
I20260709 12:18:41.619784 28990 rpcz_store.cc:276] 0709 12:18:40.191565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.191697 (+   132us) service_pool.cc:224] Handling call
0709 12:18:40.209064 (+ 17367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:40.209346 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:40.209373 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:40.518528 (+309155us) spinlock_profiling.cc:243] Waited 6.09 ms on lock 0x7b2c00095d78. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:18:40.519029 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:18:40.704952 (+185923us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:40.704986 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.705060 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:41.593043 (+887983us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:41.619128 (+ 26085us) raft_consensus.cc:1685] finished
0709 12:18:41.619175 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.619321 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89500928,"thread_start_us":185470,"threads_started":1,"wal-append.queue_time_us":322900}
I20260709 12:18:41.616392 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:41.622494 28290 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:41.622617 28287 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:41.615123 28364 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:18:41.624498 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:41.625463 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:41.575817 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:41.626822 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.626670 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:41.548733 32499 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.635056 28289 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:41.636324 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:41.637027 28289 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:41.638470 28289 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:41.640036 28289 pending_rounds.cc:85] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 4
I20260709 12:18:41.640362 28289 pending_rounds.cc:107] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:18:41.640615 28289 raft_consensus.cc:2889] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:41.656014 28292 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:41.661942 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:41.662804 28288 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:41.664716 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:41.665344 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:41.666738 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:41.554426 32537 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:41.619498 32592 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:41.694514 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.696012 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:41.697207 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:41.704077 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:41.681756 32672 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:41.683795 32538 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:41.693775 32510 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.696437 32517 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:18:41.712963 32666 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:41.713517 32480 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:18:41.716490 32646 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.891s	user 0.000s	sys 0.002s
I20260709 12:18:41.718683 28283 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 12:18:41.729111 28283 pending_rounds.cc:85] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 12
I20260709 12:18:41.729425 28283 pending_rounds.cc:107] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:18:41.729704 28283 raft_consensus.cc:2889] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:41.692298 32554 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:41.644752 32641 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.681725 32573 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:18:41.701045 32581 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.734371 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:41.744318 28990 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:41.694139 32500 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:18:41.745568 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:41.706122 32677 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.701761 32682 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:41.726735 32630 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:41.726931 32544 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.737198 32541 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:41.758664 32516 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:41.752594 32554 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:41.754652 32677 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.768437 32516 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.739823 32531 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:18:41.732151 32510 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:41.772804 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.772877 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2107) took 1178 ms. Trace:
I20260709 12:18:41.778800 28285 rpcz_store.cc:276] 0709 12:18:40.594897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.607485 (+ 12588us) service_pool.cc:224] Handling call
0709 12:18:41.772688 (+1165203us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54003db3d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:41.772802 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287510912}
I20260709 12:18:41.762669 32682 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:41.798610 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:41.800021 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:41.791188 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2401) took 1244 ms. Trace:
I20260709 12:18:41.769748 32677 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.805183 28432 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:41.804948 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:41.806205 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 93825.8us
I20260709 12:18:41.806911 28439 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.805009 28433 rpcz_store.cc:276] 0709 12:18:40.546794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.546938 (+   144us) service_pool.cc:224] Handling call
0709 12:18:40.547346 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:40.547772 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:40.547800 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:40.548332 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:18:40.672019 (+123687us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:40.672072 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.672159 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:41.715737 (+1043578us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:41.772265 (+ 56528us) raft_consensus.cc:1685] finished
0709 12:18:41.772290 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:41.791095 (+ 18805us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7060,"raft.queue_time_us":57971,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":12291,"spinlock_wait_cycles":114917248,"thread_start_us":123249,"threads_started":1,"wal-append.queue_time_us":237934}
I20260709 12:18:41.756215 32581 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.811745 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:41.812512 28440 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.771587 32516 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:41.818773 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.819816 28433 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:41.821926 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:41.822197 28433 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:18:41.823532 28363 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:41.825232 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:41.765237 32537 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.813417 28285 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:41.764153 32544 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.809127 32541 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:41.837779 28440 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:41.838967 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.841308 28433 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:41.841491 28364 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:18:41.814049 32639 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.841485 28441 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:41.848109 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:41.849035 28364 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:41.839545 32668 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.540s	user 0.004s	sys 0.000s
I20260709 12:18:41.854218 28439 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:41.841609 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:41.857583 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.859746 28440 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:41.872134 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 482) took 1609 ms. Trace:
I20260709 12:18:41.872637 28985 rpcz_store.cc:276] 0709 12:18:40.262569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.279018 (+ 16449us) service_pool.cc:224] Handling call
0709 12:18:40.359790 (+ 80772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.606472 (+246682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.266935 (+660463us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54005bfed4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:41.871890 (+604955us) spinlock_profiling.cc:243] Waited 80.4 ms on lock 0x7b54005bfed0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:41.872059 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":732919552}
I20260709 12:18:41.876120 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:41.876893 28441 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:41.882572 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:41.884195 28364 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:41.888794 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:41.848519 32681 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:41.896669 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 485) took 1165 ms. Trace:
I20260709 12:18:41.897136 28986 rpcz_store.cc:276] 0709 12:18:40.730961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.740742 (+  9781us) service_pool.cc:224] Handling call
0709 12:18:40.763619 (+ 22877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.217924 (+454305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.896443 (+678519us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54002ab554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:41.896609 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041977600}
W20260709 12:18:41.891781 32699 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:18:41.883756 32682 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.836318 32554 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.911016 28366 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:18:41.911404 28360 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:18:41.912930 32451 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:41.915261 32451 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:41.926666 32581 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:41.851866 32541 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.932349 32541 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:41.893316 32681 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.934188 32681 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:41.938798 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 56312.9us
I20260709 12:18:41.929600 32510 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:41.939692 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.852109 32639 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:41.887959 32500 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.961997 28362 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:41.984932 32678 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:41.988917 32703 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:41.954721 32697 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 12:18:41.954493 32693 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 12:18:41.957883 32698 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 12:18:41.952560 32694 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260709 12:18:41.977231 32700 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:18:41.997653 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:42.001385 28360 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:18:41.979468 32696 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:18:41.977445 32641 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:41.948689 32702 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.953850 32500 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.989610 32703 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:42.008816 32497 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:42.009374 32678 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:41.997795 32581 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.011170 28436 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:42.105937 28436 pending_rounds.cc:85] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
I20260709 12:18:42.106187 28436 pending_rounds.cc:107] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:42.106446 28436 raft_consensus.cc:2889] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:42.011447 28287 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:42.108143 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:42.014532 32544 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:42.108582 32684 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:42.018941 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:41.997992 32705 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:18:42.111581 32639 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:42.111924 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:42.113492 28286 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:42.115244 32703 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:42.116456 28286 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:42.023121 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:42.117661 28990 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:42.048714 28986 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:41.994750 32510 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 12:18:42.098979 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:42.106952 32497 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:42.119730 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:42.121909 28287 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:42.005811 32567 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 12:18:42.052911 32702 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.094231 32500 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.105401 32581 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:42.136049 32426 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.147815 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 604) took 1272 ms. Trace:
I20260709 12:18:42.148926 28284 rpcz_store.cc:276] 0709 12:18:40.874942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.921648 (+ 46706us) service_pool.cc:224] Handling call
0709 12:18:42.147572 (+1225924us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b54000028d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:42.147745 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103519488}
W20260709 12:18:42.152550 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 175156us
I20260709 12:18:42.142513 32702 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.150601 32665 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:42.195840 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:42.152305 32426 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:42.203397 32677 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:42.236004 32695 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.461s	user 0.001s	sys 0.000s
I20260709 12:18:42.239081 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2145) took 1209 ms. Trace:
I20260709 12:18:42.242993 32630 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:42.243563 32630 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.248867 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2151) took 1159 ms. Trace:
I20260709 12:18:42.253073 28986 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:18:42.255647 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.257107 32663 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:42.257750 32663 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:42.260074 32663 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:42.260627 32663 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:42.261089 32663 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:42.262758 32661 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.263353 32661 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:42.265458 32661 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:42.270318 32638 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:42.268477 32636 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:42.270798 32641 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.299201 32678 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:42.299836 32641 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:42.300805 32678 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.302531 32641 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:42.249354 28436 rpcz_store.cc:276] 0709 12:18:41.089925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.090141 (+   216us) service_pool.cc:224] Handling call
0709 12:18:41.166819 (+ 76678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.107126 (+940307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.248564 (+141438us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b54002db4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:42.248787 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1941002112}
W20260709 12:18:42.267858 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 799823us
I20260709 12:18:42.304136 28404 leader_election.cc:400] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
I20260709 12:18:42.304462 28404 leader_election.cc:403] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:42.305956 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:42.269965 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.307461 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.307884 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.301338 32678 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:42.309939 28986 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:42.304329 32653 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:42.312109 32636 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.311865 32653 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:42.298623 32638 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:42.314611 32653 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:42.317277 32638 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:42.294386 32517 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:42.363704 32517 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.364126 32517 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.296038 32663 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:42.319408 32678 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:42.320526 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:42.311762 32661 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10.
I20260709 12:18:42.344033 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2144) took 1314 ms. Trace:
I20260709 12:18:42.239550 28437 rpcz_store.cc:276] 0709 12:18:41.030188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.030316 (+   128us) service_pool.cc:224] Handling call
0709 12:18:41.039799 (+  9483us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:41.073931 (+ 34132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:41.073966 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:41.163512 (+ 89546us) log.cc:844] Serialized 78 byte log entry
0709 12:18:41.299147 (+135635us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:41.363986 (+ 64839us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.383052 (+ 19066us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.495466 (+112414us) spinlock_profiling.cc:243] Waited 7.9 ms on lock 0x7b18004971a0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:41.642240 (+146774us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b4800210258. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c37b0aa1 00007f85c37a7b37 00007f85c37a79ca 00007f85c389f26f 00007f85c389ef37 00007f85c38b355c 00007f85c38b3508 00007f85c38b3488 00007f85c38b3443 00007f85c38b26ec 00007f85c27dca96
0709 12:18:42.238515 (+596275us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b18004971a0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:18:42.238705 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.238788 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:42.238834 (+    46us) raft_consensus.cc:1685] finished
0709 12:18:42.238848 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.239008 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74498,"prepare.queue_time_us":86998,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":128953,"raft.run_cpu_time_us":3411,"raft.run_wall_time_us":265697,"spinlock_wait_cycles":27285760,"thread_start_us":81999,"threads_started":1,"wal-append.queue_time_us":177134}
W20260709 12:18:42.362114 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 56233.1us
I20260709 12:18:42.385663 28438 rpcz_store.cc:276] 0709 12:18:41.029240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.029351 (+   111us) service_pool.cc:224] Handling call
0709 12:18:41.043508 (+ 14157us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:41.446482 (+402974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:41.446513 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:41.457292 (+ 10779us) log.cc:844] Serialized 78 byte log entry
0709 12:18:41.567714 (+110422us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:41.567751 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.343501 (+775750us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b540026d4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:42.343559 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:42.343616 (+    57us) raft_consensus.cc:1685] finished
0709 12:18:42.343630 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.343944 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":831300,"raft.queue_time_us":17421,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":18818,"spinlock_wait_cycles":759289216,"thread_start_us":160818,"threads_started":2,"wal-append.queue_time_us":224460}
I20260709 12:18:42.387207 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:42.388100 32661 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:42.390838 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:42.362884 32641 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:42.391857 32641 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:42.394295 32641 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:42.195138 32589 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:42.401607 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.197378 32665 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:42.404074 32523 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.577s	user 0.009s	sys 0.010s
W20260709 12:18:42.417347 32622 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
W20260709 12:18:42.417078 32697 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 12:18:42.508481 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:18:42.496031 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 53699.9us
I20260709 12:18:42.499141 32641 raft_consensus.cc:476] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Not starting pre-election -- already a leader
I20260709 12:18:42.441267 32426 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:42.493831 32523 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.667s	user 0.012s	sys 0.010s
I20260709 12:18:42.448738 32661 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:42.507526 32681 consensus_meta.cc:287] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.000s
I20260709 12:18:42.533624 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:42.414162 32589 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:42.465173 32663 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 12:18:42.565172 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 56793.4us
I20260709 12:18:42.565806 32691 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:42.566363 32505 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:42.570410 32708 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:18:42.578794 32711 log.cc:927] Time spent T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.355s	user 0.000s	sys 0.000s
W20260709 12:18:42.579751 32709 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:18:42.580605 32710 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:18:42.581048 32426 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:42.584033 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 50512.3us
W20260709 12:18:42.588222 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59587us
I20260709 12:18:42.512964 32716 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.498051 32517 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:42.613917 28330 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:42.621155 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:42.521258 32661 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:42.682461 28363 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:42.677695 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:42.614741 32638 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.683333 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:42.542069 32677 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:42.684341 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:42.685252 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:42.685889 28288 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:42.559875 32681 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:42.616429 32717 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.177s
I20260709 12:18:42.963797 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2142) took 1365 ms. Trace:
I20260709 12:18:42.784348 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:42.686249 28285 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:42.784063 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:42.813632 28361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.014s	sys 0.025s
I20260709 12:18:42.617009 32523 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:42.695634 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:42.746219 28433 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:42.705494 28439 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:42.716648 28332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.029s	sys 0.017s
W20260709 12:18:42.746775 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 67350.8us
W20260709 12:18:42.999225 28361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.020s	sys 0.031s
I20260709 12:18:42.994777 28432 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:42.947767 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:42.938668 32718 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.353s	user 0.000s	sys 0.000s
I20260709 12:18:42.993628 28365 rpcz_store.cc:276] 0709 12:18:41.598336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.622554 (+ 24218us) service_pool.cc:224] Handling call
0709 12:18:41.622950 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:41.663820 (+ 40870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:41.663851 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:41.693783 (+ 29932us) log.cc:844] Serialized 52 byte log entry
0709 12:18:42.026193 (+332410us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:42.491234 (+465041us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.574099 (+ 82865us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.576035 (+  1936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.576134 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:42.963507 (+387373us) raft_consensus.cc:1685] finished
0709 12:18:42.963532 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.963720 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56413,"prepare.queue_time_us":179652,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":84,"raft.queue_time_us":114115,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":17677,"thread_start_us":695815,"threads_started":2,"wal-append.queue_time_us":500466}
W20260709 12:18:43.057567 28358 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.001s	sys 0.021s
W20260709 12:18:43.057934 28358 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.002s	sys 0.021s
W20260709 12:18:43.000686 28990 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.012s	sys 0.011s
W20260709 12:18:43.059190 28990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.012s	sys 0.013s
I20260709 12:18:43.055003 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:42.919263 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2180) took 1365 ms. Trace:
W20260709 12:18:42.651605 28258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.027s	sys 0.058s
W20260709 12:18:43.063050 28258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.028s	sys 0.059s
I20260709 12:18:43.062893 28289 rpcz_store.cc:276] 0709 12:18:41.553586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.639404 (+ 85818us) service_pool.cc:224] Handling call
0709 12:18:41.639803 (+   399us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:41.704260 (+ 64457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:41.704296 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:41.704892 (+   596us) log.cc:844] Serialized 52 byte log entry
0709 12:18:42.190939 (+486047us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:42.190989 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.191078 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:42.918997 (+727919us) raft_consensus.cc:1685] finished
0709 12:18:42.919020 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.919188 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24183,"raft.queue_time_us":26346,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":23898,"thread_start_us":461903,"threads_started":1,"wal-append.queue_time_us":514698}
W20260709 12:18:42.934911 32713 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 12:18:42.803229 28284 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:18:42.787722 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 173897us
W20260709 12:18:42.660983 32622 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:18:43.053053 28332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.037s	sys 0.027s
I20260709 12:18:43.059666 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.084266 28432 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:42.753499 28360 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:43.058403 28433 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:42.937891 32686 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.817014 32451 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.961452 32506 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:43.066138 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:42.745668 28287 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:43.004405 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:42.752876 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.107388 28437 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:43.071499 28285 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:43.107604 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:43.081411 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:42.761401 32666 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.014s	sys 0.029s
I20260709 12:18:43.082767 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:43.109598 28984 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:42.960088 32589 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:42.941223 32675 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.082141 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:42.661652 32630 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:42.752738 28359 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:43.085487 28432 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:43.117267 28432 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.118264 28432 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:43.085251 28440 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:18:43.084661 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 888845us
I20260709 12:18:43.115866 28439 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
W20260709 12:18:42.993587 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 308396us
W20260709 12:18:42.829991 32628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.187s	user 0.010s	sys 0.009s
W20260709 12:18:42.783282 32535 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.009s	sys 0.029s
I20260709 12:18:43.115370 32545 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:42.884268 32537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.013s	sys 0.007s
I20260709 12:18:42.784202 32385 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.326s
W20260709 12:18:43.015967 32516 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.182s	user 0.008s	sys 0.005s
I20260709 12:18:43.098083 32727 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.088683 32580 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.746632 32659 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:42.682799 32497 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.688189 32641 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:43.147275 32641 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.146435 32726 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.148252 32726 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.149668 32641 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:43.153707 32641 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:43.154848 32641 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:43.150971 32726 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.151420 32510 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:18:43.157474 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2151) took 1198 ms. Trace:
I20260709 12:18:43.152707 32672 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.158702 32672 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.160908 32672 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.164494 28984 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:43.166869 28432 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:43.168048 32727 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.171231 32727 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:43.172914 32681 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.175495 32630 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:42.975286 32517 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:43.101982 28433 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:18:42.969480 32665 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:43.068224 32541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.005s	sys 0.061s
I20260709 12:18:42.949982 32639 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.687134 32555 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.083213 28283 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:42.792547 32581 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:42.977849 32245 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:43.048122 32661 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:43.056756 32523 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:42.747272 32670 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.364s
I20260709 12:18:43.081717 32480 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:43.115669 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.011390 32573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.013s	sys 0.000s
W20260709 12:18:43.196318 32541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.006s	sys 0.061s
I20260709 12:18:43.121027 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.081156 32715 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:42.938109 32714 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.449s
I20260709 12:18:42.784663 32410 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:43.113564 32666 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.022s	sys 0.037s
I20260709 12:18:42.618362 32505 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:42.683701 32426 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:43.120695 32628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.478s	user 0.012s	sys 0.015s
I20260709 12:18:43.105580 32506 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:43.121053 32535 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.009s	sys 0.030s
I20260709 12:18:43.141896 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.163012 32580 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.106436 32686 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.143962 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:42.687842 32716 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:43.124027 32537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.015s	sys 0.008s
I20260709 12:18:43.204838 32653 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:18:43.146003 32659 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.146529 32728 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.120884 32733 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:42.925052 32636 consensus_meta.cc:287] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.612s	user 0.011s	sys 0.000s
I20260709 12:18:43.157907 28361 rpcz_store.cc:276] 0709 12:18:41.959477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.959920 (+   443us) service_pool.cc:224] Handling call
0709 12:18:41.960259 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.155483 (+195224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.155515 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.156000 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.056648 (+900648us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:43.056694 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.056775 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.157195 (+100420us) raft_consensus.cc:1685] finished
0709 12:18:43.157223 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.157400 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17861,"raft.queue_time_us":49630,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"thread_start_us":1031934,"threads_started":2,"wal-append.queue_time_us":954263}
I20260709 12:18:43.200822 32555 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.290438 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:43.188005 28433 pending_rounds.cc:85] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
I20260709 12:18:43.134498 32701 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.195662 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:43.161598 32702 consensus_meta.cc:287] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.954s	user 0.007s	sys 0.000s
I20260709 12:18:43.109162 32451 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.293628 28361 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:43.198181 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 65648.9us
I20260709 12:18:43.296382 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:43.200773 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:43.206568 28283 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2194) took 1182 ms (client timeout 1539 ms). Trace:
I20260709 12:18:42.687422 32663 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.420s
I20260709 12:18:43.144619 32539 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:18:43.212572 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:43.217006 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:43.306986 28359 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:43.150064 32497 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.223568 28287 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 12:18:43.307955 28287 pending_rounds.cc:85] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 11
I20260709 12:18:43.308219 28287 pending_rounds.cc:107] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260709 12:18:43.308468 28287 raft_consensus.cc:2889] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:43.213989 32573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.014s	sys 0.000s
I20260709 12:18:43.214236 32480 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.140223 32675 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.323141 28988 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:43.121371 32545 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:43.273386 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 129516us
I20260709 12:18:43.200479 32639 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.277503 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.291520 28433 pending_rounds.cc:107] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:18:43.188856 32516 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.294263 28363 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
W20260709 12:18:43.298223 28283 rpcz_store.cc:269] 0709 12:18:42.024375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.098287 (+ 73912us) service_pool.cc:224] Handling call
0709 12:18:43.206258 (+1107971us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b540035f2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:43.206476 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175807232}
I20260709 12:18:43.301579 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:43.246047 32670 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:43.232563 32665 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:43.303725 32289 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.028s	sys 0.037s
I20260709 12:18:43.275676 32517 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:43.236037 32719 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.100s
W20260709 12:18:43.401998 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 71954.3us
I20260709 12:18:43.322829 32480 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.338810 28365 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:43.339996 32535 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:43.331702 28433 raft_consensus.cc:2889] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:43.275199 32523 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.303866 32702 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.348436 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:43.306670 32537 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:43.299853 32636 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:43.402307 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:43.494230 28437 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:43.323603 32727 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.237661 32410 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.316779 32681 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:43.340549 32675 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:43.411552 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:43.218353 32245 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.434302 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.378185 32717 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.118s
W20260709 12:18:43.354960 28362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.008s	user 0.007s	sys 0.040s
W20260709 12:18:43.519393 28362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.172s	user 0.007s	sys 0.041s
I20260709 12:18:43.520349 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2154) took 1540 ms. Trace:
W20260709 12:18:43.355374 28986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.972s	user 0.018s	sys 0.012s
W20260709 12:18:43.521185 28986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.138s	user 0.019s	sys 0.013s
W20260709 12:18:43.439482 28366 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.036s	user 0.016s	sys 0.024s
I20260709 12:18:43.304163 32701 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.528491 32451 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:43.341789 32663 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:43.316991 32541 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:18:43.529232 32720 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:43.487057 28332 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:43.529912 32686 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:43.530148 32675 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:18:43.530900 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 43956.2us
I20260709 12:18:43.530572 32686 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:43.491248 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:43.529281 32451 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:43.495920 28364 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:43.532507 32451 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:43.529189 32663 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:43.348886 32670 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:43.497181 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 66582.4us
I20260709 12:18:43.535125 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:43.509884 28438 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:43.510836 28440 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:43.536767 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:43.537396 28440 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:43.355203 28434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.013s	sys 0.020s
W20260709 12:18:43.538259 28434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.014s	sys 0.021s
I20260709 12:18:43.538414 28285 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:43.378938 32665 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:43.401188 32500 consensus_meta.cc:287] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.252s	user 0.009s	sys 0.005s
I20260709 12:18:43.529944 32720 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.320111 32733 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:43.522143 28366 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.016s	sys 0.024s
I20260709 12:18:43.327545 32506 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.432122 32682 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 12:18:43.531574 32537 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:43.499444 32517 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:43.530397 32739 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
I20260709 12:18:43.316677 32728 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.494736 32722 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:43.542397 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 663) took 2315 ms. Trace:
I20260709 12:18:43.581934 28437 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:43.581137 28434 rpcz_store.cc:276] 0709 12:18:41.226973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.252565 (+ 25592us) service_pool.cc:224] Handling call
0709 12:18:41.252950 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.253208 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:41.332348 (+ 79140us) log.cc:844] Serialized 14 byte log entry
0709 12:18:41.449531 (+117183us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.678261 (+1228730us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.541873 (+863612us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.542049 (+   176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:43.542098 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:43.542145 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.542216 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.542336 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":872679,"prepare.queue_time_us":998423,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":4811,"raft.queue_time_us":837389,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":16012,"thread_start_us":1348313,"threads_started":3}
W20260709 12:18:43.577678 28989 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.003s	sys 0.025s
W20260709 12:18:43.582999 28989 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.003s	sys 0.025s
I20260709 12:18:43.583642 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:43.584918 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:43.500496 32523 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:43.438907 32289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.029s	sys 0.044s
W20260709 12:18:43.547071 28435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.010s	sys 0.012s
W20260709 12:18:43.590091 28435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.011s	sys 0.013s
I20260709 12:18:43.557842 32581 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:43.550836 28257 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.3:33721 timed out after 1.940s (SENT) [suppressed 11 similar messages]
I20260709 12:18:43.559815 28441 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:43.431691 32544 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:18:43.548449 32500 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.558139 32578 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.576859 32506 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:43.585525 28434 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:43.572356 28440 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:43.520769 28362 rpcz_store.cc:276] 0709 12:18:41.980296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.980434 (+   138us) service_pool.cc:224] Handling call
0709 12:18:41.980771 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:41.980989 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:41.981310 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.346803 (+365493us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b18002e0780. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:18:43.519874 (+1173071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:43.519956 (+    82us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:43.520013 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.520100 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.520274 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":162,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":33771,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":30992128,"thread_start_us":1172882,"threads_started":1}
I20260709 12:18:43.531739 32686 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:43.534432 32675 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:43.544356 32554 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:43.500926 32727 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.570557 32537 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.590670 32505 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.627065 28289 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:43.593286 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.578598 32517 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.533557 32451 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.515204 32681 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.575742 32639 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.565843 32426 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.586311 32636 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.605973 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.638633 28440 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:43.639755 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:43.640388 28440 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:18:43.534013 32663 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.603358 32545 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.629240 28437 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:43.642383 32716 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.615669 32659 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:43.626143 32537 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:43.625946 32555 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.591274 32722 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:43.627337 32554 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:43.597000 32581 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:43.646519 28288 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.011s	sys 0.017s
W20260709 12:18:43.661203 28288 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.011s	sys 0.017s
I20260709 12:18:43.603753 32578 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.633706 28434 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
W20260709 12:18:43.665544 28956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.003s	sys 0.019s
W20260709 12:18:43.666005 28956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.003s	sys 0.019s
I20260709 12:18:43.637290 32686 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:43.666864 28286 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.009s	sys 0.018s
W20260709 12:18:43.666913 28954 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.004s	sys 0.016s
W20260709 12:18:43.667688 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 719979us
W20260709 12:18:43.677546 28286 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.009s	sys 0.018s
W20260709 12:18:43.677772 28954 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.005s	sys 0.016s
W20260709 12:18:43.800971 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 735576us
W20260709 12:18:43.801824 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 121031us
I20260709 12:18:43.802884 28956 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:43.816639 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.775198 28440 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
W20260709 12:18:43.776300 28982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.013s	sys 0.009s
W20260709 12:18:43.857792 28982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.014s	sys 0.010s
W20260709 12:18:43.857961 28367 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.023s	sys 0.015s
W20260709 12:18:43.858412 28367 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.023s	sys 0.015s
W20260709 12:18:43.776355 28406 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.009s	sys 0.014s
W20260709 12:18:43.859632 28406 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.009s	sys 0.014s
W20260709 12:18:43.860283 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.860740 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.862907 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.863171 28432 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.000s	sys 0.018s
W20260709 12:18:43.863574 28432 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.000s	sys 0.018s
W20260709 12:18:43.666786 32691 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.009s	sys 0.019s
I20260709 12:18:43.871012 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.871852 28440 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:43.670547 32663 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:43.877241 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:43.926980 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.834920 32555 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:43.929418 32717 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:43.872241 32691 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.009s	sys 0.020s
W20260709 12:18:43.934384 28985 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.008s	sys 0.026s
I20260709 12:18:43.834187 32702 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:43.969035 28985 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.008s	sys 0.026s
W20260709 12:18:43.874858 28404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.011s	sys 0.040s
I20260709 12:18:43.868461 32426 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:43.950345 32745 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.293s	user 0.000s	sys 0.000s
I20260709 12:18:43.951058 32752 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:43.951021 32727 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:43.893567 32686 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:43.944939 32245 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:43.876089 32410 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:43.838590 32634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.028s	sys 0.015s
W20260709 12:18:43.957187 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.957926 28437 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
W20260709 12:18:44.008250 32740 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.542s	user 0.002s	sys 0.009s
I20260709 12:18:44.008930 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.643855 32636 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:43.958694 28287 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.007s	sys 0.028s
W20260709 12:18:44.010047 28287 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.007s	sys 0.028s
I20260709 12:18:44.009680 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 623) took 1691 ms. Trace:
W20260709 12:18:44.011260 32746 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:18:44.010865 28990 rpcz_store.cc:276] 0709 12:18:42.318121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.318413 (+   292us) service_pool.cc:224] Handling call
0709 12:18:42.318818 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:42.319270 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.319302 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:42.379273 (+ 59971us) spinlock_profiling.cc:243] Waited 58.6 ms on lock 0x7b2c00095d78. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:18:42.379986 (+   713us) log.cc:844] Serialized 52 byte log entry
0709 12:18:43.061718 (+681732us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:43.061785 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.061874 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.007552 (+945678us) spinlock_profiling.cc:243] Waited 73.6 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:44.009431 (+  1879us) raft_consensus.cc:1685] finished
0709 12:18:44.009454 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.009614 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":470,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":304098816,"thread_start_us":681307,"threads_started":1,"wal-append.queue_time_us":939834}
I20260709 12:18:43.981727 32717 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.013192 28437 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:43.959345 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:43.987560 32715 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.014092 28440 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:44.014420 32715 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:43.970528 28404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.015s	sys 0.043s
I20260709 12:18:43.658035 32675 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.012080 32636 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:43.943535 32758 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.943274 32748 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.992973 32578 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:43.977178 32555 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:43.994045 28285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:44.017640 28285 raft_consensus.cc:606] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:44.017217 32748 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.018383 32758 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.020025 32555 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:43.957139 32554 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.020553 32758 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.018098 28285 raft_consensus.cc:614] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:44.016752 32634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.030s	sys 0.016s
W20260709 12:18:44.023581 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 880676us
I20260709 12:18:44.025264 32748 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:44.016100 28440 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:44.026903 28440 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:44.027756 28440 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.028712 28440 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.999734 32681 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:44.002945 32735 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.509s	user 0.000s	sys 0.014s
W20260709 12:18:44.003610 32638 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.942s	user 0.004s	sys 0.016s
W20260709 12:18:44.032740 28439 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.012s	sys 0.034s
I20260709 12:18:44.039083 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:44.041157 28404 leader_election.cc:400] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14.
I20260709 12:18:44.041574 28404 leader_election.cc:403] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 12:18:44.042459 28439 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.012s	sys 0.034s
W20260709 12:18:44.026381 32742 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:18:44.046759 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:43.982414 32752 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.989154 32733 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.072693 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.074000 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:44.075978 28362 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:44.078073 28360 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:44.059397 32535 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:44.060389 32510 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:44.120424 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.125005 28365 raft_consensus.cc:2368] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:44.057821 32748 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:44.060606 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:44.081182 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 277379us
I20260709 12:18:44.058838 32716 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:44.033797 32516 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.163154 28365 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:44.156570 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 96054.2us
W20260709 12:18:44.148087 28332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.015s	sys 0.015s
W20260709 12:18:44.158874 28257 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.014s	sys 0.013s
W20260709 12:18:44.154385 28435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.007s	sys 0.020s
W20260709 12:18:44.168967 28435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.007s	sys 0.021s
W20260709 12:18:44.171098 28989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.001s	sys 0.027s
W20260709 12:18:44.171669 28989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.001s	sys 0.028s
W20260709 12:18:44.168848 28257 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.015s	sys 0.014s
W20260709 12:18:44.172825 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 621995us
I20260709 12:18:44.047019 32638 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:43.986023 32426 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.178195 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:44.157406 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:44.048393 32636 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14.
W20260709 12:18:44.055438 32741 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 12:18:44.168534 28332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.015s	sys 0.017s
I20260709 12:18:44.195158 28283 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:44.048132 32720 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.046005 32665 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.161413 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:44.180619 28288 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.016s	sys 0.041s
I20260709 12:18:44.182116 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.212343 28362 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:44.186537 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:44.213183 28288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.016s	sys 0.041s
I20260709 12:18:44.176362 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:44.187994 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 493) took 1275 ms. Trace:
I20260709 12:18:44.189958 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.195883 28283 raft_consensus.cc:606] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:44.044579 32641 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.146s
I20260709 12:18:44.059226 32666 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:44.084934 32752 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.123780 32245 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:44.207044 28404 connection.cc:256] RPC call timeout handler was delayed by -2.6138863999999558s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
I20260709 12:18:44.165108 32681 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:44.177667 32728 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:44.194456 32750 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:18:44.171519 32500 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.172369 32517 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.150655 32535 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.175118 32716 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.185801 32748 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:44.214133 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2202) took 1144 ms. Trace:
I20260709 12:18:44.053606 32751 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.166s
I20260709 12:18:44.150338 32510 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.194677 32516 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:44.218214 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.283218 28362 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:44.235785 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.233572 28283 raft_consensus.cc:614] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:44.217572 28440 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:18:44.284487 28440 pending_rounds.cc:85] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 6
I20260709 12:18:44.284708 28440 pending_rounds.cc:107] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:18:44.284922 28440 raft_consensus.cc:2889] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:44.284988 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:44.286042 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.237186 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.238606 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2171) took 1044 ms. Trace:
I20260709 12:18:44.287703 28432 rpcz_store.cc:276] 0709 12:18:43.194321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.194506 (+   185us) service_pool.cc:224] Handling call
0709 12:18:43.195237 (+   731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.195489 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:43.864071 (+668582us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.238248 (+374177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.238312 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:44.238357 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.238422 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.238540 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":80378,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":4046,"raft.queue_time_us":513,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":22319,"thread_start_us":1027213,"threads_started":2}
I20260709 12:18:44.245000 28285 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:44.290388 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.245888 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.234460 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:44.246779 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 409322us
I20260709 12:18:44.278488 28288 rpcz_store.cc:276] 0709 12:18:43.069348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.103423 (+ 34075us) service_pool.cc:224] Handling call
0709 12:18:43.104177 (+   754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.104428 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:43.661711 (+557283us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.213778 (+552067us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.213840 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:44.213886 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.213955 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.214070 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41016,"prepare.queue_time_us":846945,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":88,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":12226,"thread_start_us":1086274,"threads_started":2,"wal-append.queue_time_us":614230}
I20260709 12:18:44.292689 28361 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:44.246763 28292 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:44.260244 28404 connection.cc:256] RPC call timeout handler was delayed by -2.2583864000000631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
I20260709 12:18:44.218011 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:44.295797 28404 connection.cc:256] RPC call timeout handler was delayed by -2.0594863999999689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22450000000000003s for a response
W20260709 12:18:44.296147 28404 connection.cc:256] RPC call timeout handler was delayed by -1.7714779840000574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
W20260709 12:18:44.296407 28404 connection.cc:256] RPC call timeout handler was delayed by -1.7516779840000254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
W20260709 12:18:44.296730 28404 connection.cc:256] RPC call timeout handler was delayed by -1.2719864000000598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31200000000000006s for a response
I20260709 12:18:44.297155 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:44.298429 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.297286 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:44.233436 28989 rpcz_store.cc:276] 0709 12:18:42.913074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.913215 (+   141us) service_pool.cc:224] Handling call
0709 12:18:42.913538 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.913794 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:43.583379 (+669585us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.172285 (+588906us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.186970 (+ 14685us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.187188 (+   218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.187277 (+    89us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:44.187389 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.187495 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.187885 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1052585,"prepare.run_cpu_time_us":169,"prepare.run_wall_time_us":7261,"raft.queue_time_us":99450,"thread_start_us":1238859,"threads_started":2}
I20260709 12:18:44.300581 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:44.298375 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.303254 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.303858 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.305040 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 622) took 1989 ms. Trace:
I20260709 12:18:44.305342 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.305418 28986 rpcz_store.cc:276] 0709 12:18:42.315735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.321474 (+  5739us) service_pool.cc:224] Handling call
0709 12:18:42.321850 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.322473 (+   623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.322517 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.382875 (+ 60358us) spinlock_profiling.cc:243] Waited 59.7 ms on lock 0x7b2c00095d78. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:18:42.383293 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.521610 (+1138317us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:43.521660 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.521772 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.287248 (+765476us) raft_consensus.cc:1685] finished
0709 12:18:44.287275 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.304973 (+ 17698us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":137267584,"thread_start_us":1137900,"threads_started":1,"wal-append.queue_time_us":1281014}
I20260709 12:18:44.307476 28432 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:44.309337 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:44.312163 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.313861 28285 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:44.312124 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:44.255429 32702 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:44.275874 32500 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.317330 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.317819 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.259796 32681 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.315320 28288 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:44.260675 32581 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.323175 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.302457 32245 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.235184 32666 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:44.265951 32638 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:44.334692 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.266126 32573 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:18:44.322885 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2149) took 2364 ms. Trace:
I20260709 12:18:44.327041 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:44.351071 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:44.353101 28404 leader_election.cc:400] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:44.270398 32722 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.327587 28432 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:44.254673 32545 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.305943 32641 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:44.333978 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:44.346033 28288 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:44.371110 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.325901 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:44.327594 32506 consensus_meta.cc:287] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.723s	user 0.004s	sys 0.008s
I20260709 12:18:44.347260 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.372495 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.372906 28359 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:18:44.320482 32702 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.348522 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.353276 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2172) took 1132 ms. Trace:
I20260709 12:18:44.349524 28366 rpcz_store.cc:276] 0709 12:18:41.958877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.959070 (+   193us) service_pool.cc:224] Handling call
0709 12:18:41.959521 (+   451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:42.180007 (+220486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.180043 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:42.198712 (+ 18669us) log.cc:844] Serialized 52 byte log entry
0709 12:18:43.549180 (+1350468us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:43.586797 (+ 37617us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.587145 (+   348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.587218 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.322658 (+735440us) raft_consensus.cc:1685] finished
0709 12:18:44.322680 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.322817 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18117,"prepare.queue_time_us":28645,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":49969,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":2886,"thread_start_us":1321768,"threads_started":2,"wal-append.queue_time_us":1436501}
I20260709 12:18:44.325785 32636 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:18:44.328779 32681 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:44.360172 32686 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.379228 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:44.350821 32523 consensus_meta.cc:287] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.003s
I20260709 12:18:44.380303 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.373117 28285 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:18:44.381878 28292 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:44.371851 28432 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:44.370460 28291 raft_consensus.cc:606] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:44.348773   300 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:44.374950 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.387318 28366 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:44.377878 28440 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.366649 28404 leader_election.cc:403] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:44.342347 32733 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:18:44.328454 32500 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:44.418598 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65631.8us
I20260709 12:18:44.419003 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.419663 28359 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:44.367318 32245 raft_consensus.cc:2764] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:18:44.377441 32702 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.372381 32545 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.371454 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.407735 28440 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:44.414036 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.432333 28366 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 7.
I20260709 12:18:44.368155 32659 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:18:44.351946 32638 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:44.436107   306 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:18:44.382586 32755 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:18:44.436837 32545 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.438257 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:44.384083 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.433543 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.433701 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.437253 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.440917 28432 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:44.442250 28366 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:44.440944 28288 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:18:44.441951 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.381150 32722 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.444504 28432 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:44.445990 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:44.446765 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.448156 28291 raft_consensus.cc:606] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:44.448593 28291 raft_consensus.cc:614] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:44.449671 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 247160us
W20260709 12:18:44.434975   301 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:18:44.451434 28366 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:44.451872 28988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 492) took 1562 ms. Trace:
W20260709 12:18:44.452528 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 295211us
I20260709 12:18:44.458451 28956 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:44.453086 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:44.455245 28441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2169) took 1346 ms (client timeout 1696 ms). Trace:
I20260709 12:18:44.457352 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:44.459285 28441 rpcz_store.cc:269] 0709 12:18:43.108770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.109756 (+   986us) service_pool.cc:224] Handling call
0709 12:18:44.455030 (+1345274us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b540026a7d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:44.455172 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793110784}
I20260709 12:18:44.436829 32638 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:44.457970 28988 rpcz_store.cc:276] 0709 12:18:42.890002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.890175 (+   173us) service_pool.cc:224] Handling call
0709 12:18:42.890509 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.323670 (+433161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.451590 (+1127920us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b5400762d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:44.451801 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":994101120}
I20260709 12:18:44.466305 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:44.462448 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.463007 32535 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.468001 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.430878   300 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:44.428694 32757 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:44.472770 28441 raft_consensus.cc:2410] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:44.368824 32748 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.456395 32716 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.486744 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:44.477377   317 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.112s	user 0.000s	sys 0.005s
I20260709 12:18:44.472232   307 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:44.497476 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2153) took 1817 ms. Trace:
I20260709 12:18:44.497860 28438 rpcz_store.cc:276] 0709 12:18:42.680132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.680286 (+   154us) service_pool.cc:224] Handling call
0709 12:18:42.783991 (+103705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.535850 (+751859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.497280 (+961430us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b5400416754. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:44.497419 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1668818176}
I20260709 12:18:44.495142 28432 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:44.380779 32666 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.454494 32245 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:18:44.480232 32729 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:44.499848 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 203191us
I20260709 12:18:44.496133 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:44.457314 32722 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.480423 32767 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:44.505890 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2212) took 1070 ms. Trace:
I20260709 12:18:44.506006 28284 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:44.472242 32545 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 12:18:44.501439 32765 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 12:18:44.523358 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.483218 32715 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.497835   307 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:44.501581   305 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:18:44.491596 32727 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:18:44.524050 28438 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:44.518138 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:44.515156 28290 rpcz_store.cc:276] 0709 12:18:43.435933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.485143 (+ 49210us) service_pool.cc:224] Handling call
0709 12:18:43.485629 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:43.575061 (+ 89432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:43.575101 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:43.575623 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.983340 (+407717us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:43.983388 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.452422 (+469034us) spinlock_profiling.cc:243] Waited 89 ms on lock 0x7b54000b1dd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:44.452474 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.505571 (+ 53097us) raft_consensus.cc:1685] finished
0709 12:18:44.505593 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.505827 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5210,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":3735,"spinlock_wait_cycles":204609280,"thread_start_us":407350,"threads_started":1,"wal-append.queue_time_us":619651}
I20260709 12:18:44.532419 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:44.534231   300 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:44.534950   308 log.cc:927] Time spent T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:44.515353 32666 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:44.536748 32722 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.537909 32500 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:44.538193   312 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:18:44.551819 32717 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.538928 32701 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:44.518772 32539 consensus_queue.cc:1175] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 14 status { last_received { term: 14 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 14 index: 9 } }
I20260709 12:18:44.501286 32555 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:44.522835 32767 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:44.541327 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.498049 32581 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
W20260709 12:18:44.563912   304 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:18:44.496819 32728 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:18:44.511129 32510 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.564833 32767 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.522549 32578 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:44.565194 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2162) took 1692 ms. Trace:
I20260709 12:18:44.515046 32245 consensus_queue.cc:1243] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:18:44.562783 32760 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
I20260709 12:18:44.565626 28435 rpcz_store.cc:276] 0709 12:18:42.873338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.873558 (+   220us) service_pool.cc:224] Handling call
0709 12:18:42.907666 (+ 34108us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:43.590615 (+682949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:43.590651 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:43.591277 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:18:44.170710 (+579433us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:44.170764 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.170837 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.563357 (+392520us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:44.564837 (+  1480us) raft_consensus.cc:1685] finished
0709 12:18:44.564865 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.565113 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":245434,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":4882176,"thread_start_us":1216787,"threads_started":2,"wal-append.queue_time_us":723113}
I20260709 12:18:44.565340 32510 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.567341 32733 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:44.567728 32733 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:44.567768 32510 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.568701 32733 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:44.569414 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:44.570197   302 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:18:44.571138 32497 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.571623 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2164) took 1390 ms. Trace:
I20260709 12:18:44.567723 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.573275 28435 raft_consensus.cc:2410] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:18:44.574342 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.575709 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.576524 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.570247 32510 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.572207 32727 consensus_queue.cc:1243] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:44.577704 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:44.572023 28367 rpcz_store.cc:276] 0709 12:18:43.180827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.181021 (+   194us) service_pool.cc:224] Handling call
0709 12:18:43.201886 (+ 20865us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:43.209170 (+  7284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:43.209216 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:43.210170 (+   954us) log.cc:844] Serialized 78 byte log entry
0709 12:18:43.858835 (+648665us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:43.858887 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.858965 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.562181 (+703216us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:44.571256 (+  9075us) raft_consensus.cc:1685] finished
0709 12:18:44.571279 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.571537 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1618,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":7843,"spinlock_wait_cycles":13183872,"thread_start_us":636629,"threads_started":1,"wal-append.queue_time_us":1013417}
I20260709 12:18:44.583200 28440 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:18:44.583590 28440 pending_rounds.cc:85] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 7
I20260709 12:18:44.583851 28440 pending_rounds.cc:107] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:18:44.584091 28440 raft_consensus.cc:2889] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:44.584813 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2430) took 1040 ms. Trace:
I20260709 12:18:44.577296 28988 raft_consensus.cc:2368] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14.
W20260709 12:18:44.586366   310 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:18:44.587580   315 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:18:44.588804 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 670) took 1088 ms. Trace:
I20260709 12:18:44.589174 28437 rpcz_store.cc:276] 0709 12:18:43.500252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.014013 (+513761us) service_pool.cc:224] Handling call
0709 12:18:44.014422 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:44.029569 (+ 15147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:44.029607 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:44.036937 (+  7330us) log.cc:844] Serialized 26 byte log entry
0709 12:18:44.373272 (+336335us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:44.373314 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.373382 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.588553 (+215171us) raft_consensus.cc:1685] finished
0709 12:18:44.588577 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.588737 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36450,"raft.queue_time_us":17752,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3482,"thread_start_us":336008,"threads_started":1,"wal-append.queue_time_us":447128}
I20260709 12:18:44.590873 28404 leader_election.cc:400] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14.
I20260709 12:18:44.591223 28404 leader_election.cc:403] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:44.591504 28437 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:44.592213 32245 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14.
I20260709 12:18:44.545048   307 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:44.579015 32545 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:44.595252 32545 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:44.579252 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.516644 32729 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:44.556946   318 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:18:44.558081 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 668) took 1065 ms. Trace:
I20260709 12:18:44.579691 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.598276 28433 rpcz_store.cc:276] 0709 12:18:43.492517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.492661 (+   144us) service_pool.cc:224] Handling call
0709 12:18:43.493024 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:43.503393 (+ 10369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:43.503427 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:43.504015 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.956639 (+452624us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:43.956687 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.956760 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.557825 (+601065us) raft_consensus.cc:1685] finished
0709 12:18:44.557852 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.558012 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":161075,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5026,"thread_start_us":422387,"threads_started":1,"wal-append.queue_time_us":709172}
I20260709 12:18:44.558714 32580 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:44.594254 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 76216.3us
I20260709 12:18:44.559325   325 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:44.559715 32480 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.092s	user 0.012s	sys 0.002s
W20260709 12:18:44.558862   313 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:18:44.610586 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:44.611369 28291 raft_consensus.cc:606] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:44.550308 28332 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:44.554955 32661 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.342s	user 0.007s	sys 0.007s
I20260709 12:18:44.592131 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2431) took 1047 ms. Trace:
I20260709 12:18:44.581476 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.375797 28439 rpcz_store.cc:276] 0709 12:18:43.220576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.220831 (+   255us) service_pool.cc:224] Handling call
0709 12:18:43.233600 (+ 12769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.233897 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:44.078061 (+844164us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.352816 (+274755us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.352879 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:44.352940 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.353015 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.353208 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":89281,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":13667,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":8844,"thread_start_us":1046885,"threads_started":2}
I20260709 12:18:44.578465 28989 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:18:44.599998 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.567574 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:44.612717 28291 raft_consensus.cc:614] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:44.614758 28438 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:44.585132 28440 rpcz_store.cc:276] 0709 12:18:43.544855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.582550 (+1037695us) service_pool.cc:224] Handling call
0709 12:18:44.582867 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.584566 (+  1699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.584756 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:44.604753 32675 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.604887 32715 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.608632 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:44.604943 32727 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
W20260709 12:18:44.589068   316 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:18:44.578833 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.597218 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:44.610316 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.636355 28983 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:44.602385 32733 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:44.612504 28437 rpcz_store.cc:276] 0709 12:18:43.545156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.590817 (+1045661us) service_pool.cc:224] Handling call
0709 12:18:44.591174 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.591886 (+   712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.592079 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:44.594223   322 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:44.619547 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 69534.9us
I20260709 12:18:44.603956 32729 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.643563 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.631130 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:44.636960 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.634791 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2433) took 1089 ms. Trace:
I20260709 12:18:44.646862 28440 rpcz_store.cc:276] 0709 12:18:43.545650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.616178 (+1070528us) service_pool.cc:224] Handling call
0709 12:18:44.634079 (+ 17901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.634304 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:44.634329 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.634369 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:44.634395 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.634450 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.634724 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:44.611732 32426 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:44.649469 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2435) took 1103 ms. Trace:
I20260709 12:18:44.650041 28439 rpcz_store.cc:276] 0709 12:18:43.546281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.635207 (+1088926us) service_pool.cc:224] Handling call
0709 12:18:44.648908 (+ 13701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.649126 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:44.649153 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.649202 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:44.649230 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.649299 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.649409 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:44.656498 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:44.617539 32523 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:44.632786 32767 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:44.633886 32580 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:44.662352 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:44.664695 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:18:44.665889 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.666980 28437 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:18:44.667395 28437 pending_rounds.cc:85] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
I20260709 12:18:44.667610 28437 pending_rounds.cc:107] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:44.667883 28437 raft_consensus.cc:2889] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:44.668630 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2436) took 1122 ms. Trace:
I20260709 12:18:44.668932 28437 rpcz_store.cc:276] 0709 12:18:43.546538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.638766 (+1092228us) service_pool.cc:224] Handling call
0709 12:18:44.658028 (+ 19262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.668321 (+ 10293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.668555 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:44.665717 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:44.663355 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:44.677245 32752 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.680387 32752 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.605072 32702 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.682209 28990 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:44.683801 32702 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.688936 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.689953 28360 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:44.692342 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.578555 32681 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.693835 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:44.696146 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:44.618187 32480 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.628463 32661 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.697082 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:44.657397 32555 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:44.702559 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:44.705014 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:44.704509 28360 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.663182 32701 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:44.707440 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:18:44.709190 28364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56626 (request call id 551) took 2803 ms (client timeout 2757 ms). Trace:
W20260709 12:18:44.709565 28364 rpcz_store.cc:269] 0709 12:18:41.888407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.888526 (+   119us) service_pool.cc:224] Handling call
0709 12:18:44.691378 (+2802852us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54003ba6d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:44.691552 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3694127872}
I20260709 12:18:44.710861 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.712075 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:44.686218 32758 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:44.712646 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.712646 28364 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:44.702477 32537 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.043s	user 0.010s	sys 0.003s
I20260709 12:18:44.656303 32500 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:18:44.701871 32702 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:44.731379 28436 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:44.700672 32659 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:44.664309 32523 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.706941 32578 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:44.703826 32410 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:44.743999 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2429) took 1199 ms. Trace:
I20260709 12:18:44.744793 28435 rpcz_store.cc:276] 0709 12:18:43.544081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.573974 (+1029893us) service_pool.cc:224] Handling call
0709 12:18:44.579300 (+  5326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.579516 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:44.595292 (+ 15776us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.741965 (+146673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.742083 (+   118us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:44.743390 (+  1307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.743644 (+   254us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.743887 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15328,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":167,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":127,"thread_start_us":145474,"threads_started":1}
I20260709 12:18:44.737517 28360 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:44.681355 32670 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:44.741264 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 191672us
I20260709 12:18:44.718107 32581 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:44.644078 32545 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.719508 32758 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.731477 32727 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:44.730140 32701 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:44.736568 32702 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.763818 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.734727 32555 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:44.743468 32410 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:44.768191 28435 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:44.743189 32578 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:44.769686 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 64778.4us
I20260709 12:18:44.771318 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.776242 28364 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
W20260709 12:18:44.774405 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 177306us
I20260709 12:18:44.775413 28360 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
W20260709 12:18:44.780506 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:44.765028 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 57689.8us
I20260709 12:18:44.776870 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.764967 32581 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.735047 32537 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.772979 32758 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.742928 32659 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.786414 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:44.831081 28435 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.782878 32410 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.787149 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.788713 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:44.773496 32665 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:44.776484 32702 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:44.846436 28359 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:18:44.798270 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.848057 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.800163 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.848878 28360 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:44.800977 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2428) took 1257 ms. Trace:
I20260709 12:18:44.848091 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2207) took 1502 ms. Trace:
I20260709 12:18:44.850262 28287 rpcz_store.cc:276] 0709 12:18:43.345298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.345845 (+   547us) service_pool.cc:224] Handling call
0709 12:18:43.346173 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.346396 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:44.010443 (+664047us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.558280 (+547837us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:44.847514 (+289234us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.847732 (+   218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.847781 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:44.847832 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.847901 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.848035 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":567821,"prepare.queue_time_us":605299,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":102,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":15650,"spinlock_wait_cycles":13277952,"thread_start_us":846266,"threads_started":2,"wal-append.queue_time_us":369924}
I20260709 12:18:44.801761 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.802163 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.802794 32510 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.772503 32727 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:44.852563 28435 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:44.790187 32500 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:44.791350 32751 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:18:44.771373   319 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:18:44.811880 32681 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:44.857857 32744 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.355s	user 0.006s	sys 0.025s
I20260709 12:18:44.788190 32581 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:44.861617 32523 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:44.862576 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.863307 28360 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:44.863865 32641 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:44.864343 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.865569 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:44.866127 28360 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:44.815054 32743 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.378s	user 0.002s	sys 0.011s
I20260709 12:18:44.818464 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.820663 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2437) took 1249 ms. Trace:
I20260709 12:18:44.821667 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.822764 32661 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:44.819682   307 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.786603 32715 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:44.825094 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.826117 32701 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.864261 32641 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:44.838536 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:44.780889 32578 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:44.784628 32545 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.848246 28984 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:44.799325 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.919589 28982 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:44.849610 28441 rpcz_store.cc:276] 0709 12:18:43.543784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.481992 (+938208us) service_pool.cc:224] Handling call
0709 12:18:44.482358 (+   366us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:44.482786 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:44.482817 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:44.487879 (+  5062us) log.cc:844] Serialized 52 byte log entry
0709 12:18:44.616587 (+128708us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:44.616637 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.616717 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.800645 (+183928us) raft_consensus.cc:1685] finished
0709 12:18:44.800675 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.800890 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":7342,"thread_start_us":102266,"threads_started":1,"wal-append.queue_time_us":231533}
I20260709 12:18:44.832075 32573 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:44.805964 32670 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:44.922050 32573 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.871440 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2150) took 2912 ms. Trace:
I20260709 12:18:44.925190 28358 rpcz_store.cc:276] 0709 12:18:41.959184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.959298 (+   114us) service_pool.cc:224] Handling call
0709 12:18:41.959621 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.210595 (+250974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.210635 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.400406 (+189771us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.058268 (+657862us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:43.058315 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.058468 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:44.846411 (+1787943us) spinlock_profiling.cc:243] Waited 60.5 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:44.866031 (+ 19620us) raft_consensus.cc:1685] finished
0709 12:18:44.866055 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.871363 (+  5308us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18399,"raft.queue_time_us":50092,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":54,"spinlock_wait_cycles":139212672,"thread_start_us":794952,"threads_started":2,"wal-append.queue_time_us":914939}
I20260709 12:18:44.927440 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.928781 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.872313 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:44.935444 28362 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:44.936518 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:44.937359 28362 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:44.873845 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:44.854858   336 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:18:44.877604 28367 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:18:44.921444 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.870522 28440 rpcz_store.cc:276] 0709 12:18:43.570940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.647981 (+1077041us) service_pool.cc:224] Handling call
0709 12:18:44.662743 (+ 14762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.767276 (+104533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:44.767316 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.767366 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:44.767400 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.820401 (+ 53001us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540026b1d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:44.820450 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.820595 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239812992}
I20260709 12:18:44.828951 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2205) took 1753 ms. Trace:
I20260709 12:18:44.883013 28360 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:18:44.890286 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:44.894946 32659 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:44.895537 32480 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.855515 32665 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:44.952152 32743 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:18:44.898731 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:44.890066   307 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.955041 32665 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:44.955889 28435 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:44.958997 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:44.866340 32727 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:44.965137 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:44.966329 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 127910us
I20260709 12:18:44.967203 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:44.968400 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:44.969283 28435 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:44.970539 28362 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:44.906014   322 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:44.986932 28956 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:44.779597 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:44.997301 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 217778us
I20260709 12:18:44.948935 28286 rpcz_store.cc:276] 0709 12:18:43.075213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.075364 (+   151us) service_pool.cc:224] Handling call
0709 12:18:43.075708 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.075943 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:43.799892 (+723949us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.155111 (+355219us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.828029 (+672918us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b1800415fc0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85c227f3c7 00007f85be8919a7 00007f85be8a10bf 00007f85be8a1038
0709 12:18:44.828280 (+   251us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.828493 (+   213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.828545 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:44.828595 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.828676 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.828886 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":645641,"prepare.queue_time_us":1553645,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":122,"raft.queue_time_us":141,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"spinlock_wait_cycles":12483456,"thread_start_us":1048590,"threads_started":2,"wal-append.queue_time_us":526134}
I20260709 12:18:44.785882 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:45.000497 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:44.978814   315 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:18:45.001976 28362 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:45.006608 28435 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:45.010088 32641 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.011509 32641 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.986148 32686 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:45.018918 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 50551.9us
I20260709 12:18:45.020593 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:45.002791 32245 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:45.021486 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 181511us
I20260709 12:18:44.869186 32510 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:44.872502 32500 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.032593 32500 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:44.963822 32715 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:44.978088 32663 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:45.024721 32245 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:45.034238 32245 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:45.035601 32715 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:45.036393 28365 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
W20260709 12:18:45.037699 32736 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.571s	user 0.007s	sys 0.010s
I20260709 12:18:45.039979 28367 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:18:45.025933 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:45.041522 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:18:45.028111 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.028858 32523 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:45.030632 32510 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:45.044288 32523 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:45.049202 32523 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.022490   340 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:45.051654   340 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:45.052080   340 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.054816 28358 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:45.018421 32751 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:45.058563 32510 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.058804 32751 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:45.059209 32510 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:45.060052 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.061990 32510 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:45.063504 32573 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:45.069604 28435 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:45.066500 28289 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2201) took 1997 ms (client timeout 1771 ms). Trace:
W20260709 12:18:45.072216 28289 rpcz_store.cc:269] 0709 12:18:43.068604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.200257 (+131653us) service_pool.cc:224] Handling call
0709 12:18:45.066265 (+1866008us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54003ab954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:45.066413 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737761536}
I20260709 12:18:45.073009 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2155) took 2391 ms. Trace:
I20260709 12:18:45.073455 28436 rpcz_store.cc:276] 0709 12:18:42.681430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.681789 (+   359us) service_pool.cc:224] Handling call
0709 12:18:42.783722 (+101933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.732320 (+1948598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.072725 (+340405us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54000d8c54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:45.072932 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4478711552}
W20260709 12:18:45.076594 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.023918 32686 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:45.091132 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.101243 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:45.103706 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.104311 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.109294 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 323619us
I20260709 12:18:45.047859 32663 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:45.040288   319 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:18:45.077308 32661 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:45.099634 32670 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.065680   340 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.157136 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.115813 32659 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:18:45.148691 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.152179 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 151753us
I20260709 12:18:45.144297 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:45.160410 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.161249 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2427) took 1617 ms. Trace:
I20260709 12:18:45.164044 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:45.169751 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:45.168776 28434 rpcz_store.cc:276] 0709 12:18:43.543531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.068101 (+524570us) service_pool.cc:224] Handling call
0709 12:18:44.455627 (+387526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.455874 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:44.478146 (+ 22272us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.600991 (+122845us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.601443 (+   452us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.601494 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:44.601549 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.601616 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:44.816944 (+215328us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b1800458aa0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:45.161044 (+344100us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540026a7d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:45.161173 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15400,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":4018,"raft.queue_time_us":1644,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":213,"spinlock_wait_cycles":893851264,"thread_start_us":120304,"threads_started":1,"wal-append.queue_time_us":276485}
I20260709 12:18:45.066128 32758 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.117805 32663 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:45.135450 32573 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:45.173453 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.178125 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 629) took 1087 ms. Trace:
I20260709 12:18:45.178649 28990 rpcz_store.cc:276] 0709 12:18:44.073542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.073712 (+   170us) service_pool.cc:224] Handling call
0709 12:18:45.160475 (+1086763us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54007413d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:45.160619 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210650496}
I20260709 12:18:45.184285 32510 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.185918 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:45.102763 32757 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:18:45.186760 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:45.187777 28986 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:45.155920 32480 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:45.184980 32510 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.192199 32510 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:45.193271 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.193842 32715 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:45.194607 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.059831 32751 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.146052 32500 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:45.112821 32451 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:18:45.099572   300 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:18:45.195827 28292 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:18:45.194336   340 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:45.203293 28292 pending_rounds.cc:85] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 7
I20260709 12:18:45.203581 28292 pending_rounds.cc:107] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:18:45.203857 28292 raft_consensus.cc:2889] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:45.150079 32661 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:45.139292 32581 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.202334 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:45.185048 32758 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:45.211156 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.176389 32663 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.195266 28290 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 9.
I20260709 12:18:45.216707 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:45.217392 28290 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.164842 32555 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:18:45.172875 32638 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:45.194748 32752 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:45.224298 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.167358 32670 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.211800 32661 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:45.240168 32733 consensus_meta.cc:287] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.000s
I20260709 12:18:45.240334 32555 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:45.243400 32410 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.274744 28433 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:45.244159   341 raft_consensus.cc:993] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:45.245674 32581 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:45.249712 28286 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:45.199772 32715 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:45.287199 32715 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.213038 32480 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:45.284145 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2438) took 1695 ms. Trace:
I20260709 12:18:45.290300 28439 rpcz_store.cc:276] 0709 12:18:43.588712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.651090 (+1062378us) service_pool.cc:224] Handling call
0709 12:18:44.675390 (+ 24300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:44.691772 (+ 16382us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:44.691812 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:44.701583 (+  9771us) log.cc:844] Serialized 52 byte log entry
0709 12:18:44.788731 (+ 87148us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:44.788774 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.788841 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:45.055972 (+267131us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b4800210258. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c37b0aa1 00007f85c37a7b37 00007f85c37a79ca 00007f85c389f26f 00007f85c389ef37 00007f85c38b355c 00007f85c38b3508 00007f85c38b3488 00007f85c38b3443 00007f85c38b26ec 00007f85c27dca96
0709 12:18:45.258883 (+202911us) raft_consensus.cc:1685] finished
0709 12:18:45.258909 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.284038 (+ 25129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11383,"raft.run_cpu_time_us":3253,"raft.run_wall_time_us":364514,"spinlock_wait_cycles":12770048,"thread_start_us":52256,"threads_started":1,"wal-append.queue_time_us":325837}
I20260709 12:18:45.189971 32686 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.206462 32500 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.209153 32717 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:18:45.250109 32748 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.212801 32510 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:18:45.219544 32541 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:18:45.220427   332 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.159s
I20260709 12:18:45.257827 32555 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:45.249326 32638 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:45.336177 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 131311us
I20260709 12:18:45.313336 32681 raft_consensus.cc:1081] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:45.313912 32480 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:45.257311 32523 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:45.341514 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.343456 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 148046us
I20260709 12:18:45.318238 32727 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.321220 32748 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.358892 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:45.370005 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 167738us
I20260709 12:18:45.371248 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:45.373410 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:45.375111 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 230958us
I20260709 12:18:45.377859 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:45.381371 28290 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:45.382884 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.405085 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:45.407068 28285 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:45.389158 32573 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:45.410643 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.418594 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 246905us
I20260709 12:18:45.367965 32659 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:45.422216 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:45.414634   333 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:45.428685 28436 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:45.422621 32578 consensus_meta.cc:287] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.000s
I20260709 12:18:45.407136 32757 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:45.469795 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47741us
I20260709 12:18:45.466200 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.474828 28439 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.418707 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:45.475895 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 130370us
I20260709 12:18:45.476634 28290 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:45.417665 32681 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:45.387665 32717 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:18:45.429040   340 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:45.370509 32555 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:45.466961 32545 consensus_meta.cc:287] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.000s
I20260709 12:18:45.472460   333 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.494529 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.514407   333 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:45.516127 32641 consensus_meta.cc:287] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.002s
I20260709 12:18:45.516850 32641 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:45.462436   345 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 12:18:45.491963 32757 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.518514   357 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.519143   357 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.519552 32757 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:45.520181 32757 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.520648 32757 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.421904 32480 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.486104 32578 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.522773 32638 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:45.523633 32578 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:45.493289   362 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:45.502455   347 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:18:45.502816 32410 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:45.510326 32748 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:45.526822 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2440) took 1898 ms. Trace:
I20260709 12:18:45.526872   352 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:45.527222 28437 rpcz_store.cc:276] 0709 12:18:43.628196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.669699 (+1041503us) service_pool.cc:224] Handling call
0709 12:18:44.670876 (+  1177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:44.672050 (+  1174us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:44.711773 (+ 39723us) log.cc:844] Serialized 14 byte log entry
0709 12:18:45.057808 (+346035us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b1800208420. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be42723a 00007f85be891a2a
0709 12:18:45.258311 (+200503us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:45.282497 (+ 24186us) log.cc:844] Serialized 26 byte log entry
0709 12:18:45.282671 (+   174us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:45.282709 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.282783 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:45.526501 (+243718us) raft_consensus.cc:1685] finished
0709 12:18:45.526598 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.526756 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1609,"prepare.queue_time_us":39699,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":245958,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":18005,"spinlock_wait_cycles":27704576,"thread_start_us":200267,"threads_started":1,"wal-append.queue_time_us":689710}
W20260709 12:18:45.530951 32702 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.000s
I20260709 12:18:45.531530 32702 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.532850 28292 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:45.497107 32675 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:45.535574 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.533721 28439 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
W20260709 12:18:45.503137   348 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:18:45.503733   358 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.506224 32555 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.507902 28289 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:45.508864 32670 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:45.508601 32245 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.510314 32727 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:45.511397 32523 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:45.513075 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:45.513834 32500 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:45.527678 32451 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 12:18:45.539242 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:45.544438 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.540438 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.629597 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:45.630985 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:45.563982 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.606472 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:45.632544 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.633832 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:45.634617 28292 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:45.579481 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.571318 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.576287 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.612210 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:45.635771 28435 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:45.636857 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:45.635067 28436 raft_consensus.cc:2376] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12. Re-sending same reply.
W20260709 12:18:45.638614 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 151766us
I20260709 12:18:45.638988 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.639580 28436 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.640134 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.517397 32641 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.521556   357 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:45.615227 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.615360 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.620973 28358 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:45.648084 28439 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:18:45.537586 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 200237us
I20260709 12:18:45.496708   340 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:45.653929 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:45.530982 32757 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:45.636857 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.655215 28435 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.430475 32541 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.207s
I20260709 12:18:45.642635 28432 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:45.643776 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:45.642125 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:45.650192 28288 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:45.652786 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.496358 32659 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:45.660336 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 179851us
I20260709 12:18:45.410467 32573 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.637290 32510 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:45.667958 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:45.668273 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:45.667254 28435 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:45.684665 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.685899 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.687075 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.571022 32661 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.668584 28360 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:45.697238 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:45.698550 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:45.698724 32641 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:45.663072 28292 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:45.699826 32663 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.701016 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.701377 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:45.663901 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.702497 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:45.700038 28360 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:45.689711 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.704885 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.705494 28439 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:45.685932 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:45.706661 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.707396 28439 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:45.707950 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:45.695004 32661 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:45.695776 32702 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:45.662415   340 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:45.652232 32670 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.647673   353 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:45.696030 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.703224 28292 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:18:45.714226 28435 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
W20260709 12:18:45.719018 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45764.3us
I20260709 12:18:45.719554 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.719710 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.702487 28288 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:45.720846 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.721276 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.714847 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:45.721853 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:45.662557 32753 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:45.715160   346 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 12:18:45.672887 32539 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:45.708217 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:45.746052 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.695951   338 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.886s	user 0.002s	sys 0.004s
I20260709 12:18:45.715870 32641 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:45.716912 32663 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:45.717600 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 491) took 2840 ms. Trace:
I20260709 12:18:45.716915 28365 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:45.723635 28435 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:45.754885 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.683400   357 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:45.739312 28436 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:45.720508 32686 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:45.745038 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 76856.6us
I20260709 12:18:45.745256 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:45.761333 28954 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:45.761490 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.681941   332 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:45.746269 28439 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:45.766314 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.767086 28439 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.747561 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2189) took 1062 ms. Trace:
I20260709 12:18:45.750490 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.771505 28440 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.771661 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:45.724788 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:45.612897   358 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.762547 28436 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:45.730181 32757 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:45.786034 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:45.773922 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.774183 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:45.786767 28983 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.788400 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:45.777189 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:45.787421 28436 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:45.756366 28985 rpcz_store.cc:276] 0709 12:18:42.877663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.901880 (+ 24217us) service_pool.cc:224] Handling call
0709 12:18:42.902302 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.276969 (+374667us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:43.969666 (+692697us) log.cc:844] Serialized 14 byte log entry
0709 12:18:44.488091 (+518425us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:44.488160 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:44.488215 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.717344 (+1229129us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54004c18d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:45.717393 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.717509 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":321,"prepare.queue_time_us":706675,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":47,"raft.queue_time_us":77011,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":15580,"spinlock_wait_cycles":860947840,"thread_start_us":1184111,"threads_started":3,"wal-append.queue_time_us":488551,"wal-append.run_cpu_time_us":2010,"wal-append.run_wall_time_us":1139395}
I20260709 12:18:45.799248 28439 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:45.800595 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.801468 28439 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:45.748412 32541 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:45.753352   319 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:45.753105 32641 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:45.802412 32541 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.819094 28440 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:45.820335 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.822382 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:45.823515 28440 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:18:45.824518   367 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:18:45.828023   358 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:45.828337   334 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.020s	user 0.003s	sys 0.000s
I20260709 12:18:45.790308   332 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:45.829989   332 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.837988 28983 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:45.840534 32537 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:45.787910   356 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:45.866750   356 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:45.867941   356 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.848812   309 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:45.744313 32753 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:45.809834   319 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:45.815117 32641 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:45.859335 32733 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:18:45.865471   370 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
I20260709 12:18:45.835502 32541 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.880082   319 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:45.876746   309 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:45.882954 32733 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:45.929330 32733 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.946103   340 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:45.976831   335 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.167s	user 0.003s	sys 0.001s
I20260709 12:18:45.977380   340 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:45.979197   329 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.171s	user 0.000s	sys 0.003s
I20260709 12:18:45.990895   333 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:45.983471   337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.168s	user 0.000s	sys 0.023s
W20260709 12:18:46.033047 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.033581 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.723106 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.036394 28283 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:46.135051 28283 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:46.135598 28288 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:46.136780 28283 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:45.722189 28292 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:46.138573 32245 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:46.139897 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:45.770263 28361 rpcz_store.cc:276] 0709 12:18:44.684895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.415976 (+731081us) service_pool.cc:224] Handling call
0709 12:18:45.421186 (+  5210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:45.423282 (+  2096us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:45.434459 (+ 11177us) log.cc:844] Serialized 14 byte log entry
0709 12:18:45.747141 (+312682us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:45.747211 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:45.747265 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.747347 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.747490 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13465,"prepare.queue_time_us":1667,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":12559,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":9301,"thread_start_us":302561,"threads_started":1,"wal-append.queue_time_us":464675}
I20260709 12:18:46.047976 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 554) took 1567 ms. Trace:
W20260709 12:18:46.055009 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.055935 28434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.016s	sys 0.041s
W20260709 12:18:46.056650 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.802551 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:46.062582 28437 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:46.069531 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2136) took 1503 ms. Trace:
I20260709 12:18:46.071913 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 555) took 1589 ms. Trace:
I20260709 12:18:46.073796 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.079767 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.080772 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.045431 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2192) took 1265 ms. Trace:
I20260709 12:18:46.117489   333 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:46.055824 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 330976us
I20260709 12:18:46.131475 32675 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:46.038777 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2122) took 1792 ms. Trace:
W20260709 12:18:46.142800 28434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.017s	sys 0.041s
I20260709 12:18:46.143045 28366 rpcz_store.cc:276] 0709 12:18:44.480802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.480933 (+   131us) service_pool.cc:224] Handling call
0709 12:18:44.481274 (+   341us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:44.485011 (+  3737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:44.485040 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:44.493184 (+  8144us) log.cc:844] Serialized 52 byte log entry
0709 12:18:44.618092 (+124908us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:44.618139 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.618209 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:45.946913 (+1328704us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:46.047218 (+100305us) raft_consensus.cc:1685] finished
0709 12:18:46.047280 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.047740 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3331,"raft.queue_time_us":3909,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":6648,"spinlock_wait_cycles":70102272,"thread_start_us":102662,"threads_started":1,"wal-append.queue_time_us":262739}
W20260709 12:18:46.146319 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.147277 28283 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.147789 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.166142 32733 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.968524   309 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.169701   368 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.171636 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2432) took 2626 ms. Trace:
I20260709 12:18:46.172364 32537 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:46.142215 32753 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:46.137357 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.178671 28367 rpcz_store.cc:276] 0709 12:18:44.566422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.951072 (+384650us) service_pool.cc:224] Handling call
0709 12:18:44.951641 (+   569us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:45.040521 (+ 88880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.068716 (+1028195us) spinlock_profiling.cc:243] Waited 88 ms on lock 0x7b5400135954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:46.069310 (+   594us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202445568}
I20260709 12:18:46.179070 28359 rpcz_store.cc:276] 0709 12:18:44.482011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.482148 (+   137us) service_pool.cc:224] Handling call
0709 12:18:44.484582 (+  2434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:44.846927 (+362345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.071114 (+1224187us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54001d8154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:46.071648 (+   534us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825117952}
I20260709 12:18:46.182624 28284 rpcz_store.cc:276] 0709 12:18:44.246459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.246881 (+   422us) service_pool.cc:224] Handling call
0709 12:18:44.505729 (+258848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.515301 (+  9572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.038217 (+1522916us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54000b1dd0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:46.038510 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594573696}
I20260709 12:18:46.188174 28433 rpcz_store.cc:276] 0709 12:18:43.545413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.609493 (+1064080us) service_pool.cc:224] Handling call
0709 12:18:44.609873 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:45.275328 (+665455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.170966 (+895638us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b5400185954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:46.171401 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528042880}
I20260709 12:18:46.193565 32675 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.179518 28360 rpcz_store.cc:276] 0709 12:18:44.779431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.704694 (+925263us) service_pool.cc:224] Handling call
0709 12:18:45.705019 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:45.705251 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:45.705577 (+   326us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.043606 (+338029us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.044233 (+   627us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.044426 (+   193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:46.044498 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:46.044645 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.044774 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.045064 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":56478,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":201796,"raft.run_cpu_time_us":839,"raft.run_wall_time_us":122722,"spinlock_wait_cycles":6361728,"thread_start_us":336425,"threads_started":1,"wal-append.queue_time_us":265460}
W20260709 12:18:46.202909   337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.304s	user 0.000s	sys 0.024s
I20260709 12:18:46.213380 28289 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:46.219348 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.224686 28292 raft_consensus.cc:2410] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:46.221720 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.226841 28291 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:46.234385 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:46.234752 32580 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.000s	sys 0.019s
W20260709 12:18:46.235296 32580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.000s	sys 0.019s
I20260709 12:18:46.236629 28287 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:46.237454 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 625) took 1045 ms. Trace:
W20260709 12:18:46.238162 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.238426 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 537007us
W20260709 12:18:46.238812 28255 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.019s	sys 0.005s
W20260709 12:18:46.239197 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 508195us
I20260709 12:18:46.241976 28288 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:46.243229 28287 rpcz_store.cc:276] 0709 12:18:45.191540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.235697 (+1044157us) service_pool.cc:224] Handling call
0709 12:18:46.236308 (+   611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.237119 (+   811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.237370 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:46.249167 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.250962   300 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:46.252535   300 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:46.252898   300 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:46.253888 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.254618 28255 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.020s	sys 0.005s
W20260709 12:18:46.261981 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 725851us
I20260709 12:18:46.262686 28332 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.265277 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:46.215116 32537 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.266181   396 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:46.219743 32753 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:46.238313 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:46.237995 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.267376   300 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.269789 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:46.274952 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:46.279101 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 627) took 1064 ms. Trace:
I20260709 12:18:46.283121 28288 rpcz_store.cc:276] 0709 12:18:45.192128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.241081 (+1048953us) service_pool.cc:224] Handling call
0709 12:18:46.241611 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.256576 (+ 14965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.256804 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.286669 28364 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:46.291440 28289 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:18:46.291644 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2220) took 1094 ms. Trace:
I20260709 12:18:46.291891 28289 pending_rounds.cc:85] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 9
I20260709 12:18:46.292179 28289 pending_rounds.cc:107] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:18:46.292436 28289 raft_consensus.cc:2889] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:46.293290 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2218) took 1102 ms. Trace:
I20260709 12:18:46.242115 28436 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:18:46.294157 28436 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:46.294749 28436 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.282261 32638 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:46.293601 28289 rpcz_store.cc:276] 0709 12:18:45.191318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.243012 (+1051694us) service_pool.cc:224] Handling call
0709 12:18:46.290923 (+ 47911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.293015 (+  2092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.293217 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20864}
I20260709 12:18:46.297544 28291 rpcz_store.cc:276] 0709 12:18:45.197393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.280753 (+1083360us) service_pool.cc:224] Handling call
0709 12:18:46.290940 (+ 10187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.291252 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.291277 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:46.291321 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:46.291360 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.291413 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.291547 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28800}
I20260709 12:18:46.303565   385 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:46.306032 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:46.225589   368 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.311221 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:46.311942 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:46.313762 32659 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.314535 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:46.315148 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 241815us
I20260709 12:18:46.315946 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:46.318454   309 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.323668 32537 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:46.324841 28292 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:46.325989 32748 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:46.329047 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2226) took 1081 ms. Trace:
I20260709 12:18:46.343128 28291 rpcz_store.cc:276] 0709 12:18:45.248013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.309940 (+1061927us) service_pool.cc:224] Handling call
0709 12:18:46.328333 (+ 18393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.328673 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.328700 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:46.328758 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:46.328790 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.328845 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.328959 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.330161 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:46.331171 32663 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.358168 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 628) took 1165 ms. Trace:
I20260709 12:18:46.362000 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:46.363055 28283 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:46.361629 28292 rpcz_store.cc:276] 0709 12:18:45.192383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.278031 (+1085648us) service_pool.cc:224] Handling call
0709 12:18:46.324488 (+ 46457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.357838 (+ 33350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.358079 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.361534 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:46.282465 32541 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:46.371311 28283 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2124) took 2098 ms (client timeout 2008 ms). Trace:
W20260709 12:18:46.379596 28283 rpcz_store.cc:269] 0709 12:18:44.268772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.284704 (+ 15932us) service_pool.cc:224] Handling call
0709 12:18:46.035762 (+1751058us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b54000f1020. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7fedc0 00007f85be88ae18 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:46.366912 (+331150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100495104}
I20260709 12:18:46.389333 28360 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.390094 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:46.391342   368 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:46.393780 28361 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.399238 28358 raft_consensus.cc:2410] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:18:46.389539 32728 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:18:46.401783 28436 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.413072 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:46.427980 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2227) took 1117 ms. Trace:
I20260709 12:18:46.428504 28288 rpcz_store.cc:276] 0709 12:18:45.310775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.308318 (+997543us) service_pool.cc:224] Handling call
0709 12:18:46.383625 (+ 75307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.421348 (+ 37723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.421384 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:46.427627 (+  6243us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:46.427681 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.427750 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.427909 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.433872 32659 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.435842   340 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:46.436151 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:46.436748   340 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.437265 32659 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:46.387569   353 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:46.439319   322 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:46.439208   353 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.439731 32539 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:46.441947 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 212997us
I20260709 12:18:46.455377 28291 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:18:46.455852 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.458416 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2230) took 1099 ms. Trace:
I20260709 12:18:46.460104 28291 rpcz_store.cc:276] 0709 12:18:45.359319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.357621 (+998302us) service_pool.cc:224] Handling call
0709 12:18:46.388438 (+ 30817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.455855 (+ 67417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.458343 (+  2488us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.460739 28436 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:46.461413 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 196408us
W20260709 12:18:46.468797 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 106780us
I20260709 12:18:46.470935 32541 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:46.483944 32426 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:46.493811 28332 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.454659 32663 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.460242 32537 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:18:46.480623   337 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:46.489493 32245 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:46.505867   403 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:18:46.497407 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:18:46.512836 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.513127 28291 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 12:18:46.484006 32686 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.494462 32426 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:46.522308 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:46.522529 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 630) took 1046 ms. Trace:
I20260709 12:18:46.522987 28291 rpcz_store.cc:276] 0709 12:18:45.476091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.465136 (+989045us) service_pool.cc:224] Handling call
0709 12:18:46.498121 (+ 32985us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.522176 (+ 24055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.522459 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.526906 32245 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:46.513813 32555 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.895s	user 0.006s	sys 0.015s
I20260709 12:18:46.507009   340 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:46.548009 32426 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.549360 32426 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.550081 28437 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:46.551818 28437 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:46.552127   353 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:46.552799 28435 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:18:46.554003 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.562098 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 77269.7us
I20260709 12:18:46.562783 28437 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:46.563696 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:46.564688 28437 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:46.566054   358 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:46.566586   340 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:46.566975 32753 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:46.567492 32753 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:46.567943 32753 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:46.570779 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 96096.2us
I20260709 12:18:46.572038 32729 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:46.572695 32729 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:46.573184 32729 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.565997 32541 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:46.574182 32541 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:46.566661   358 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:46.575727   358 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:46.576874 32757 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:46.577615 32757 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.573526 28332 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:46.580547 32757 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:46.580842 32510 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:46.581306 32510 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:46.581715 32510 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:46.582176 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 84604us
I20260709 12:18:46.581075 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.583774 28437 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:46.584148 32554 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:46.584692 32554 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:46.584771 28332 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.585197 32554 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.575495 32541 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.559902 32245 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:46.557034   337 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.566161 28290 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
W20260709 12:18:46.575078 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 99983.8us
I20260709 12:18:46.585671 28437 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:18:46.592195 32641 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.629s	user 0.013s	sys 0.003s
I20260709 12:18:46.591974   368 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:46.600533   368 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:46.600466 32641 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.601059   368 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.603046 32641 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:46.587196 32541 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.610373 28438 raft_consensus.cc:2410] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:46.613710   356 consensus_queue.cc:1059] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:46.614905   375 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:46.616194   332 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:46.616664   332 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:46.617084 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:46.617928   332 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.619804 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:46.620903 32245 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:46.621359 32245 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:46.621886 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.622076 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.623117   307 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:46.624070 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:46.626688   307 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:46.542912 32686 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.632066 32686 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:46.632875 32686 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:18:46.634029 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 133645us
I20260709 12:18:46.634603 32245 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:46.636619 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.638942 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 689) took 1215 ms. Trace:
I20260709 12:18:46.639674 28434 rpcz_store.cc:276] 0709 12:18:45.423459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.423626 (+   167us) service_pool.cc:224] Handling call
0709 12:18:45.424043 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.208723 (+784680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.208794 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:46.262360 (+ 53566us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.369160 (+106800us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:46.369201 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.382768 (+ 13567us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.638671 (+255903us) raft_consensus.cc:1685] finished
0709 12:18:46.638693 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.638864 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97944,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"thread_start_us":864941,"threads_started":2,"wal-append.queue_time_us":236112}
I20260709 12:18:46.641993   375 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:18:46.642736   337 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:46.622970 28285 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:46.643324   324 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:18:46.643877   324 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:46.645037   324 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.640755 28433 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:46.639791   333 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:18:46.658830   307 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:46.659284   307 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:46.592571 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2225) took 1344 ms. Trace:
I20260709 12:18:46.659938 28290 rpcz_store.cc:276] 0709 12:18:45.247727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.494226 (+246499us) service_pool.cc:224] Handling call
0709 12:18:46.592328 (+1098102us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003da4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:46.592497 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2394921216}
I20260709 12:18:46.660334   307 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.662580 28435 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 13. Ops: []
I20260709 12:18:46.663497 28286 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:46.646935 28436 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:46.647951   324 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:46.648422   300 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.671015 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2140) took 1070 ms. Trace:
I20260709 12:18:46.648923 32641 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.651444 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2198) took 1020 ms. Trace:
W20260709 12:18:46.665059 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.565516 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:46.621582 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.619705   332 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:46.633562 32686 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:46.616496   319 consensus_meta.cc:287] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.707s	user 0.006s	sys 0.007s
I20260709 12:18:46.664229 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2241) took 1136 ms. Trace:
I20260709 12:18:46.682163 28285 rpcz_store.cc:276] 0709 12:18:45.527976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.621484 (+1093508us) service_pool.cc:224] Handling call
0709 12:18:46.664163 (+ 42679us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.674175 28432 rpcz_store.cc:276] 0709 12:18:45.630842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.077819 (+446977us) service_pool.cc:224] Handling call
0709 12:18:46.078341 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.078917 (+   576us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:46.186674 (+107757us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.650646 (+463972us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b18002fcda0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be42723a
0709 12:18:46.650904 (+   258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:46.650964 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:46.651018 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.651096 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.651257 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":151625,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":23734,"raft.queue_time_us":1989,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":136,"spinlock_wait_cycles":2692352,"thread_start_us":195463,"threads_started":2,"wal-append.queue_time_us":291115}
I20260709 12:18:46.683368 32641 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.671572 28286 rpcz_store.cc:276] 0709 12:18:45.600178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.625397 (+1025219us) service_pool.cc:224] Handling call
0709 12:18:46.663215 (+ 37818us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.670688 (+  7473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.670948 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.676963 28361 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:46.673002 28283 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:46.689131 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2144) took 1087 ms. Trace:
I20260709 12:18:46.689527 28283 rpcz_store.cc:276] 0709 12:18:45.601278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.652909 (+1051631us) service_pool.cc:224] Handling call
0709 12:18:46.665928 (+ 13019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.688797 (+ 22869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.689050 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:46.692688 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2196) took 1084 ms. Trace:
I20260709 12:18:46.693195 28439 rpcz_store.cc:276] 0709 12:18:45.608728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.064651 (+455923us) service_pool.cc:224] Handling call
0709 12:18:46.065273 (+   622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.068419 (+  3146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.068465 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:46.069369 (+   904us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.188037 (+118668us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:46.188538 (+   501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.188619 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.692264 (+503645us) raft_consensus.cc:1685] finished
0709 12:18:46.692295 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.692476 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60405,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":258,"thread_start_us":118108,"threads_started":1,"wal-append.queue_time_us":448525}
I20260709 12:18:46.687369 32641 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:46.698285   364 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:46.699306 32663 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:46.700920 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.696691 28283 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:18:46.702728 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 137324us
I20260709 12:18:46.702970 28290 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:46.705044 32555 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:46.707340 28360 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:46.685382   319 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.701908   307 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.706398 28185 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:46.686772 32686 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:46.721100 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.708309 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.717322 32555 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:46.731962 32555 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.706849 32757 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:46.699129   364 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.766184 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:46.769279 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 147859us
W20260709 12:18:46.770764 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.782714 28283 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:46.791069   356 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:46.796983 32675 consensus_meta.cc:287] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.553s	user 0.006s	sys 0.007s
I20260709 12:18:46.797531 32675 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.799561 32245 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.800637 28985 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:46.801054 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:46.803105 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.781589 32537 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:18:46.811095 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 121957us
I20260709 12:18:46.820807 28185 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:46.822904 28436 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:46.826836 28290 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:46.826300 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:46.835394 28283 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:18:46.823313 29016 heartbeater.cc:646] Failed to heartbeat to 127.26.123.62:36149 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.26.123.62:36149 timed out after 15.000s (SENT)
W20260709 12:18:46.840806 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 106564us
I20260709 12:18:46.823390 28185 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:46.842859 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:46.843415 28185 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:46.852480   375 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:18:46.850375 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:46.856952 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:46.857630 32541 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.824623 32675 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:46.854038   309 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:46.825069   356 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:18:46.862119   309 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:46.863291   309 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.864841   309 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.866632 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.868674 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:46.869681 32675 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:46.870519 32675 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:46.877004   340 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:46.877768   340 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:46.878831   340 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.882339 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:46.851064 32245 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:46.845948 32728 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:46.893790 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2222) took 1685 ms. Trace:
I20260709 12:18:46.889715   363 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:46.894255 28287 rpcz_store.cc:276] 0709 12:18:45.208891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.309853 (+1100962us) service_pool.cc:224] Handling call
0709 12:18:46.387701 (+ 77848us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:46.433296 (+ 45595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.433340 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:46.479884 (+ 46544us) log.cc:844] Serialized 52 byte log entry
0709 12:18:46.706871 (+226987us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:46.714548 (+  7677us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.715103 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.715302 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.893454 (+178152us) raft_consensus.cc:1685] finished
0709 12:18:46.893477 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.893673 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7017,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":580,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":301,"thread_start_us":189054,"threads_started":1,"wal-append.queue_time_us":313165}
I20260709 12:18:46.895555 32510 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.898563   325 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:46.899156   325 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:46.899644   325 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.899263 32753 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.900671 28332 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:46.901825 32675 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.902139 32728 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:46.902763 32728 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:46.903708 32728 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:46.845675 28185 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 3 to 8, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 8 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:46.886380   319 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:46.845522 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:46.838402 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:18:46.855031   404 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:18:46.921072 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 351525us
I20260709 12:18:46.891055   364 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:46.929237 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2238) took 1395 ms. Trace:
I20260709 12:18:46.930033 28291 rpcz_store.cc:276] 0709 12:18:45.527139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.539886 (+1012747us) service_pool.cc:224] Handling call
0709 12:18:46.604120 (+ 64234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:46.604653 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:46.604997 (+   344us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.921910 (+316913us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:46.921976 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:46.922149 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.922228 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.922383 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":836,"prepare.queue_time_us":418,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":5105,"thread_start_us":268850,"threads_started":1}
I20260709 12:18:46.918466   356 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:18:46.933472   363 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.962054   362 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:46.964068 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.964555 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:18:46.966676 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 121259us
I20260709 12:18:46.935866 32753 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:46.953823 32245 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:46.949481 32510 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:46.975783 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.971755 32578 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:46.984793   396 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:18:46.987551 28190 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 224) took 25445 ms (client timeout 15000 ms). Trace:
W20260709 12:18:46.990800 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 182367us
W20260709 12:18:46.992901 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 92354.5us
I20260709 12:18:46.987984 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:46.996091 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 169860us
I20260709 12:18:47.000020 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2219) took 1808 ms. Trace:
W20260709 12:18:46.978195 32733 consensus_meta.cc:287] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.776s	user 0.016s	sys 0.000s
I20260709 12:18:46.971905 32245 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:46.961031 32541 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:46.990176   364 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:47.009955 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:47.000830 28284 rpcz_store.cc:276] 0709 12:18:45.191612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.238855 (+1047243us) service_pool.cc:224] Handling call
0709 12:18:46.239368 (+   513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.285619 (+ 46251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.285654 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:46.286309 (+   655us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.367015 (+ 80706us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:46.367060 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.367145 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:46.999766 (+632621us) raft_consensus.cc:1685] finished
0709 12:18:46.999791 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.999946 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38181,"raft.queue_time_us":44733,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":177,"thread_start_us":79465,"threads_started":1,"wal-append.queue_time_us":400659}
I20260709 12:18:47.011476 32245 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:47.034614   368 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:47.035117 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.035339 32541 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:47.035706 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:47.037547 28432 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:47.006630 28185 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 6 to 11, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4), now has a pending config: VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1), VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2), VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3), VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 11 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:47.034637 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:47.043843 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.062347 28985 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:47.007834 32510 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:18:47.008073 32729 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:46.999836   363 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:47.063622 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.010280 32753 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 12:18:47.079154   419 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:18:47.081486 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45944.9us
I20260709 12:18:47.088231 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 631) took 1605 ms. Trace:
W20260709 12:18:47.122031 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 108982us
I20260709 12:18:47.028585   333 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:18:47.092994 28292 rpcz_store.cc:276] 0709 12:18:45.483255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.473940 (+990685us) service_pool.cc:224] Handling call
0709 12:18:46.498017 (+ 24077us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.519654 (+ 21637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.519687 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:46.547002 (+ 27315us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.831861 (+284859us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:46.831897 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.831965 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:47.082143 (+250178us) raft_consensus.cc:1685] finished
0709 12:18:47.082165 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:47.088133 (+  5968us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":129133,"threads_started":1,"wal-append.queue_time_us":358756}
I20260709 12:18:47.121907 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:47.125694 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.134600 28185 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:47.118443 32757 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:47.137658 32757 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.140404 32757 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:47.141424 32733 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:47.142062 32733 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:47.142534 32733 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:47.145952 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 68792.4us
I20260709 12:18:47.082633   319 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:47.151818 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:47.165689 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.138199 32675 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:47.157042 32554 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.571s	user 0.001s	sys 0.019s
W20260709 12:18:46.988157 28190 rpcz_store.cc:269] 0709 12:18:21.541834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.542168 (+   334us) service_pool.cc:224] Handling call
0709 12:18:46.987276 (+25445108us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:38.669930 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:38.928560 (+258630us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:38.933849 (+  5289us) write_op.cc:435] Acquired schema lock
0709 12:18:38.933873 (+    24us) tablet.cc:662] Decoding operations
0709 12:18:39.198352 (+264479us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:39.198470 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:39.198504 (+    34us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:18:39.200055 (+  1551us) tablet.cc:701] Row locks acquired
0709 12:18:39.200109 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:18:39.200322 (+   213us) write_op.cc:270] Start()
0709 12:18:39.200507 (+   185us) write_op.cc:276] Timestamp: P: 1783599519200242 usec, L: 0
0709 12:18:39.200543 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:18:39.201476 (+   933us) log.cc:844] Serialized 7450 byte log entry
0709 12:18:40.221072 (+1019596us) op_driver.cc:464] REPLICATION: finished
0709 12:18:40.544299 (+323227us) write_op.cc:301] APPLY: starting
0709 12:18:40.586990 (+ 42691us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:40.807448 (+220458us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88432a 00007f85be809f14 00007f85be809ec3 00007f85be809e38 00007f85be809df3 00007f85be8090dc 00007f85be832e16 00007f85be81acbb 00007f85be80539d 00007f85be808d75
0709 12:18:41.712706 (+905258us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b18001fa020. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be5c622a 00007f85be612c0a 00007f85be6124f7 00007f85be612490 00007f85be5bd754 00007f85c22ddf13 00007f85c22de532 00007f85c2325b93 00007f85c21d2de3
0709 12:18:42.024994 (+312288us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:42.053727 (+ 28733us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:44.663808 (+2610081us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:44.776639 (+112831us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 12:18:44.781166 (+  4527us) write_op.cc:312] APPLY: finished
0709 12:18:44.805176 (+ 24010us) log.cc:844] Serialized 172 byte log entry
0709 12:18:44.962570 (+157394us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:44.994210 (+ 31640us) write_op.cc:454] Released schema lock
0709 12:18:45.023736 (+ 29526us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:45.060649 (+ 36913us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:45.060682 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22370,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":251887,"apply.run_cpu_time_us":70406,"apply.run_wall_time_us":4530004,"cfile_cache_hit":74,"cfile_cache_hit_bytes":77293,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6216,"cfile_init":2,"lbm_read_time_us":36856,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":9,"num_ops":19,"prepare.queue_time_us":165690,"prepare.run_cpu_time_us":38720,"prepare.run_wall_time_us":844648,"raft.queue_time_us":410566,"raft.run_cpu_time_us":83798,"raft.run_wall_time_us":687840,"replication_time_us":994762,"spinlock_wait_cycles":17226624,"thread_start_us":720904,"threads_started":5,"wal-append.queue_time_us":401123,"wal-append.run_cpu_time_us":19819,"wal-append.run_wall_time_us":2391863}]]}
I20260709 12:18:47.172739   362 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:47.177026   409 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.163s	user 0.000s	sys 0.004s
I20260709 12:18:47.186270 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2223) took 1967 ms. Trace:
I20260709 12:18:47.181656 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:47.195891 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.195601 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 633) took 1158 ms. Trace:
I20260709 12:18:47.189159 28289 rpcz_store.cc:276] 0709 12:18:45.218457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.340440 (+1121983us) service_pool.cc:224] Handling call
0709 12:18:46.419201 (+ 78761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.483248 (+ 64047us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:46.483279 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:46.494777 (+ 11498us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.916334 (+421557us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:46.916378 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.916841 (+   463us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:47.186016 (+269175us) raft_consensus.cc:1685] finished
0709 12:18:47.186038 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:47.186194 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1128,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4432,"thread_start_us":377870,"threads_started":1,"wal-append.queue_time_us":655754}
I20260709 12:18:47.197659 28285 rpcz_store.cc:276] 0709 12:18:46.037239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.682976 (+645737us) service_pool.cc:224] Handling call
0709 12:18:46.683392 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.683932 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.683964 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:46.684541 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.913959 (+229418us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:46.913998 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.914061 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:47.195354 (+281293us) raft_consensus.cc:1685] finished
0709 12:18:47.195380 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:47.195531 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2262,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":19266,"thread_start_us":229010,"threads_started":1,"wal-append.queue_time_us":349172}
I20260709 12:18:47.186614   362 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:47.199126   324 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 12:18:47.165259   319 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:47.215329 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 56707us
I20260709 12:18:47.215989 28284 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:47.224406 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:47.136760 28185 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:47.192767 32554 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:47.233426 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:47.229715   307 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.000s
I20260709 12:18:47.241148 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:47.239471 32757 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:47.246930 32757 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.247362 32757 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:47.247663   358 consensus_meta.cc:287] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.002s
I20260709 12:18:47.248175   358 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.238406 32578 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:18:47.256209 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.266413 28185 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:47.268906 32767 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:47.279461   325 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.281118 28985 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:47.285945 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.269574 32767 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:47.309489 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:47.310353 28432 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
W20260709 12:18:47.319329 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 103731us
I20260709 12:18:47.301084   368 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:47.330564 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 102929us
I20260709 12:18:47.341209 28985 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:47.342825 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2442) took 1652 ms. Trace:
I20260709 12:18:47.343300 28440 rpcz_store.cc:276] 0709 12:18:45.654111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.375039 (+720928us) service_pool.cc:224] Handling call
0709 12:18:46.446929 (+ 71890us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:46.503719 (+ 56790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.503756 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:46.504566 (+   810us) log.cc:844] Serialized 52 byte log entry
0709 12:18:46.808017 (+303451us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:46.878752 (+ 70735us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.879021 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.879110 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:47.176124 (+297014us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:47.303713 (+127589us) raft_consensus.cc:1685] finished
0709 12:18:47.305982 (+  2269us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:47.306213 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20111,"prepare.queue_time_us":70498,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":9367,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":251,"spinlock_wait_cycles":5977344,"thread_start_us":302953,"threads_started":1,"wal-append.queue_time_us":440037}
I20260709 12:18:47.341300   358 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:47.345974 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:47.347066 28441 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:47.349560 32728 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:47.349681 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:47.347185 28436 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:18:47.354460 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:47.355331 28432 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:47.325256   428 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:18:47.358585   428 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:47.359004   428 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:47.360013   428 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.361405   428 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:47.362428   420 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 12:18:47.363219 32686 consensus_meta.cc:287] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.573s	user 0.007s	sys 0.008s
I20260709 12:18:47.364060 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2231) took 1957 ms. Trace:
I20260709 12:18:47.364502 28288 rpcz_store.cc:276] 0709 12:18:45.406224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.454416 (+1048192us) service_pool.cc:224] Handling call
0709 12:18:46.491782 (+ 37366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:46.525191 (+ 33409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:46.525225 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:46.597468 (+ 72243us) log.cc:844] Serialized 26 byte log entry
0709 12:18:46.937325 (+339857us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:46.937374 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.937475 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:47.363722 (+426247us) raft_consensus.cc:1685] finished
0709 12:18:47.363755 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:47.363962 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":229992,"threads_started":1,"wal-append.queue_time_us":507925}
I20260709 12:18:47.366739 32686 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:47.366741 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:47.370808 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.363979 32555 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:47.346379   426 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:18:47.375010 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.308686 32733 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.389686 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:47.372176   363 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.391631 28432 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:47.393682 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:47.269130 28185 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:47.412189 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:47.415369 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:47.416182 32729 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.416901 32757 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:47.418210   412 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:18:47.418581 28185 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:47.420680 32686 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:47.422643 32245 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.411334   358 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:47.424767   358 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:47.425160   358 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.425800 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:47.411134 32555 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.377872 32510 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:18:47.428059   368 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:47.377566   362 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.422986 32733 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:47.434417   368 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.436434 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:47.437693   324 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:47.438247   324 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:47.442252 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:47.434096   363 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:47.448185 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.416198 32554 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.436348   368 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:47.445500   362 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.424930 32245 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:47.451124 28366 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:47.451231 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.449913   363 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:47.421406 32686 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:47.457319 28437 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:47.472692 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:47.439249   324 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.480284 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.456980 32578 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.487267 28361 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:47.492870 28358 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:47.493870 32578 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.496075 32578 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.475505   362 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.496933   362 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.490053 28440 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:47.498534 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:47.488181 32510 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:47.498952 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:47.499706 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:47.500406 28440 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:18:47.500842   319 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.501251   307 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:47.501669   307 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:47.502092   307 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:47.503319 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.506999 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:47.499622 32510 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.497428 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:47.517066 28434 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.496356   325 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:47.519922 28360 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:47.522223 28366 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:47.472257   332 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:18:47.421411 28185 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 9 to 10, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 10 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:47.533838 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.533809 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.537654 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.538533 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.498976   362 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.507723 32555 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.543998 28367 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:47.550177 28358 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:47.559726 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.558305 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:47.562945 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:47.562975 28361 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:18:47.564111 28363 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:18:47.531824   425 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:47.568128 28359 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 12:18:47.568413 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:47.581475 32686 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.582226 32686 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.582659 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:47.584467 32686 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:47.584625 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:47.585028 32686 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:47.585692 32686 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:47.585937 32245 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:47.586355 32686 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.588534 32686 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:47.591185 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:47.599100 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 117261us
I20260709 12:18:47.601150 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:47.601822 28285 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:47.606659 28989 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:18:47.607568 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:47.608494 28984 raft_consensus.cc:2410] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 12:18:47.605516 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:47.609764 32733 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:47.610553 32733 consensus_queue.cc:1243] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:47.617213 28986 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:18:47.616483 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:47.619387 32733 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:47.621021 32733 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:47.560010 32578 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.621940 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:47.622603 28361 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:18:47.625941 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 56740.4us
I20260709 12:18:47.640019 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.641213 28285 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:47.627682 28360 pending_rounds.cc:85] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 9
I20260709 12:18:47.643852 28360 pending_rounds.cc:107] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:18:47.644133 28360 raft_consensus.cc:2889] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:47.644378 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:47.646346 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.652513 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:18:47.654045   433 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:18:47.653887 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:47.622428 32578 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.663064 28365 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:47.664296 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.668154 28364 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:47.626590   362 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:47.660056   332 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:47.676676 28440 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:47.680655 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.646983 32659 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.683954 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:47.684679 28990 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:47.686563   332 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:47.687678   332 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.691406 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:47.692247 28285 raft_consensus.cc:2393] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:47.693254 32767 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.694588 32767 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:47.696049   437 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:18:47.697782 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:47.699121 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:47.700178 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:47.700683 28435 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.702199 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:47.702329 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:47.703001 28288 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:47.703250 28436 raft_consensus.cc:2410] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:47.703907 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:47.704397 28436 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:47.704511 28440 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:47.686837   440 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:18:47.713639 28983 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
W20260709 12:18:47.714823 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 46510.3us
I20260709 12:18:47.715597 28990 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
W20260709 12:18:47.716739 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.701463 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.720062 28283 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:47.668536 32555 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:47.723636 28362 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:47.669636 32757 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:47.728291 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:47.725497 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:47.731078   428 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.744822   428 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.746487   362 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.724674 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:47.749569 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:47.744397 28434 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
W20260709 12:18:47.775321 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 52318.9us
I20260709 12:18:47.776101 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:47.769556 28286 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:18:47.649554   425 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.780064 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.796399 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:47.737967 32555 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:47.810328 28286 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:47.814790 28288 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.815147 28434 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:47.818539 28366 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:47.824823 28985 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:47.826889 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:47.726001   358 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:47.809476   419 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:18:47.829779 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.702492 28185 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:18:47.835989   430 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.442s	user 0.004s	sys 0.001s
I20260709 12:18:47.818878 32757 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:47.838227   429 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:47.838740 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:47.838971 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.841285 28359 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:47.738897 32659 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.844319 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:47.843555 28441 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:47.814467 32555 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:47.813838 32733 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:47.866930 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:47.870139 28365 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:47.880188 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.866191 28358 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 8.
I20260709 12:18:47.891958 28362 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:47.891371 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:47.881572 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:47.884562 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 151347us
I20260709 12:18:47.871620 28363 raft_consensus.cc:2393] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:47.894507 28441 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
W20260709 12:18:47.905294 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 55765.7us
I20260709 12:18:47.894893 28286 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:47.890113 28366 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
W20260709 12:18:47.899680 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.916298 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.913206 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.919450 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:47.863955   333 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.877038 32659 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:47.880883 32757 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:47.926020 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.936936 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.938261 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:47.904784 32510 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.943223 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:47.947321 28440 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:47.957085 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.927013 32539 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:47.962071 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:47.961148 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:47.965039 28283 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:47.961611 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.959683 28440 pending_rounds.cc:85] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
I20260709 12:18:47.967597 28440 pending_rounds.cc:107] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:18:47.968663 28440 raft_consensus.cc:2889] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:47.946197 32686 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:47.970839 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.971750 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.970918 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:47.956408 32659 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:47.877090   307 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:47.983132 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.983108 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.984531 28365 raft_consensus.cc:2368] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:47.984714 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:47.986994 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.989128 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.989889 28358 raft_consensus.cc:2368] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:47.990410 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:47.990733 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:47.989845 28363 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.991509 28358 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:47.992653 28358 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:18:47.994086 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49259.1us
I20260709 12:18:47.983707 32578 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:47.995525   358 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:47.969178 32539 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:47.937513   333 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.000928 28358 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:47.988906   431 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:48.019493   463 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:48.025420 28435 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:48.035396 28362 raft_consensus.cc:1217] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 6.5->[8.6-12.8]   Dedup: 8.6->[10.7-12.8]
I20260709 12:18:48.035983 28362 pending_rounds.cc:85] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 6
I20260709 12:18:48.036193 28362 pending_rounds.cc:107] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:18:48.036420 28362 raft_consensus.cc:2889] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:48.036576 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:48.037649 28990 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:47.997587 32539 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:48.046455 28330 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.771s (SENT)
I20260709 12:18:48.059898 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:48.011878   431 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:48.073333 28365 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:48.013451   436 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.014233 32659 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:48.075171 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:48.094904 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:48.105130 28404 leader_election.cc:400] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:48.105516 28404 leader_election.cc:403] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:18:48.106360 28404 leader_election.cc:400] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:48.106660 28404 leader_election.cc:403] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:48.107578 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:48.107355 32555 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.108723 28292 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:48.113520 28363 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:48.109244 32245 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:48.090170   358 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.118273   358 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.120390   358 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:48.122658   307 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:48.013712   319 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Committing config change with OpId 9.7: config changed from index 2 to 7, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:18:48.127972 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 92756.5us
W20260709 12:18:48.128787 28330 leader_election.cc:336] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.786s (SENT)
W20260709 12:18:48.129783 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.130321 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.135748 28364 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:18:48.137053 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 100572us
I20260709 12:18:48.090961   322 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:48.091427   425 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:48.155467 32659 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.158119   476 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.158500   456 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:48.159385   456 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:48.160481   471 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:47.977383 28185 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:48.073766 32757 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.163864 28185 catalog_manager.cc:5697] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 7 to 11, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 11 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:48.097872   470 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:48.178946 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.182142   431 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.171764   470 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.199430 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.211403 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:48.148203   436 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:48.221058 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 89960.5us
W20260709 12:18:48.219900 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 110964us
I20260709 12:18:48.202499   470 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.225273 28288 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:48.225778 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:48.231422 28288 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:48.233873 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:48.234894 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:48.210011   469 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:48.217051   358 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:48.240890 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:48.244065 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:48.246760 28438 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:48.250068   333 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:48.252713 28440 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:48.252975   469 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.255136   476 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:48.255609   469 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:48.256095   476 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:48.256333   469 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:48.256992   469 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:48.258064   469 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.258471 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:48.259497 28435 raft_consensus.cc:2410] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:18:48.267800 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.282464 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:48.229439   463 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:48.310289 28432 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:48.312705 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:48.261972   436 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:48.263764   431 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:18:48.321436 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51632.9us
I20260709 12:18:48.323009 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:48.324034   363 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:48.325977 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:48.325920   363 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:48.326963   363 consensus_queue.cc:1243] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:48.327648   425 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:48.328039   425 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:48.329053   425 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.312336 28289 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 2 to 7, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:48.332095 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:48.334456 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:48.335767   445 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:48.336608   445 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:48.336685   463 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:48.337137   445 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:48.338181   445 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.340762 28288 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:48.338650 28291 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:48.216976 28185 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 3 to 7, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 7 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:48.344449   307 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:48.347347   307 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:18:48.347775   461 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.378s	user 0.000s	sys 0.001s
I20260709 12:18:48.338567 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:18:48.326858   480 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.103s	user 0.000s	sys 0.003s
I20260709 12:18:48.364641 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:48.368723 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:48.379741 28432 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 13. Ops: []
W20260709 12:18:48.366966   368 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.893s	user 0.014s	sys 0.000s
I20260709 12:18:48.387172 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:48.365861   385 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:48.387811   385 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.388211   385 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:48.385625   479 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.003s
I20260709 12:18:48.392141 32675 raft_consensus.cc:2764] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:18:48.395443   477 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:18:48.378286   362 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:48.386998   368 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.386164   458 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:48.416674 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.436565 28367 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:48.438162 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 69382.9us
W20260709 12:18:48.436568 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 210949us
W20260709 12:18:48.445839 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:48.450156 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.455312 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.456180 28283 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:48.456923 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:48.457230 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.458484 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.460240 32675 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.460878 32675 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.463697 32675 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:48.464444   482 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:18:48.464857   425 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:48.470162 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:48.456463 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.471567 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.472800 28286 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:18:48.474726   484 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:18:48.475723 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:48.476507 28285 raft_consensus.cc:2368] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:48.478048 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.480093 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.480805 28285 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:48.482316 28290 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:48.484113 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:48.484923 28290 raft_consensus.cc:2393] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:48.486522 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:48.487102   455 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.609s	user 0.002s	sys 0.002s
I20260709 12:18:48.488299   470 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.489946   337 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:48.490391   337 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:48.490782   470 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:48.491482   337 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:48.487340 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 71528.8us
I20260709 12:18:48.494645 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:48.494186 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:48.488902 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:48.495438 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 55572.9us
I20260709 12:18:48.500003 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.500865 28283 raft_consensus.cc:2368] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
W20260709 12:18:48.501143 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 165034us
I20260709 12:18:48.502728 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:48.503999 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:48.494235 28332 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:48.504588 28286 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.507330 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.504678 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:48.516875 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:48.518541 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:48.521216 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 79799.2us
I20260709 12:18:48.524041   471 raft_consensus.cc:476] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Not starting pre-election -- already a leader
I20260709 12:18:48.531347 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:48.536612 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:48.537462 28364 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.509125   477 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:48.548079 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:48.520677   337 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:48.549777 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.520241 32675 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:48.553035 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:48.548285 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:48.541157   458 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:48.551401 28285 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:18:48.561512 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:48.544922   477 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.542477   471 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:48.570514 28366 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:48.562912 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:48.552608   363 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:48.585768 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 91610.3us
I20260709 12:18:48.552568   470 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:48.588574 28983 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:48.620121 28364 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:48.565335   362 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 12:18:48.576875   477 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.673417   324 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:48.673758 32539 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:48.680737 28286 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
W20260709 12:18:48.676858 32733 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.027s	sys 0.035s
I20260709 12:18:48.559589 32728 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:18:48.583678   428 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:48.686015 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:48.682279 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 134081us
W20260709 12:18:48.684449 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 144824us
I20260709 12:18:48.691906 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:48.684974 32733 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.028s	sys 0.035s
W20260709 12:18:48.696308 28954 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 2.214s (SENT) [suppressed 9 similar messages]
I20260709 12:18:48.724388   368 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:48.725682 28332 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:48.682029   491 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:48.585624   471 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.682431 32539 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:48.674154   324 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:48.687400   447 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:18:48.732079 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:48.674616   494 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.135s
I20260709 12:18:48.677380   470 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:48.747386 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 196551us
I20260709 12:18:48.714617 32757 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:48.728526 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 165661us
I20260709 12:18:48.740726   428 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:48.724152 32733 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:48.685118   463 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:48.730396 28289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:48.769181 28332 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:48.750259 32659 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:48.781270 28332 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:48.783860 28434 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:48.787483 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:48.777670 28289 raft_consensus.cc:606] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:48.760694 32757 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.768548   469 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:48.773018   368 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:48.770336   428 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:48.772630   502 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:48.835763   502 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.836166   502 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.837791 32717 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:18:48.817348 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:48.842691   498 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:48.843256   498 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.843695   498 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:48.843699   428 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.830077   324 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:48.846793   493 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 12:18:48.847996   495 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:18:48.850176   385 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.852633   385 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:48.754388   491 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.855258   463 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.857374   491 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:48.832304 28288 raft_consensus.cc:2368] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:48.835166   512 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:48.807570 32659 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.831427   469 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.831936 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:48.862195   498 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:48.937275 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.864308   368 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:48.842978 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:18:48.866746 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 56382.6us
I20260709 12:18:48.942353 28332 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:48.786266   458 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:18:48.869380 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 62134.4us
I20260709 12:18:48.790766 32675 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:48.791273 32757 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:48.778800 32767 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:18:48.777449   362 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:18:48.877810 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.843351 32245 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:48.956425 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 113553us
I20260709 12:18:48.792338   494 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:18:48.961757 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:48.958412   362 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:48.966295 28987 raft_consensus.cc:2393] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:48.966652   362 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.967773   362 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:48.962515 32510 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:48.963510   483 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:48.965090 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:48.980643 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.982069 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:48.939244   340 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:48.994417 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 302593us
I20260709 12:18:48.874446   307 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:18:48.995397 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:48.955860   491 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:48.964717   456 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:48.961844   469 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:48.972020 32510 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:49.033017 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:48.869657 32539 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:48.992965   504 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:49.029525 32728 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:48.957412 32733 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:48.978984 32245 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.033855   469 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:49.056803   497 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:18:49.043049   456 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:49.078747   456 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:49.056715   481 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.737s	user 0.001s	sys 0.001s
I20260709 12:18:49.081137 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2482) took 1573 ms. Trace:
I20260709 12:18:49.081632 28441 rpcz_store.cc:276] 0709 12:18:47.507694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.905850 (+398156us) service_pool.cc:224] Handling call
0709 12:18:47.906432 (+   582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:47.983531 (+ 77099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:47.983671 (+   140us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:48.002964 (+ 19293us) log.cc:844] Serialized 26 byte log entry
0709 12:18:48.178422 (+175458us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:48.178467 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.178546 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:49.055867 (+877321us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:49.080716 (+ 24849us) raft_consensus.cc:1685] finished
0709 12:18:49.080744 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.081034 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89381,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":60,"spinlock_wait_cycles":70788480,"thread_start_us":183441,"threads_started":2,"wal-append.queue_time_us":216125}
I20260709 12:18:49.016992   483 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.028033   491 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.024555   340 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.059851   507 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:49.039494   494 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:18:49.049054   471 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:49.059741   504 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:49.065478   505 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:18:49.115778   508 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:18:49.115547   505 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:49.114807 28987 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:18:49.114530   504 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.117982 28987 raft_consensus.cc:606] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:49.086704   515 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:49.130429 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 129973us
I20260709 12:18:49.085669   469 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.111085   324 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:18:49.144323 28367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2151) took 1583 ms (client timeout 1932 ms). Trace:
I20260709 12:18:49.142937 28332 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:49.144773 28367 rpcz_store.cc:269] 0709 12:18:47.558468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.565055 (+  6587us) service_pool.cc:224] Handling call
0709 12:18:49.142100 (+1577045us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b5400135954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:49.142262 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1829513984}
W20260709 12:18:49.167503 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 159321us
I20260709 12:18:49.165077 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:49.148451   442 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 1.478s	user 0.006s	sys 0.009s
I20260709 12:18:49.112223 32510 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.166793   470 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:49.146672   324 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.136229   515 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.182616   517 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:49.132673   505 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:49.212790 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 230797us
I20260709 12:18:49.216063 28441 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:49.173432   516 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:18:49.183277 32728 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:49.222851 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.203389   507 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:49.225962 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 54616.1us
I20260709 12:18:49.229565 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:49.231025 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:49.232014 28437 raft_consensus.cc:2410] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:18:49.234066   505 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:49.234570   425 consensus_meta.cc:287] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.769s	user 0.014s	sys 0.001s
I20260709 12:18:49.234884 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:49.235729 28435 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:49.237142   491 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:49.237995   324 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:49.226251 32510 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.239888   507 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.240495   340 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:49.228062 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:49.242321 28436 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:49.174791   522 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:18:49.228981 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:49.250304 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:49.251070 28437 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:49.252317   491 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:49.256007 28404 leader_election.cc:400] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:49.268182 28437 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:18:49.270746 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:49.217944   523 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:49.233029   470 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:49.212399   428 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.283841 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.287020   469 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:49.286626   520 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:49.289480   520 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.289973   520 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:49.290365 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.291292   471 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:49.291877 28404 leader_election.cc:400] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:49.291936   471 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:18:49.292970 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.294865 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.296330 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.297895 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.275110 32659 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:49.227967   502 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.299454 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:49.286834   519 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:18:49.327390   428 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:49.332283 32245 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:49.334407 32659 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:49.293785   471 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:49.311443   515 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:49.306114   456 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:49.316296   520 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.338649 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:49.299542   469 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:49.342648 28255 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:49.356619 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2226) took 1117 ms. Trace:
I20260709 12:18:49.399732 28432 rpcz_store.cc:276] 0709 12:18:48.238821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.314828 (+ 76007us) service_pool.cc:224] Handling call
0709 12:18:48.315190 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.380251 (+ 65061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.356314 (+976063us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b540026b6d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:49.356540 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147875072}
I20260709 12:18:49.360663   307 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:49.297116   523 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:49.366379   319 raft_consensus.cc:993] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:18:49.368763 28987 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:49.370424 28985 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:49.373183   309 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:49.382506 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:49.294133   483 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:49.397951 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2253) took 1618 ms. Trace:
I20260709 12:18:49.363646   324 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:49.402205 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:49.405879 32541 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.409916 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:49.405014 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 62465.9us
I20260709 12:18:49.417013 28441 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:49.393493 32245 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:49.416039 28287 rpcz_store.cc:276] 0709 12:18:47.779628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.791206 (+ 11578us) service_pool.cc:224] Handling call
0709 12:18:47.817934 (+ 26728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:47.818130 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:47.873581 (+ 55451us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.333531 (+1459950us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b1800392a40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be42723a 00007f85be891a2a
0709 12:18:49.397362 (+ 63831us) log.cc:844] Serialized 14 byte log entry
0709 12:18:49.397545 (+   183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:49.397594 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:49.397640 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.397718 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.397886 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43657,"prepare.queue_time_us":87269,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":10156,"raft.queue_time_us":1965,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":10347520,"thread_start_us":22413,"threads_started":1}
I20260709 12:18:49.442699 32541 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.447502 28438 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:49.398172   517 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:49.442848 28437 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 2 to 7, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:49.448639 28432 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:49.448632   337 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.402946   362 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:49.416549   423 raft_consensus.cc:1081] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:18:49.415334   309 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:49.404022 32728 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.513824   309 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.515928   309 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.455567   337 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.522722 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:49.522961 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:49.531050 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:49.537283 28284 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:49.538074 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:49.537284 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:49.539143 28287 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:49.512382   513 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:49.525386   515 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.507767   530 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 12:18:49.550033 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 44657.1us
I20260709 12:18:49.551666 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:49.552510 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:49.567019 28290 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:49.553354 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:49.567210 28283 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:49.585611 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:49.585665 28287 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:49.586405 28284 raft_consensus.cc:2376] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13. Re-sending same reply.
I20260709 12:18:49.589630 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:49.590269 28292 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.549265   423 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:49.566262   505 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.611361 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 654) took 1866 ms. Trace:
I20260709 12:18:49.617944 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:49.504495   523 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.620419 28985 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:49.611972 28990 rpcz_store.cc:276] 0709 12:18:47.734097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.734497 (+   400us) service_pool.cc:224] Handling call
0709 12:18:47.748656 (+ 14159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.038090 (+289434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:49.599746 (+1561656us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5400476b54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:49.599989 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664155392}
W20260709 12:18:49.627281 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 96298.4us
I20260709 12:18:49.627713 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:49.628259 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:49.592895   423 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.629575 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:49.630479 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:49.633406 28358 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:49.634805 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:49.635845 28290 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:18:49.550714   513 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.633558 28365 raft_consensus.cc:2410] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:18:49.631508 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:49.630630 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:49.644829 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.645253 28364 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:49.638338 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:49.570194   515 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.635155 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:49.635512 28367 raft_consensus.cc:2376] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14. Re-sending same reply.
I20260709 12:18:49.583479   507 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:18:49.626221   550 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.642409 28360 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:18:49.649683 28361 raft_consensus.cc:2376] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10. Re-sending same reply.
I20260709 12:18:49.649927 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:49.666113 28332 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:49.666507 28435 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:49.649019 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:49.674003 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:49.677372 28366 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:49.638873   423 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:49.687848   537 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:18:49.688135 32541 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:49.682969 32675 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:18:49.689756   540 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:18:49.641197   542 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:18:49.691004   542 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:49.691444   542 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:49.692396   542 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:49.693816 32767 consensus_queue.cc:1175] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 11 status { last_received { term: 10 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:49.620951   505 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:49.706959 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 78786.3us
W20260709 12:18:49.693836   460 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 1.701s	user 0.003s	sys 0.003s
I20260709 12:18:49.716749   337 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:49.721179   368 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:49.674209   520 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:49.650223   333 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.725930 28292 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:49.727434 28435 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:49.591272   517 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.677047   516 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:49.677564   522 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:49.731264 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:49.689613 32675 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:49.629356   544 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:18:49.709291   542 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:49.701021 32728 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.673811   428 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
W20260709 12:18:49.728994   477 consensus_meta.cc:287] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.059s	user 0.010s	sys 0.006s
I20260709 12:18:49.741236 28332 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.744834 28435 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:49.750173 28289 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:49.730845   556 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.733155   368 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:49.762236 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:49.744962   516 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:49.743206   337 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:49.685029   550 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:49.766566 32510 consensus_meta.cc:287] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.002s
I20260709 12:18:49.742147   333 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.770746   516 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:49.785120   368 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.786827   368 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:49.767982   556 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.770174   337 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:49.791438 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.778895   425 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:49.766250 32728 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.809036 32728 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:49.799095   565 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:49.778270 32717 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:49.776883   561 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:49.805909   550 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:49.766076   477 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.796665   337 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:49.811208 32717 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:49.811981   565 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.835703 28359 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:49.834515 32728 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:49.839341   333 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:49.840167   333 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:49.840553   333 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.840888   333 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.846280 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:49.847208 28438 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:49.850239   506 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:49.850839   506 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:49.851862   506 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.853120   506 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:49.854442   487 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.444s	user 0.010s	sys 0.000s
I20260709 12:18:49.854581 32578 raft_consensus.cc:3055] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:49.855044 32578 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:49.855970 32578 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.856403 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2170) took 2019 ms. Trace:
I20260709 12:18:49.856768   521 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:18:49.858325 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:49.856897 28362 rpcz_store.cc:276] 0709 12:18:47.843094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.000285 (+157191us) service_pool.cc:224] Handling call
0709 12:18:48.012615 (+ 12330us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:48.046698 (+ 34083us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:48.196856 (+150158us) log.cc:844] Serialized 14 byte log entry
0709 12:18:48.350041 (+153185us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:48.356485 (+  6444us) log.cc:844] Serialized 52 byte log entry
0709 12:18:48.356681 (+   196us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:48.356721 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.356799 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:48.859796 (+502997us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b180047a0a0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:49.826222 (+966426us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b24003d6e68. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6209c4 00007f85bb61eacb 00007f85bb7d9dc9 00007f85bb7d9a55 00007f85bb7d909d 00007f85be40b10e 00007f85be42288c 00007f85be421bb6 00007f85be4217d8 00007f85be42b221
0709 12:18:49.856154 (+ 29932us) raft_consensus.cc:1685] finished
0709 12:18:49.856182 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:49.856331 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71,"prepare.queue_time_us":59671,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":3797,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":16556,"spinlock_wait_cycles":74889728,"thread_start_us":232721,"threads_started":2,"wal-append.queue_time_us":213341}
I20260709 12:18:49.860481   544 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:49.860903   544 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:49.861261   544 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:49.863890   505 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:49.792466   554 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:18:49.867292   556 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:49.813184   513 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:49.868384 32728 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.871672 32728 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:49.873030   544 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.857297 32578 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:49.874663 32659 raft_consensus.cc:2764] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 12:18:49.868978   498 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:49.877600 28438 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:49.813690   425 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:49.880815   563 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:18:49.875586   565 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:49.797515 32675 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:49.857406   546 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 12:18:49.836524 32757 consensus_meta.cc:287] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.857s	user 0.007s	sys 0.007s
I20260709 12:18:49.893379 28358 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:49.857053   547 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:49.898155 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 106825us
I20260709 12:18:49.899159 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.857241   521 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:49.902083 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.905169 28440 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:18:49.917420 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:49.926147 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:49.925701   561 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:18:49.930053   560 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:49.909448   483 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:49.906217 32245 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:49.931818   483 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.914053   550 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:18:49.931898 32245 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.935968 32245 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:49.939599 32729 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:49.897516 32757 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:49.947993 32757 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:49.949023   515 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:49.940135 32729 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:49.950730 32729 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.951996 32729 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:49.954041 28437 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:49.937949   425 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:49.957212 28432 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:49.960000 28359 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2173) took 1429 ms (client timeout 1513 ms). Trace:
W20260709 12:18:49.960497 28359 rpcz_store.cc:269] 0709 12:18:48.530788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.531001 (+   213us) service_pool.cc:224] Handling call
0709 12:18:49.959695 (+1428694us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400079254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:49.959854 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2997464448}
I20260709 12:18:49.942417 32555 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:49.944796   545 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.342s	user 0.002s	sys 0.000s
I20260709 12:18:49.901594   547 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:49.962165 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.967075   560 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:49.969784 32728 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:49.970590   513 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:49.971068   505 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:49.972110   538 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:49.972482 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.972939   516 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.044287 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:49.970502   445 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:50.067459 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.078526 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.040998   554 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.081679 28441 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:50.084632 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.079536   516 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.113927 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.114780 28362 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:50.115859 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.116590 28363 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:18:50.061610   504 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:50.079577   554 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.123071 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.123826 28366 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 8.
I20260709 12:18:50.106719 32555 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:18:50.124998 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.125226   554 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:50.041812 32728 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.126091 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.126451   547 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.080281   445 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.127287 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.128031   544 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.127096 28358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:56626
I20260709 12:18:50.129134 28434 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:50.129150   445 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:49.918712 32717 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.140442 28358 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:50.140918 28358 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.076752   483 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:50.041940   513 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.143958   513 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.041584   505 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.146306   505 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:50.148653 32717 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:50.039805   538 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.150490 32728 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:50.150842   538 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:50.151327   522 consensus_queue.cc:1175] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 14 status { last_received { term: 14 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 14 index: 12 } }
W20260709 12:18:50.153859   309 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.001s
I20260709 12:18:50.162578   521 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.163992   521 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:18:50.172499 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46922.5us
I20260709 12:18:50.176083 28990 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:50.177937 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.203665 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.179358 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.125991 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:50.194365 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.195233 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.196126   502 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:50.198411 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.199234 28435 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:50.210880 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:50.211928 28363 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:18:50.200740 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 862146us
I20260709 12:18:50.181519   471 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.206660 28441 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:50.214118 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:50.220479 28404 leader_election.cc:400] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:50.220795 28404 leader_election.cc:403] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:18:50.230770 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.229362 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.214851 28358 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.233036 28366 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:50.229338 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.170797   423 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.247325 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:50.237967 28358 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:50.199304   561 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:18:50.179957   504 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.229040   505 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:50.223024   368 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.230554   491 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:50.221587   542 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.257115 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.265280 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.266009 28365 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:50.260710 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.255092   530 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:50.261833 28367 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:50.262481 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.263252 28360 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:50.229606   570 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Leader election lost for term 12. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:50.270642 28363 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:50.269223 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:50.271834 28365 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.239087   507 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:50.277544 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.251684   560 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:50.279912 28363 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:50.254004   561 consensus_queue.cc:1243] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:50.282852 28288 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:50.253536 32728 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:50.253494 32578 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:18:50.289759 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.230249   471 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.302353 28365 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:50.279798   491 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:50.243112   333 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.273118   502 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:50.255354   445 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:18:50.274366   542 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:50.356941 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 175925us
I20260709 12:18:50.318409   491 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.288496   507 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.284526   505 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:50.366896 28367 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:50.323882 28360 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 12:18:50.323664   565 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.326040 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.328014   423 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.329833 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.378594 28284 raft_consensus.cc:2410] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:18:50.331101 28287 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:18:50.379226 28287 raft_consensus.cc:606] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:50.379591 28287 raft_consensus.cc:614] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:50.338522   504 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.339483 28363 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:50.357112   502 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:50.345225   337 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 12:18:50.281114   530 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.346405 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.351287 28362 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:50.397393 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.352326 28358 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:50.398294 28362 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:50.354331 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.351408 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.282845   531 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:50.317221 32578 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:50.292423   368 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.380585 28991 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:50.343631   471 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:50.395342 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.410849 28360 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:50.397279 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.403697 28989 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:50.412285 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.413369 28365 raft_consensus.cc:2410] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:18:50.414594 28365 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:50.418967 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.423828 28362 raft_consensus.cc:2410] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:18:50.401466   337 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.425859 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:18:50.365211   483 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:18:50.382122   505 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:50.357954 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 65989us
I20260709 12:18:50.425459 28363 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:50.506371 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.450961 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:50.445948 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:50.544090 28365 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:50.426478 28982 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:18:50.543943 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:50.444914 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.480105 28364 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:50.543943 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:50.631898 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 265685us
I20260709 12:18:50.633987 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:50.634617 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:50.635915 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 275435us
I20260709 12:18:50.621862 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:50.426640   502 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.640865   502 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:50.641484 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.643359 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.644066 28292 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:50.620572 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:50.646641 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.648710 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:50.646083 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.472958   565 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:50.625288   538 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:50.651572 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.622543 28358 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.402841 32578 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:50.479554 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:50.658617 28440 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.643440   470 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:50.623932 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 173053us
I20260709 12:18:50.666775 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:50.675024 28290 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:50.676122 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.667398 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.669234 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.678134 28283 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:50.675043 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:50.680305 28988 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:50.681840 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:50.685916 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.671265 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.689873 28291 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:18:50.690526 28291 pending_rounds.cc:85] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 11
I20260709 12:18:50.690834 28291 pending_rounds.cc:107] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:18:50.691148 28291 raft_consensus.cc:2889] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:50.467821   506 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.244s
I20260709 12:18:50.712034 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:50.712349 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.712914 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.713714 28287 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:50.455308   333 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:50.712268 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.714694 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.479701   507 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:50.714948 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.626050 32717 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:50.675246 28404 connection.cc:256] RPC call timeout handler was delayed by -1.3993955319999714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
I20260709 12:18:50.723717 28361 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:50.716712 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:50.461318   570 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:50.481212   530 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:50.476217   368 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:50.726941 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 62755.9us
I20260709 12:18:50.718041 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:50.719839 28983 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:18:50.720758 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 45820.2us
I20260709 12:18:50.715934 28359 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.715418   470 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.734190 28984 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:50.726590 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.714092   505 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.722595 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.738406 28432 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:50.471427   542 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.730765 28290 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:50.724427 28292 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:50.731386 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:50.732555 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.723214 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:50.739547 28987 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:18:50.738641 28440 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:50.745263 28358 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:50.421586   531 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.746594 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:50.422416   560 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.712075 32578 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.729667 32555 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:50.751259 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.752403 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:50.754626 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.752478 28363 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:50.756152 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.737800   470 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.758872 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.759559 28358 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:50.762655 28359 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:50.763720 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.764084 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:50.765900   530 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:50.766749   530 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:50.767051   530 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:50.767184 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.767930   530 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.768311 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.769089   530 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.769335 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.769506   445 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.769979 32728 consensus_queue.cc:1059] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:50.770818 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:50.771059 32728 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:50.769949 28362 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:18:50.772076 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:50.772873 28362 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:50.774013 28362 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:50.775276 28362 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:50.775728 28359 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:50.776237 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:50.778949 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:50.729920 32717 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.800872 28359 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:50.754122   505 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:50.805357 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:50.731946   570 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.829785 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:50.807016 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:50.808297 28991 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:50.808434 28362 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:18:50.769085 32578 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:50.814605   337 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.819010   531 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:50.820757   566 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.642s	user 0.010s	sys 0.000s
I20260709 12:18:50.821628   445 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:50.826153 32717 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:50.827437 32555 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:50.780089   471 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:50.805308   505 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:50.839900 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:50.842031 28363 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:50.847105 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:50.756997   538 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:18:50.849989   477 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:50.841063   530 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:50.848160   505 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.771442 32728 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:50.893823 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46798.3us
I20260709 12:18:50.867530 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:50.918323 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 50912.3us
I20260709 12:18:50.867229 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.919135 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:50.900312 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:50.919263 28435 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:50.890456 28358 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:50.902539 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:50.921104 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:50.921655 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:50.926684 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.928592 28435 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:50.931332 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:50.932808 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:50.900046   512 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:50.854645 32555 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:50.930148   324 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:50.931808   333 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:50.949877   502 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.950542   502 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:50.938472   423 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:50.939851 28435 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:50.927899   337 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:50.956044 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.854916   531 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:50.930955   538 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:50.952726 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 313655us
I20260709 12:18:50.959297   531 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.944773   319 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:50.941452   570 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:50.963002 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:50.947090   477 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:50.949995   333 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:50.964004 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 240890us
I20260709 12:18:50.964021 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.953274   324 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.952836   530 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.975054 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:50.983569 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:50.982753 28435 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:18:50.956436   337 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.987020 28438 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:50.986847 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:50.963508   512 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.962224   531 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.958781 32728 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.021284 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:50.979466   333 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:51.017949 28439 raft_consensus.cc:1217] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Deduplicated request from leader. Original: 8.8->[10.9-14.11]   Dedup: 10.9->[13.10-14.11]
I20260709 12:18:50.983631   319 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:50.982757   324 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:51.032790 28439 pending_rounds.cc:85] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 9
I20260709 12:18:51.041999 28439 pending_rounds.cc:107] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:18:51.042294 28439 raft_consensus.cc:2889] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:51.017423   570 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:50.961067   423 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:51.050875 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.107657   570 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:51.051906   385 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:51.052815   445 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:51.108485   445 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:51.108104   423 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:51.021543   522 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:18:51.108676   570 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.109236   385 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:51.054512   591 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.045655 32767 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:18:51.045367   590 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:51.058284   319 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:51.019345 32717 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:50.979665   477 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.984303   502 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:50.974747   538 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:50.992172   530 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:51.072001 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.046456 32555 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:51.113513   588 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.062s	user 0.002s	sys 0.004s
I20260709 12:18:51.041357 32728 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:51.083678   396 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.088717   471 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:51.090747 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 517) took 1050 ms. Trace:
I20260709 12:18:51.145984 28990 rpcz_store.cc:276] 0709 12:18:50.040530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.040690 (+   160us) service_pool.cc:224] Handling call
0709 12:18:51.090526 (+1049836us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540045d8d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:51.090644 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251187072}
I20260709 12:18:51.095077   483 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:51.103791   470 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.043076   337 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:51.023459   458 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
W20260709 12:18:51.114513 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 181782us
I20260709 12:18:51.019309   512 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.127873 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:51.118712   593 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.066s	user 0.003s	sys 0.003s
I20260709 12:18:51.111681   431 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.115053 32717 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:51.146581 28289 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2276) took 1595 ms (client timeout 1858 ms). Trace:
W20260709 12:18:51.175901 28289 rpcz_store.cc:269] 0709 12:18:49.551222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.551341 (+   119us) service_pool.cc:224] Handling call
0709 12:18:51.146339 (+1594998us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400163354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:51.146490 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421679488}
W20260709 12:18:51.175427 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 131431us
I20260709 12:18:51.163877 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:51.179569 28358 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:18:51.164992 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:51.180560 28364 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:51.167310 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:51.171985 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:51.172122 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:51.172255 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:51.181119 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:51.186846 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:51.114215   591 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.187538 28440 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:51.187996 28987 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:18:51.187562 28984 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
W20260709 12:18:51.173344   571 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.007s
I20260709 12:18:51.205924 28358 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:51.178998 32717 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.210942 28432 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:51.187726 32578 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:51.152882   396 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.209812   581 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:51.178745   431 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.163797   445 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.201828   513 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 12:18:51.240063 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 59215.5us
I20260709 12:18:51.207139   506 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:18:51.250083 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:51.209029   574 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:18:51.254313 28361 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:51.263072 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:51.246938   603 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:51.220537 32578 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.279525 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2239) took 1372 ms. Trace:
I20260709 12:18:51.240170   581 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:51.262440   445 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:51.224825   505 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:51.285223 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 80764.2us
I20260709 12:18:51.285049 28437 rpcz_store.cc:276] 0709 12:18:49.907501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.941692 (+ 34191us) service_pool.cc:224] Handling call
0709 12:18:49.942028 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:49.954503 (+ 12475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.279236 (+1324733us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b5400348854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:51.279447 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27142912}
I20260709 12:18:51.260197 32733 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:51.304486   538 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.305204   538 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:51.306056   506 consensus_queue.cc:1243] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer f526369c0d0743b192be6e205540a059 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:18:51.309209   538 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.313551   538 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.317797 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 518) took 1272 ms. Trace:
I20260709 12:18:51.261258   522 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:18:51.319448 28985 rpcz_store.cc:276] 0709 12:18:50.044974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.045186 (+   212us) service_pool.cc:224] Handling call
0709 12:18:51.317494 (+1272308us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b54004c36d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:51.317653 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2179730688}
I20260709 12:18:51.334378 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:51.285717   603 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.296355   477 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.252786   596 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:18:51.298379 32733 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:51.332355   521 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.349650 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2232) took 1447 ms. Trace:
I20260709 12:18:51.340808   605 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.287160   611 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:18:51.319067   538 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:18:51.343353   530 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.343101   565 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:51.376116 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:51.344777   603 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.354583 32733 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:51.356526   521 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.297415   491 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:18:51.384066 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:51.397795 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:51.355688   613 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:51.347925   470 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:51.351758 28433 rpcz_store.cc:276] 0709 12:18:49.901992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:49.903771 (+  1779us) service_pool.cc:224] Handling call
0709 12:18:49.917707 (+ 13936us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:49.932331 (+ 14624us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:49.932648 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.385146 (+452498us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:50.394338 (+  9192us) log.cc:844] Serialized 52 byte log entry
0709 12:18:50.394505 (+   167us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:50.394546 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.394618 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:51.084051 (+689433us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b1800367960. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:18:51.336143 (+252092us) raft_consensus.cc:1685] finished
0709 12:18:51.349414 (+ 13271us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:51.349578 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15376,"prepare.queue_time_us":740,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":23285,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":13191,"spinlock_wait_cycles":3804032,"thread_start_us":452042,"threads_started":1,"wal-append.queue_time_us":464078}
I20260709 12:18:51.423173 28990 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:18:51.365247   512 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.357802   531 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.436959   591 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:51.468255   591 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.468863   591 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.424774   613 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:51.484153   620 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:18:51.490202   609 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:18:51.491624   337 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:51.460891   599 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:51.503744   608 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:18:51.433574   596 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.378549   605 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.399344   602 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:18:51.470216   396 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:51.565671 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.580269 28286 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:51.387351   581 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:51.545168   431 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:51.591747 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 352404us
I20260709 12:18:51.574101 32555 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:51.564914   617 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:51.625707 28286 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:18:51.629868 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 352837us
I20260709 12:18:51.593027 32578 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.631466 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:51.636251 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:51.610288   491 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.639932 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:51.609215   531 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.642110   470 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:51.650460   570 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:51.638458   623 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:51.642041   512 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:18:51.651865   512 consensus_queue.cc:1243] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:51.575484   599 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:51.576493   596 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.630771   505 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.651057   570 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.604972   431 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:18:51.629894   506 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:18:51.627511 32555 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.644424   628 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:51.674907   333 consensus_meta.cc:287] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.003s
I20260709 12:18:51.680574   628 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:18:51.681092   599 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.680616   333 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:51.683426   333 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.685766   627 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.631256   617 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.691491   596 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.691780   617 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:51.692412   617 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:51.692836   636 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:18:51.679319   324 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.599s	user 0.008s	sys 0.004s
I20260709 12:18:51.694600   324 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:51.649238   469 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.641709   611 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 12:18:51.662087   615 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:18:51.696013 28986 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:51.676404 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:51.706502 28289 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:18:51.706408 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:51.709601 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:51.710570 28361 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:51.711508 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:51.714175 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:51.717221 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:51.650620   470 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.718971 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:51.720707 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:51.726773 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.649591   491 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.781355   512 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:51.783895 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 116163us
I20260709 12:18:51.785892 28288 raft_consensus.cc:2410] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:18:51.786111 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.788317 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63477.1us
W20260709 12:18:51.787097 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.791292 28285 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:51.792625 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:51.837679 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:51.769188   611 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:51.705471   602 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 12:18:51.781654   596 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:51.878942 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:51.879849 28435 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:51.881448 28438 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:51.882409 28435 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:51.721666   505 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:51.692905   617 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:51.770190   521 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.864609 28359 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:51.787362   512 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.865448 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:51.784214   591 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:51.868979 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:51.839846 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 54820.9us
I20260709 12:18:51.841192 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:51.840142 28289 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:51.865718   530 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.885174 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 525) took 1531 ms. Trace:
I20260709 12:18:51.884356 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:51.885634 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:51.875682   337 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.937160 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:51.929140 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:51.838410   625 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.945762 28989 rpcz_store.cc:276] 0709 12:18:50.353788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.353932 (+   144us) service_pool.cc:224] Handling call
0709 12:18:50.365539 (+ 11607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.412693 (+ 47154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.884924 (+1472231us) spinlock_profiling.cc:243] Waited 37.9 ms on lock 0x7b540046e954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:51.885106 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87898112}
I20260709 12:18:51.950395 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:51.955482 28361 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:51.956691 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:51.957902 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:51.958591 28432 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:51.958864 28436 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:18:51.941666   614 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.380s	user 0.001s	sys 0.000s
I20260709 12:18:51.968315 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:51.651186   623 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.948810   596 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.972680   512 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:51.973527   641 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:51.885833   505 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:51.977188   628 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.001541 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:52.002893 28988 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:18:51.998878 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:52.008816 28990 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 12:18:51.984503 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.011896 28359 raft_consensus.cc:2410] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:51.987115   324 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:51.885532   616 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:52.027784   616 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:51.998459 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.030146 28435 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:52.031231 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:52.031971 28435 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:52.000140 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.032974 28438 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:52.000985 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:52.008714 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.034228 28433 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:18:51.982692 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.036388 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.037616 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.038254 28367 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:52.018411 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.033951 28982 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:52.017182 28361 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:52.009027   628 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:51.792227 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:52.038508 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.038508 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:52.046360   633 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
W20260709 12:18:52.024068   445 consensus_meta.cc:287] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.729s	user 0.008s	sys 0.002s
I20260709 12:18:51.956457   530 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.048125 28982 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:52.075240 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.075305 28284 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:51.999856   505 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:52.083843 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:52.078524 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:52.001636   512 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.082746 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2196) took 1401 ms. Trace:
I20260709 12:18:51.983311   605 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.025858   623 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:52.087220 28284 rpcz_store.cc:276] 0709 12:18:50.682253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.038255 (+1356002us) service_pool.cc:224] Handling call
0709 12:18:52.082662 (+ 44407us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:18:52.077471   614 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 12:18:52.134037 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 55677.6us
I20260709 12:18:52.136045 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:51.996411   596 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.139894 28982 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.081159   627 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.138664 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:52.145089 28359 pending_rounds.cc:85] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 5
I20260709 12:18:52.161653 28359 pending_rounds.cc:107] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:52.139595   512 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.148020   616 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:52.150511   628 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:52.137142   431 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.167434 28360 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:18:52.168107 28365 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:52.153165   587 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:52.142647   623 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:52.146123 32555 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.147423 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:52.144068 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:51.978523   625 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.003897   641 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.069144   570 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:52.137404   445 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.158814   491 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:52.162165 28359 raft_consensus.cc:2889] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:52.157308 32717 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.940s	user 0.014s	sys 0.006s
I20260709 12:18:52.143476   645 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:52.187732 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 43782.8us
I20260709 12:18:52.190901 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:52.180114 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:52.192428 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:52.193166 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:52.194679 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:52.196684 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:52.172372   603 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.746s	user 0.012s	sys 0.000s
I20260709 12:18:52.158035 32578 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:18:52.173046   623 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:52.202333 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.202395   648 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:52.206132 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:52.202327   646 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.186079   628 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.225036 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:18:52.219007 32728 consensus_meta.cc:287] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.069s	user 0.009s	sys 0.005s
I20260709 12:18:52.201339   512 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:52.198309   570 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.171382   431 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.208142   627 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:52.218227   623 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.207890   587 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.224207   616 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.207122   491 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:52.224663   639 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 12:18:52.166627   611 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:52.211184   641 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:52.200791   642 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:18:52.208410 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:52.189630 32555 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:52.239414 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 735) took 1250 ms. Trace:
I20260709 12:18:52.211413   645 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
W20260709 12:18:52.235350 32733 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.804s	user 0.011s	sys 0.002s
I20260709 12:18:52.222245   603 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.244536 28439 rpcz_store.cc:276] 0709 12:18:50.989398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.989537 (+   139us) service_pool.cc:224] Handling call
0709 12:18:51.017712 (+ 28175us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:51.188717 (+171005us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:51.208018 (+ 19301us) log.cc:844] Serialized 14 byte log entry
0709 12:18:51.315768 (+107750us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:51.316768 (+  1000us) log.cc:844] Serialized 52 byte log entry
0709 12:18:51.316984 (+   216us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:51.317407 (+   423us) log.cc:844] Serialized 14 byte log entry
0709 12:18:51.558543 (+241136us) log.cc:844] Serialized 14 byte log entry
0709 12:18:51.558773 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.558877 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:52.239142 (+680265us) raft_consensus.cc:1685] finished
0709 12:18:52.239166 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:52.239337 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27863,"prepare.queue_time_us":25211,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":3016,"raft.queue_time_us":387945,"raft.run_cpu_time_us":739,"raft.run_wall_time_us":49865,"thread_start_us":434350,"threads_started":4,"wal-append.queue_time_us":214248}
I20260709 12:18:52.287859 28437 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:52.241410   491 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:52.288565 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.290350   603 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:52.249424   611 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.245369   641 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.288650   491 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:52.299862 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.273027 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.303422   590 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:52.303758   582 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.304136   590 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.304476   582 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.306712   590 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.308036   582 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.284454   596 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:52.311408 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.312352 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.251871 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:52.314848 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 63089us
I20260709 12:18:52.301021 28365 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: [9.9-14.12]
I20260709 12:18:52.318910 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.319995 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.320515 28365 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:52.321277 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.321951   641 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.322772   641 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:52.323696   641 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:52.324067   641 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:52.304641 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.325151 28360 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:52.268468   613 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.276692 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2243) took 1148 ms. Trace:
I20260709 12:18:52.285140   646 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.326699   613 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.326586 28366 rpcz_store.cc:276] 0709 12:18:51.128706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.128849 (+   143us) service_pool.cc:224] Handling call
0709 12:18:51.164230 (+ 35381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:51.684536 (+520306us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b18003327a0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85be87f060 00007f85be87fbfb 00007f85be88a64b
0709 12:18:51.684694 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:51.684722 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:51.685205 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:18:51.841533 (+156328us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:51.841683 (+   150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.841797 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:52.276288 (+434491us) raft_consensus.cc:1685] finished
0709 12:18:52.276314 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:52.276601 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1143,"raft.queue_time_us":41537,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":3610,"spinlock_wait_cycles":6385920,"thread_start_us":204593,"threads_started":2,"wal-append.queue_time_us":323312}
I20260709 12:18:52.313477 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.330330 28358 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:52.330446 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:52.331758 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:52.282897   530 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.331422   641 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.333725 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:52.333910   641 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.327153   521 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.337937   530 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:52.338404   530 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.338099   596 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:52.337220   521 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.339486   596 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:18:52.340567   596 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:52.240180 32728 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:52.340992   596 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:52.342775   431 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.342473   596 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.345044   596 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:52.345378   632 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.648s	user 0.000s	sys 0.001s
I20260709 12:18:52.349694   385 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.350457 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.351169 28358 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:52.353892 32733 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.357800   396 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.358364   396 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.360452   396 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.362538 32733 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:52.366092   629 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.492s	user 0.002s	sys 0.000s
I20260709 12:18:52.340696   521 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:52.367306 28365 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:52.369050   591 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:52.368392 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2195) took 1697 ms. Trace:
I20260709 12:18:52.369587   591 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.369853 28287 rpcz_store.cc:276] 0709 12:18:50.671993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.720199 (+ 48206us) service_pool.cc:224] Handling call
0709 12:18:51.146478 (+426279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:51.154631 (+  8153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:51.154665 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:51.167673 (+ 13008us) log.cc:844] Serialized 26 byte log entry
0709 12:18:51.265377 (+ 97704us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:51.265417 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.265485 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:51.641994 (+376509us) raft_consensus.cc:1685] finished
0709 12:18:51.642013 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:52.368035 (+726022us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b5400163350. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:52.368319 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7704,"raft.queue_time_us":8995,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8281,"spinlock_wait_cycles":949805824,"thread_start_us":63487,"threads_started":1,"wal-append.queue_time_us":165236}
I20260709 12:18:52.324575 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2511) took 1458 ms. Trace:
I20260709 12:18:52.372932 28440 rpcz_store.cc:276] 0709 12:18:50.866191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.899975 (+ 33784us) service_pool.cc:224] Handling call
0709 12:18:52.324350 (+1424375us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54002e2cd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:52.324485 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615060864}
I20260709 12:18:52.374723 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.376628   385 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.380818 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.276750   625 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.382196   521 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:52.384091 28366 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:52.372138   591 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:52.385476 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.388497 28440 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:52.340687   470 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:52.389966   513 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:52.390410   513 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:52.390681 28185 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 116) took 20653 ms (client timeout 15000 ms). Trace:
I20260709 12:18:52.388759 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.392441 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.392776 28366 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:52.381668 28360 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:18:52.394348 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.395100 28438 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:52.396404 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.397141 28365 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:52.396937 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.394977 28433 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:52.390897 28432 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:52.391252   513 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.408746 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:52.410079 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:52.410887 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:52.448050 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.448787 28358 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:18:52.449646 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:52.443639 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.338817   530 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:52.199854 32717 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.453339 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:52.455168 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.422030 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.458832 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 659) took 1314 ms. Trace:
I20260709 12:18:52.427361 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:52.459270 28986 rpcz_store.cc:276] 0709 12:18:51.144082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.144270 (+   188us) service_pool.cc:224] Handling call
0709 12:18:52.458585 (+1314315us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54002ab554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:52.458755 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182698880}
W20260709 12:18:52.462375 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 104710us
W20260709 12:18:52.443122   637 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.460s	user 0.001s	sys 0.002s
I20260709 12:18:52.432006   625 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.456862 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.467777 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.445375 32733 consensus_queue.cc:1059] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:18:52.468158 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:52.467851 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.444991   634 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:18:52.490149   600 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.493054   605 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:52.471325   623 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.455384   587 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:52.472324   570 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:52.463137   396 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.487665 28982 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.382870   521 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.463485 32757 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.492367 28362 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:52.508596 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:52.441869   621 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:52.511910 28987 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:18:52.494220   617 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:52.541592 28991 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:52.515597 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.531131 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:52.510164   470 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:52.495790   625 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.496816   505 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.561678 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.563269 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.563992 28360 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:18:52.514533   603 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:18:52.565325 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.572624 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:52.557921   507 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.512641   600 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.576815 28288 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:18:52.580564   507 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.583470   507 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.584707 28366 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:52.592082   623 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:52.593010   625 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:52.593367 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:52.561060   470 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:52.570384 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:52.568044   603 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:52.600835 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 373622us
I20260709 12:18:52.607249 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:52.617563 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:52.618614 28990 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:18:52.555080   396 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.544061   621 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:52.624951 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54666.1us
I20260709 12:18:52.626300 28987 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 12:18:52.625033 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 226358us
I20260709 12:18:52.626189 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:52.629161 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:52.633769 28358 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:52.633698 28984 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:52.632966 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:52.621392   625 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:52.678645   625 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:52.679359   625 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:52.679440   611 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:52.680092   611 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.680557   611 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.625608   307 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:18:52.678974 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46074.8us
I20260709 12:18:52.684376 28257 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:52.617622   470 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:52.694341 28358 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:52.648389   570 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.706422 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:52.681838   605 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:52.692515   307 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.650099   445 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:18:52.693008 32717 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:52.715252 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 163844us
W20260709 12:18:52.716245 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.655680   654 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
W20260709 12:18:52.730510 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.733322 28284 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:18:52.745489 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 280787us
I20260709 12:18:52.748905 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:52.749886 28285 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:18:52.750777 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:52.751549 28285 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:18:52.752542 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:52.754470 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:52.746989 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:52.761610 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:52.763983 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:52.764704 28290 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.762949 28283 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:18:52.774037 28286 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:52.718971   307 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:52.776916 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.777638 28364 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:52.778959 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:52.782289   655 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:18:52.721393   596 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:52.718382   656 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:52.786315 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.708535   579 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:18:52.778697 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:52.787834 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:52.788359 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.788532 28287 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:52.778681 28286 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:52.790637 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.800856 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:52.791414 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.709545   570 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.799360 28283 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:52.806960 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.802536 28991 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:18:52.804450 28432 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:18:52.804064 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 57146us
I20260709 12:18:52.807721 28435 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:52.697036   603 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:52.817098 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:52.818166 28292 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:18:52.818928 28432 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:52.819506 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:52.807276 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:52.808562   653 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
W20260709 12:18:52.824227 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.824448 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:52.825398 28290 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:52.796049   596 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.750931 32728 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:18:52.840534 28358 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:52.829461 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:52.829964 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.830266 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.830798 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:52.852701 28432 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:52.832023 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2257) took 1051 ms. Trace:
I20260709 12:18:52.853575 28359 rpcz_store.cc:276] 0709 12:18:51.780547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.076733 (+296186us) service_pool.cc:224] Handling call
0709 12:18:52.093147 (+ 16414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:52.191337 (+ 98190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:52.191373 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:52.191956 (+   583us) log.cc:844] Serialized 52 byte log entry
0709 12:18:52.601201 (+409245us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:52.601255 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.601338 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:52.831754 (+230416us) raft_consensus.cc:1685] finished
0709 12:18:52.831780 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:52.831947 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7525,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5464,"thread_start_us":364009,"threads_started":1,"wal-append.queue_time_us":411632}
W20260709 12:18:52.391258 28185 rpcz_store.cc:269] 0709 12:18:31.737429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.737657 (+   228us) service_pool.cc:224] Handling call
0709 12:18:52.390552 (+20652895us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:49.287083 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:49.287770 (+   687us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:49.287825 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:18:49.287843 (+    18us) tablet.cc:662] Decoding operations
0709 12:18:49.731724 (+443881us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:49.757823 (+ 26099us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:49.760610 (+  2787us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:18:49.801339 (+ 40729us) tablet.cc:701] Row locks acquired
0709 12:18:49.801396 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:18:49.801591 (+   195us) write_op.cc:270] Start()
0709 12:18:49.801808 (+   217us) write_op.cc:276] Timestamp: P: 1783599529801514 usec, L: 0
0709 12:18:49.801846 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:18:49.802745 (+   899us) log.cc:844] Serialized 6601 byte log entry
0709 12:18:50.485490 (+682745us) op_driver.cc:464] REPLICATION: finished
0709 12:18:50.702397 (+216907us) write_op.cc:301] APPLY: starting
0709 12:18:50.702521 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:50.822027 (+119506us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:50.822133 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:51.087564 (+265431us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:51.087932 (+   368us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:18:51.087992 (+    60us) write_op.cc:312] APPLY: finished
0709 12:18:51.088463 (+   471us) log.cc:844] Serialized 148 byte log entry
0709 12:18:51.096424 (+  7961us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:51.096629 (+   205us) write_op.cc:454] Released schema lock
0709 12:18:51.096710 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:51.125696 (+ 28986us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:51.125754 (+    58us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":317,"mutex_wait_us":427191,"reported_tablets":24,"spinlock_wait_cycles":41856,"child_traces":[["op",{"apply.queue_time_us":216565,"apply.run_cpu_time_us":24180,"apply.run_wall_time_us":423230,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70975,"num_ops":16,"prepare.queue_time_us":218972,"prepare.run_cpu_time_us":44093,"prepare.run_wall_time_us":616503,"raft.queue_time_us":324854,"raft.run_cpu_time_us":10503,"raft.run_wall_time_us":160760,"replication_time_us":683440,"thread_start_us":605905,"threads_started":4,"wal-append.queue_time_us":319014,"wal-append.run_cpu_time_us":61337,"wal-append.run_wall_time_us":1980227}]]}
W20260709 12:18:52.828651   491 consensus_meta.cc:287] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.000s
W20260709 12:18:52.834916 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 204981us
W20260709 12:18:52.832846 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 196576us
I20260709 12:18:52.838788 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:52.807133   587 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:52.827520   623 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.799319   656 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:52.815354   627 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:18:52.834354   507 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:52.872045 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 71308.6us
I20260709 12:18:52.827988   600 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:52.834587 32733 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:52.859226 28292 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:52.836462 28361 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:52.859195   603 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.917433 28982 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:52.925132 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:52.931510 28982 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 663) took 1781 ms (client timeout 1992 ms). Trace:
I20260709 12:18:52.851411   605 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:52.932385 28982 rpcz_store.cc:269] 0709 12:18:51.150179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.150397 (+   218us) service_pool.cc:224] Handling call
0709 12:18:52.000864 (+850467us) spinlock_profiling.cc:243] Waited 915 us on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762eee 000000000036eb39 00007f85c37d61e0 00007f85c389f0c4 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:18:52.487428 (+486564us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b580087b6d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7e30aa 00007f85be865447 00007f85be87dfe7 00007f85be87f7de 00007f85be88b27c 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c
0709 12:18:52.931438 (+444010us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315868416}
I20260709 12:18:52.935985   445 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:52.936546   445 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:52.936755   651 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
W20260709 12:18:52.937072   675 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:18:52.939661   445 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.945353 28364 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:18:52.879735   396 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:52.849041   596 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:18:52.923223   469 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.212s	user 0.015s	sys 0.000s
W20260709 12:18:52.949787 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 322497us
W20260709 12:18:52.944336   309 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:18:52.948978 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:52.962973 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:52.951928   657 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 12:18:52.952965   640 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.999s	user 0.004s	sys 0.001s
I20260709 12:18:52.941885   621 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:52.946938   627 consensus_queue.cc:1243] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:52.943956   521 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:52.917016   623 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.941599   666 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:52.982249 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:52.942914   565 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:18:52.959658   469 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:52.942361   656 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.985695   512 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:53.001971   512 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:53.003350   663 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:18:53.003062   512 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.941031 32733 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.004987   623 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:53.005908 28434 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
W20260709 12:18:52.990800 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.948115   581 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:18:53.007071 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2259) took 1126 ms. Trace:
I20260709 12:18:52.968595   600 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.007828   600 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.007495 28367 rpcz_store.cc:276] 0709 12:18:51.880785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.134093 (+253308us) service_pool.cc:224] Handling call
0709 12:18:52.134476 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:52.136452 (+  1976us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:18:52.158062 (+ 21610us) log.cc:844] Serialized 14 byte log entry
0709 12:18:52.712918 (+554856us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:52.721177 (+  8259us) log.cc:844] Serialized 52 byte log entry
0709 12:18:52.721385 (+   208us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:18:52.721425 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.721504 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:53.006845 (+285341us) raft_consensus.cc:1685] finished
0709 12:18:53.006867 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:53.007003 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16772,"raft.queue_time_us":239363,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":17626,"spinlock_wait_cycles":5748480,"thread_start_us":522076,"threads_started":2,"wal-append.queue_time_us":534073}
I20260709 12:18:52.972198   507 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:53.009979   507 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.007128 32733 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:53.010156   600 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:53.000216 28332 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:53.030882 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:53.003362   658 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:18:53.010449   507 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:52.971046   368 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.996613 28440 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:53.018546   627 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.881875   587 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:52.991659   666 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:53.037184 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:53.051441 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.027818   521 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:53.037088 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:53.038293 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:53.052271 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:53.052534 28360 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:53.052662 28433 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
W20260709 12:18:53.053066 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 190843us
I20260709 12:18:52.874786   491 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.039273 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:53.054433 28441 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:52.994823 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:53.009667   665 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:53.055001 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.047899 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:53.059953 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65242.3us
I20260709 12:18:53.057188 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:53.056682 28437 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.062207 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:53.062242 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.064746 28358 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:53.070420   673 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:53.072100 32728 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:53.072885 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 598) took 1872 ms. Trace:
I20260709 12:18:53.073310 28363 rpcz_store.cc:276] 0709 12:18:51.200133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.200250 (+   117us) service_pool.cc:224] Handling call
0709 12:18:51.212851 (+ 12601us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:51.216498 (+  3647us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:51.296169 (+ 79671us) log.cc:844] Serialized 14 byte log entry
0709 12:18:52.315340 (+1019171us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b18000c0f40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be42723a 00007f85be891a2a
0709 12:18:52.750887 (+435547us) log.cc:844] Serialized 14 byte log entry
0709 12:18:52.779212 (+ 28325us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:52.779845 (+   633us) log.cc:844] Serialized 52 byte log entry
0709 12:18:52.779999 (+   154us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:18:52.780238 (+   239us) log.cc:844] Serialized 14 byte log entry
0709 12:18:52.780554 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 12:18:52.780791 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.780868 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:53.072592 (+291724us) raft_consensus.cc:1685] finished
0709 12:18:53.072618 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:53.072816 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44208,"prepare.queue_time_us":355783,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":133,"raft.queue_time_us":474316,"raft.run_cpu_time_us":564,"raft.run_wall_time_us":57833,"spinlock_wait_cycles":90539648,"thread_start_us":514076,"threads_started":3,"wal-append.queue_time_us":1395009}
I20260709 12:18:53.078563 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2202) took 1238 ms. Trace:
I20260709 12:18:53.079125 28288 rpcz_store.cc:276] 0709 12:18:51.840567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.840850 (+   283us) service_pool.cc:224] Handling call
0709 12:18:53.078345 (+1237495us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b540018f454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:53.078480 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460547456}
I20260709 12:18:53.079763   682 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:53.080924 28289 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:53.050664   666 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.089520   513 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:53.092446   661 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:18:53.070334   685 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:53.095070   685 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.095451   685 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:53.099073 28360 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:18:53.072788 28366 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:18:53.102061 28438 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:53.055409   665 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.107738 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.115638 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:53.117502 28432 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:18:53.053651   530 consensus_meta.cc:287] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.587s	user 0.012s	sys 0.002s
I20260709 12:18:53.056594   521 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.052101   570 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:53.125903 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.104671 28255 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:53.130398 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:53.072793 32728 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:53.135325 28360 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:53.047114   368 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.081037   565 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:53.106853   431 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:53.118498   512 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:53.148942 28437 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:53.157176 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:53.127559   521 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.139459   682 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:53.139603   627 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:53.109496   596 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.137804   687 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:53.165303 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.201315 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 144205us
I20260709 12:18:53.157337   491 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:53.163532   682 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.144229   530 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.207944 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:53.211138 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:53.176329   627 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.154212   431 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.200871   647 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:18:53.208671   470 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.004s
I20260709 12:18:53.151428   565 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:53.222275 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65186.6us
I20260709 12:18:53.205036   506 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:53.223963 28255 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:53.225914 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:53.213536   396 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:53.212070   682 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:53.225983   613 consensus_meta.cc:287] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.896s	user 0.007s	sys 0.007s
I20260709 12:18:53.236428 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:53.239511 28988 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:53.160792   512 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.220897   627 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:53.219394   647 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.222731 32728 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.203504   687 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.247635   656 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.231128   669 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:53.233269   531 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:18:53.271255 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46247.5us
W20260709 12:18:53.271718   689 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 12:18:53.272974 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:53.224112   470 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.229485   506 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.280086 28284 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:53.281265   470 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.281806 28435 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:18:53.259393   613 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:53.258671   647 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:53.261852   625 consensus_meta.cc:287] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.008s
W20260709 12:18:53.292109 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 429307us
I20260709 12:18:53.262785   531 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:53.296541 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:53.276789   669 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:53.290795   307 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.301959   669 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.302973 28991 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:53.304234   307 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.307322   512 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.308305   512 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:53.308749   512 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.309123   512 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.310333   688 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.315613 28289 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:18:53.289608   613 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.320116   565 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.320621   506 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.323093   687 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.323565   445 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.325995   505 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:53.329290 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.317315 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44443.5us
I20260709 12:18:53.330513 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:53.335991   687 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.327973   654 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:53.343178 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:53.314416   368 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.349150   565 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:53.329208   613 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:53.336644   687 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.296149   625 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.324170   665 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.337121   596 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.368513 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:53.346596   654 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:53.370839 28988 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "114fa146124747358c997ac1008e1b93"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:18:53.378221 28988 raft_consensus.cc:606] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:53.378614 28988 raft_consensus.cc:614] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:53.383677 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 53275.5us
I20260709 12:18:53.384933 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:53.318511   666 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.364102   445 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:53.378997   611 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.003s
I20260709 12:18:53.425669   702 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:53.318912   685 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:53.440608 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 211326us
I20260709 12:18:53.379688   700 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:53.444906 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 60065.3us
I20260709 12:18:53.445786 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:53.381867   565 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:18:53.458014   704 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:53.458616   704 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.363863   531 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.445819   590 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:18:53.460057   531 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.445436   611 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.445557   702 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:53.463243   702 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.464418 28291 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:53.459007   704 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.476248 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.476936 28358 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:53.477782 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.480093 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:53.480880 28363 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:53.482026 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.483273 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.484349   625 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.484387 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.487140   665 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:53.488790   396 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:18:53.451705   513 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:18:53.500116 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 122429us
I20260709 12:18:53.506937 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:53.503396 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:53.499500 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2534) took 1138 ms. Trace:
I20260709 12:18:53.507808 28439 rpcz_store.cc:276] 0709 12:18:52.360861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.360979 (+   118us) service_pool.cc:224] Handling call
0709 12:18:52.361328 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.438864 (+ 77536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:52.438892 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:52.438935 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:52.438963 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.499240 (+1060277us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b540026a2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:53.499290 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:53.499438 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177838336}
W20260709 12:18:53.507623 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.510694 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:53.500356 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:53.513509 28330 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:53.502038   625 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:18:53.516283   672 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:53.518888 28358 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:53.522845 28367 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:53.521587 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:53.524466 28366 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:53.528528 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.499437   611 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.529233 28292 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:18:53.529788 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:53.534446 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:53.518361   505 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:53.530817   506 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:53.481632   682 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.536654 28289 pending_rounds.cc:85] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 11
I20260709 12:18:53.518198   695 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:53.558521   505 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:53.559394   666 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.540283 28289 pending_rounds.cc:107] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:18:53.520762   513 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:53.541008 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:53.533965   396 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:53.454651   654 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.565435   654 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.546518   611 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:53.531136   709 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:18:53.550753   669 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:53.551020   679 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:53.551546   368 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.541603   565 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:18:53.553213 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:53.535678   507 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.555572 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2535) took 1194 ms. Trace:
I20260709 12:18:53.562357 28289 raft_consensus.cc:2889] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:53.591374 28441 rpcz_store.cc:276] 0709 12:18:52.361155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.361498 (+   343us) service_pool.cc:224] Handling call
0709 12:18:52.361972 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.054816 (+692844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.555283 (+500467us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b5400085554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:53.555493 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1592093952}
I20260709 12:18:53.581331   565 consensus_queue.cc:1243] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:18:53.581575 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:53.586263 28174 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 114fa146124747358c997ac1008e1b93 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f526369c0d0743b192be6e205540a059 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:53.566499   611 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:53.578452   669 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.579986   368 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.561182   506 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:53.590359   513 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:53.568699 32578 consensus_queue.cc:1175] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6634fa3fbd594726a5ec3c1686900284" responder_term: 13 status { last_received { term: 13 index: 7 } last_committed_idx: 4 last_received_current_leader { term: 13 index: 7 } }
I20260709 12:18:53.585289   704 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.611439 28358 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:53.610745 28283 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:53.613880 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:53.579259   679 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:53.617427   682 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.600629   687 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.626130   634 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:53.626672   634 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:53.627565   634 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.628719   634 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:53.629439 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2247) took 2424 ms. Trace:
I20260709 12:18:53.612097   700 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:53.630277 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53280 (request call id 749) took 1237 ms. Trace:
I20260709 12:18:53.631058 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.633271 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:53.630686 28440 rpcz_store.cc:276] 0709 12:18:52.393254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.394759 (+  1505us) service_pool.cc:224] Handling call
0709 12:18:53.630071 (+1235312us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b5400415854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:53.630191 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1254181248}
I20260709 12:18:53.634189 28364 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:53.634310   512 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.634867   625 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:18:53.610869   506 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.629910 28362 rpcz_store.cc:276] 0709 12:18:51.205265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.211671 (+  6406us) service_pool.cc:224] Handling call
0709 12:18:51.220339 (+  8668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.541323 (+1320984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.629101 (+1087778us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400370ad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:53.629344 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2924910336}
I20260709 12:18:53.660202 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:53.661801 28360 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:53.662813 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.664683 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:53.666801 28358 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:53.668982 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:53.669867 28437 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:53.672266   680 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:53.672933   680 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.675794 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:53.676661 28440 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:53.677989   648 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:53.678495   648 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:53.678926   507 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.658192   625 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:53.679490   648 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.679881 28361 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:18:53.680610   625 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.683511   512 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.637637   700 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.684576   700 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.691943   538 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:53.681032   648 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.694911   687 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:18:53.705392 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:53.705653   712 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:18:53.637815   647 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.717834   647 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.706213   679 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.707635   685 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.707907   587 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:53.705286 28289 pending_rounds.cc:85] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 11
I20260709 12:18:53.725698 28289 pending_rounds.cc:107] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:18:53.726025 28289 raft_consensus.cc:2889] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:53.692543   538 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:53.713891 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:53.718039   368 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.715193   704 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.635759   688 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:53.720458 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 118357us
I20260709 12:18:53.716128   687 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:53.613083   627 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:53.710018   521 consensus_meta.cc:287] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.005s
W20260709 12:18:53.730502 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.744444 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:53.750852 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.751662 28358 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.742470   470 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:53.742321   431 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.753383   538 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.753611   470 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.630280   611 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.736377   687 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:53.764897 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:53.761497 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:53.752736   603 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:53.785055 28433 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 6. (term mismatch)
I20260709 12:18:53.785533 28433 pending_rounds.cc:85] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 5
I20260709 12:18:53.785795 28433 pending_rounds.cc:107] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:53.786032 28433 raft_consensus.cc:2889] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:53.786470 28433 pending_rounds.cc:107] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:18:53.786734 28433 raft_consensus.cc:2889] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:53.799448   470 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.800249   470 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:53.800649   470 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.800995   470 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:53.749997   521 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.776209   538 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.807933 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.810582 28359 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.811641 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.812371 28366 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.816751 28360 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:18:53.809594   716 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:18:53.778283   603 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.828272   603 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.840894 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.841464 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.842166 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.842381 28283 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:53.844527 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.845355 28287 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:18:53.847308 28287 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:53.847544 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.848255 28290 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
W20260709 12:18:53.850454 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 85605.4us
I20260709 12:18:53.851778 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:53.852627   720 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:53.852727 28366 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:53.853145   720 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.853545   720 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:53.845223 28283 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:53.854844 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:53.842803 28288 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:53.858515 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.859143 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:53.860292 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.861184 28359 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
W20260709 12:18:53.865027 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.866534 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.861682 28358 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:53.868251 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.869073 28289 raft_consensus.cc:2368] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:53.864029 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:53.864957 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.868494 28287 raft_consensus.cc:2368] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15.
I20260709 12:18:53.825881   587 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:53.867623 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:53.872506 28991 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:53.862319 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.873483 28366 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:53.874683 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.875628 28359 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 10: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:53.854728   627 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.855471   396 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.883216 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.892398 28991 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:18:53.883961 28283 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:18:53.894400 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:53.901072 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.897891 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.902993 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:53.903668 28289 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:53.904119 28359 raft_consensus.cc:2393] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:53.901963   603 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:18:53.907168 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:53.901475 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.825424   687 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.897496   656 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.896101   669 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.918241 28366 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
W20260709 12:18:53.920779 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 49175.2us
I20260709 12:18:53.922233 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:53.908803   627 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:53.900360   396 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.935412 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.943590   702 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.905372   721 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:18:53.947021 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:53.947091 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.951632   704 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:53.952509   396 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.953421 28359 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:53.934376   627 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:53.954777   627 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.952282   704 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.963657   704 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:53.934059   669 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 12:18:53.957698   718 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:18:53.965135   702 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:53.967504 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.947672 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:53.967819   700 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:53.969259 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2284) took 1140 ms. Trace:
I20260709 12:18:53.969667 28434 rpcz_store.cc:276] 0709 12:18:52.829307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.830003 (+   696us) service_pool.cc:224] Handling call
0709 12:18:53.969056 (+1139053us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400415d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:53.969183 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352787328}
I20260709 12:18:53.970029   726 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:53.970610   726 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:53.970840 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:53.972078   368 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:53.960428   521 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:53.974186 28287 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:53.972002 28991 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 12:18:53.976861 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:53.978849 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.979457   385 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:53.980145   385 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:53.981042   385 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:53.982190 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.979761 28287 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:53.995015 28362 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:53.996523 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:53.997398 28283 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:53.997578 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:53.998407 28991 raft_consensus.cc:2368] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:18:53.999866 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.000202 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.000846 28290 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:53.995255 28366 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:54.000674 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.003006 28286 raft_consensus.cc:2410] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:54.003767 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.004472 28982 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:54.010080 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.010305 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.011185 28287 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:54.010125 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:53.998131   613 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.623s	user 0.014s	sys 0.000s
I20260709 12:18:54.012825 28982 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:54.003014 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:54.018165 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:53.985086   666 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:18:54.023912 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.024569 28290 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:54.026345 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.026562   669 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:54.021535 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2295) took 1271 ms. Trace:
I20260709 12:18:54.026969   669 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.021466 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.026132 28332 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:54.027412   669 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.028077 28434 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:54.030207   665 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:54.027917   368 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:54.031464   368 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.031879   368 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.032514 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.022812 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2304) took 1239 ms. Trace:
I20260709 12:18:54.033208 28361 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:54.027240 28284 rpcz_store.cc:276] 0709 12:18:52.750509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.751107 (+   598us) service_pool.cc:224] Handling call
0709 12:18:52.751560 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.285637 (+534077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.021291 (+735654us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54000b0254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:54.021466 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211792128}
I20260709 12:18:54.035460 28988 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "189fdd6055df4826a42af9c225596e34"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:18:54.036115 28988 raft_consensus.cc:606] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:54.036446 28988 raft_consensus.cc:614] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:54.038465 28435 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:18:54.038861 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:54.039251 28290 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:54.039888 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:54.040643 28362 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:54.042222   368 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.043627   669 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.044009 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:54.044894 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:54.046967 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.047574 28433 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:54.048617 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:54.049381 28434 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:54.051029 28435 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:18:54.051532 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.052402 28284 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:18:54.052865 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.053193 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.033540 28291 rpcz_store.cc:276] 0709 12:18:52.783794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.783910 (+   116us) service_pool.cc:224] Handling call
0709 12:18:52.784217 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.464953 (+680736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.022519 (+557566us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54003f9654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:54.022738 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1563778944}
I20260709 12:18:53.999301   685 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:18:54.002074   506 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.054929 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.056341 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.055932 28956 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.058041 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.058357 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.053546 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.059995 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:54.060369 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.063189   595 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:54.017488   613 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.066913 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.123592 28433 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:18:54.122108 28285 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:54.061640   726 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.123338 28434 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:54.123405 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.132696 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.133360 28434 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.053484 28288 raft_consensus.cc:2376] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13. Re-sending same reply.
W20260709 12:18:54.113386 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.057021   721 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:18:54.129956 28289 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
W20260709 12:18:54.137137 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 214987us
I20260709 12:18:54.127384 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:54.141986 28433 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:54.148020 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.151688 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.152920 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:54.155488 28292 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:54.156888 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:54.131018   595 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.166494 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.159974 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:54.162850 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.167269 28438 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.167508 28436 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.172340 28991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:18:54.173355 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:54.164798 28434 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:54.182989 28435 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:18:54.166811   603 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.181176 28362 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:18:54.173902   595 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.172415   634 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:54.174685   648 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:54.182742 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.222671 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:54.218576 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:54.180451   470 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.176648   722 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:18:54.200083   634 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:54.208135   506 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:54.209698   521 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:54.212323   613 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:54.226580   700 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:54.249924 28436 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:18:54.206542   727 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:54.217417   538 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:18:54.260636 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:54.261278 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.264633 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.230549   728 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:54.267262 28440 raft_consensus.cc:2368] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18.
I20260709 12:18:54.267760 28436 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.267117 28434 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:54.233050   530 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:54.256953   634 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.265931   722 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:54.283330 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.286983 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.266320   727 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:54.269647   669 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:54.313256 28434 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:54.312867 28330 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.290293   513 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:54.326421 28436 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:54.328023 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.267835   521 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.332811 28434 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.283812   722 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:54.312359   666 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:54.331568 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.329166 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.311946   665 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:54.338241 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 119661us
I20260709 12:18:54.318565 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:54.315073   613 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:54.346098 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:54.316241   727 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.274731   728 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.330902   670 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:54.351542 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:54.354431 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.333645   722 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.349393 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.357558 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.357859 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.358536 28284 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.359760 28404 leader_election.cc:400] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15.
I20260709 12:18:54.359941 28434 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:18:54.360249 28404 leader_election.cc:403] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:54.361083 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:54.361799 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.366460 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:54.347997   669 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.374081 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.375820 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:54.376106 28432 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:54.357658   603 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.337257   720 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.362284   666 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:54.363724   726 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:54.385810 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:54.388332 28363 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.386555 28437 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:54.381310 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:54.390527 28437 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:18:54.390872 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:54.352061   613 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.395427 28437 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:54.384212   670 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:54.400831 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.402796 28284 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:54.403954 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:54.404323 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.404695 28437 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:18:54.407032 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:54.407812   695 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:54.414219   732 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 12:18:54.415337   730 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:18:54.421712   735 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:18:54.422847   687 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.428189 28283 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
I20260709 12:18:54.433866   743 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.434818   743 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.435480   743 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15.
I20260709 12:18:54.439677   729 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:54.441049   728 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:54.443311   613 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:54.392258   726 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:54.444067   726 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:54.441912   743 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.445124   743 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.447850   743 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:54.399765   742 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:18:54.454645   742 consensus_queue.cc:1243] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:54.455097   729 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:54.455237   742 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:54.455742   742 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:54.456090   729 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.456547   742 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.460738   625 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:54.463610 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:54.464241 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.464888 28290 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:54.465306 28367 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:18:54.456971 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.455371   680 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:54.469417   521 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:54.397635   669 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:18:54.470790   669 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:54.471159   669 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:54.471171 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.472254 28991 raft_consensus.cc:606] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:54.472086   669 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.457247   729 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:54.385526   603 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.394156   396 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:54.472029 28982 raft_consensus.cc:2393] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:54.472637 28991 raft_consensus.cc:614] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:54.474212 28367 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:54.494058 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:54.494962 28438 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
W20260709 12:18:54.496208 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 136586us
I20260709 12:18:54.497215 28404 leader_election.cc:400] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:54.399175   470 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:54.380530   385 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:18:54.477892 28436 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:18:54.484045 28290 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:54.487370 28363 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
W20260709 12:18:54.475502 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.497560 28404 leader_election.cc:403] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:18:54.475659 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.498764 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:54.499962 28437 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:54.411728   665 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:54.497826   737 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:18:54.406026   672 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.498697 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.511631 28440 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:18:54.502570   738 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:18:54.506111 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:54.406955   666 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.508065 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:54.492297   670 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:54.497895   736 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:18:54.467016   720 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:54.505776 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.517650 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:54.508361   590 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:18:54.504093   742 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:54.517936 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.518724 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.520192 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.506966   309 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:54.523947 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.527765 28984 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:54.546217 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2274) took 1073 ms. Trace:
I20260709 12:18:54.546795 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:54.546398 28989 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:18:54.511197   470 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:18:54.507964   669 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.543201 28432 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:18:54.544823 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.524930   307 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.550683   664 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:54.551151   664 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:54.554221 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.555140 28291 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:54.557421 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.558287 28987 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
W20260709 12:18:54.559473   734 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.209s	user 0.004s	sys 0.004s
I20260709 12:18:54.546828   634 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.562456   634 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:54.567557 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:54.568509   666 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:54.569376   680 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:54.569639 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:54.569897 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.570559 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:54.570760   627 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:54.571456 28441 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:54.571688   368 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:54.572789   627 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:54.573137   647 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:54.573683   647 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:54.574048 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:54.574649   647 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.573164   506 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:54.576262   506 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:54.525489   728 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:54.577507   728 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:54.577232   506 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.578058   679 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.578542   506 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.576342   309 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.523236   709 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.569147   666 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.582254 28983 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:54.546649 28364 rpcz_store.cc:276] 0709 12:18:53.473170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.486570 (+ 13400us) service_pool.cc:224] Handling call
0709 12:18:53.633842 (+147272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.636624 (+  2782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.545995 (+909371us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400079750. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:54.546148 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337731840}
I20260709 12:18:54.571421 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:54.568427 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:54.583838 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:54.586199 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.596024 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:54.597719 28435 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:54.598984 28404 leader_election.cc:400] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:18:54.599316 28404 leader_election.cc:403] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:18:54.601131   680 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:54.601239 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:54.581562   309 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:54.603116 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.598816 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:54.603917 28982 raft_consensus.cc:2368] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:54.600322 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:54.606101   587 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:54.606542   587 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:54.607419   587 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.608682   587 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:54.609869   385 raft_consensus.cc:2764] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 12:18:54.611910   725 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.470s	user 0.001s	sys 0.001s
I20260709 12:18:54.613476 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:54.614226 28438 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:18:54.541498   603 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:54.600435 28440 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:54.627367 28332 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.546623   720 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:54.679179 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 78916.8us
I20260709 12:18:54.648658 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:54.680774 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:54.680825 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.682683 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:54.679527 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52262.5us
I20260709 12:18:54.686265 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.687259 28989 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:54.687624 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:54.572831   368 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
W20260709 12:18:54.691092 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 119749us
I20260709 12:18:54.699261 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:54.700459 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:54.691560 28404 leader_election.cc:400] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:54.701510 28404 leader_election.cc:403] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:54.602382   743 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:18:54.677440   700 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:54.717020 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:54.734328 28991 raft_consensus.cc:2410] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:18:54.690382   581 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:18:54.750289   713 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.036s	user 0.004s	sys 0.001s
I20260709 12:18:54.689579   745 raft_consensus.cc:2764] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:18:54.784785   726 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.791574   666 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:54.792829   691 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:18:54.794169   581 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:54.795275   581 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.735821   670 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.692986   603 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:54.711258   531 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:54.798862   531 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:54.711998   680 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.799799   531 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.696862   729 raft_consensus.cc:2764] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:18:54.687930   665 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:54.795825 28330 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.937s (SENT) [suppressed 8 similar messages]
I20260709 12:18:54.696121   538 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:54.697885   685 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.730378   530 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:54.696228   627 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:54.708571   695 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:54.738215   720 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:54.825083   530 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:54.578498   728 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.841063   530 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.842139   695 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.740221   700 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:54.844208   700 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:54.823470   727 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.848413   727 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:54.850134 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.850536 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.851274 28286 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:54.852313 28285 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:54.852283 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.852617 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.853452 28290 raft_consensus.cc:2368] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
I20260709 12:18:54.853797 28283 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:18:54.854230 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.854511 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.855345 28285 raft_consensus.cc:2410] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:18:54.853832 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.856300 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.851199 28289 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:54.841105 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 162325us
I20260709 12:18:54.853050 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:54.852223 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:54.712904 32578 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 12:18:54.862286 28436 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:18:54.812676   665 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.744639   743 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:54.836546   396 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:54.861306 28286 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:54.858721 28332 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.883457 28991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:18:54.884263 28991 raft_consensus.cc:606] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:54.884650 28991 raft_consensus.cc:614] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:54.863202 28436 pending_rounds.cc:85] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 6
I20260709 12:18:54.865518   727 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:54.823971   538 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:54.887820   538 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.900287 28991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:37070
W20260709 12:18:54.905504 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46871.5us
I20260709 12:18:54.907943   685 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:54.908525   685 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:54.864161 28983 raft_consensus.cc:2410] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 12:18:54.819273   595 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.618s	user 0.009s	sys 0.003s
I20260709 12:18:54.885650 28436 pending_rounds.cc:107] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:18:54.881835   396 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:54.886849   664 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.916711 28287 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:54.917083   664 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:54.918749   396 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.920150   396 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:54.872256 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:54.923856   627 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:54.863139 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:54.913561 28436 raft_consensus.cc:2889] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:54.927815 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 64746us
W20260709 12:18:54.930202 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:54.930773 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.931440 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:54.914121   708 raft_consensus.cc:2764] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:18:54.874366 28290 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:54.916103   743 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:54.940156 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:54.950632 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.862190   688 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:54.996613 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.866293   665 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.939180   720 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:54.912847   625 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.138s
I20260709 12:18:54.984429   744 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:54.909502   685 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.044749 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:55.045025 28988 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
W20260709 12:18:55.045513 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 249706us
I20260709 12:18:54.983279   595 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.051847 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:55.052049 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:55.035699   744 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.053292 28367 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:55.050180 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:55.049961 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:55.049571 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:55.051021 28366 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:55.054831 28360 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:55.055572 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:55.054206 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:55.054419 28365 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:18:55.047976 28362 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:55.054299 28363 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.056756 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:55.057283 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.061022 28364 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:55.061653 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.062564 28359 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: [10.7-13.8]
W20260709 12:18:55.047844   722 consensus_meta.cc:287] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.679s	user 0.010s	sys 0.002s
I20260709 12:18:55.041062   743 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:55.071549 28989 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:55.022014   688 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.076056 28986 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:55.058053   744 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.058154   720 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:55.089231 28991 raft_consensus.cc:606] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:55.074842   679 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.074669   745 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:55.086596   579 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:55.102293   748 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:18:55.124414 28363 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:18:55.138711 28360 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:18:55.074563   458 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:18:55.077056   722 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:55.144923 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 82411.6us
I20260709 12:18:55.130188   579 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.149704   579 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:55.130759   726 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:55.140134   752 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.106634   679 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:55.167016   754 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:18:55.178318 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:55.177211 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:55.180936 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:55.187115   751 log.cc:927] Time spent T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:18:55.189726   762 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.076s	user 0.001s	sys 0.001s
I20260709 12:18:55.187714   745 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:55.182037   634 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:55.217001   743 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:55.190042   720 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:18:55.235105 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 86365.3us
I20260709 12:18:55.182017   721 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:18:55.187820   726 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:18:55.247468   726 consensus_queue.cc:1243] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:55.247958   726 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:55.248272   726 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:55.249035   726 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.246488   721 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:55.249945   721 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:55.250823   721 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:55.246955   760 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:18:55.254776   722 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:55.255993 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:55.256608   721 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.257066 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:55.242645   745 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:55.259727   745 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.241463   634 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.262090   720 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:55.265645   720 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:55.264638   506 consensus_meta.cc:287] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.003s
I20260709 12:18:55.161321   752 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.271962   752 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:55.277539 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:55.279731 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:55.280541 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:55.281805   744 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.283989   744 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:55.284706   679 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:55.291775 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:55.292768 28436 raft_consensus.cc:2410] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:18:55.297036 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:55.297439 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:55.298189 28991 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.297942 28360 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.255975   688 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:55.304970 28433 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:55.307138   634 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:55.307739   556 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:55.311820   726 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.243993   743 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.321846   595 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:55.322556 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:55.323191 28436 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:55.325997 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 81642.4us
I20260709 12:18:55.328799 28360 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
W20260709 12:18:55.329881 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.330976 28991 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:55.336688 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:55.337865   726 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:55.338524   726 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:55.364523 28436 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:55.369659   764 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.370277   764 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.372413   764 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:55.373199 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:55.376628 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:55.379488   743 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:55.381359   666 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:18:55.384126 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:55.384846 28361 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:55.388145 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:55.388950 28991 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:18:55.390151 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:55.386633 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:55.348229   669 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:18:55.400439   766 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:55.401015   766 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:55.401396   766 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.386972   666 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:55.402417   666 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.403407   666 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:55.339545   726 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.393617   743 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:55.409015   743 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:55.409386   743 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.398300   685 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:18:55.410017   685 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.410352   685 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:18:55.404541   595 raft_consensus.cc:993] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:55.421002   309 raft_consensus.cc:1081] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:18:55.455365 28432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:53280
I20260709 12:18:55.456013 28432 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:55.456425 28432 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.457237 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:55.458057 28433 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:55.458923 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:55.459259 28437 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:55.461228   695 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:55.462119 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:55.421311   665 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.462742 28434 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:55.463990 28441 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:55.464326   665 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:55.465356 28440 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:55.467060 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:55.467968 28438 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:55.470853 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:55.471719 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:18:55.472003 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.472499 28982 raft_consensus.cc:2368] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12.
I20260709 12:18:55.472796 28436 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.474107 28330 leader_election.cc:400] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12.
I20260709 12:18:55.474422 28330 leader_election.cc:403] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:55.482503 28436 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
W20260709 12:18:55.470340 28174 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 114fa146124747358c997ac1008e1b93 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f526369c0d0743b192be6e205540a059 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:55.482136 28432 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.489662 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:55.501104 28361 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:18:55.503180 28987 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:55.505445 28432 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:55.508324 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:55.509910 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 804368us
W20260709 12:18:55.510617 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.513892 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:55.514997 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.515491 28361 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
W20260709 12:18:55.516458 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 644244us
I20260709 12:18:55.518250 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:55.493204   727 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.492990   695 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12.
I20260709 12:18:55.528241 28358 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 12:18:55.515460 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.520452 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:55.516866 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.530443 28360 raft_consensus.cc:2410] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
W20260709 12:18:55.522763 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.534039 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.523780 28983 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:55.545197 28983 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:55.551093 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:55.551836 28286 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:55.553035 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:55.553772 28291 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:55.557993 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:55.558956 28360 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:55.560285 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:55.561823 28287 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:18:55.560859 28983 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.563138 28983 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.544896   654 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:55.544652   648 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.515990 32578 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
W20260709 12:18:55.568418   774 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:18:55.526189   647 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:18:55.547479   698 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:55.574931 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44972.3us
W20260709 12:18:55.558911 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.575066   654 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:18:55.579923 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 61780.7us
I20260709 12:18:55.615043 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:55.535673   727 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:55.619822 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.625895 28983 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:55.649040 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:55.647306   698 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.637506   756 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:55.670236   756 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.577066   506 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:18:55.674955   698 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:55.678834   756 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:55.683820 28440 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:55.685966   538 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:55.686482 28434 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:18:55.687029   531 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:55.687606   531 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:55.688442 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:55.689524 28436 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:55.690721 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:55.691231 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:55.692394   627 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:55.693065   627 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.695182 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:55.695506   627 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:55.696497   709 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:55.639657   530 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:55.697482   530 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:55.688486   531 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.698534   530 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.705958 28440 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:55.617178   654 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.696985   709 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:55.713335   565 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:18:55.647024   728 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:55.729873   752 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:55.730356   752 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:55.730746   752 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:55.733945   385 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:55.735030   743 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.735984   756 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:55.736379   756 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:55.737274   743 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:55.739939   779 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:18:55.717624   766 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.742810   766 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:55.706377   445 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:55.580384   648 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:55.710420 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 52707.7us
I20260709 12:18:55.743574   445 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.744558 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:55.723711   685 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.740633 28438 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:55.747581   556 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:55.748064   556 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.748286 28440 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:18:55.748476   556 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.749594   648 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:55.729393   709 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.743451   385 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:18:55.751883   680 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:55.737210   756 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.753060   756 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.707777   521 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.754698   521 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.747799   685 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:55.746011 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:55.758612   688 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:55.759294   688 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:18:55.760763 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:55.768813 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:55.770452 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:55.752272   680 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:55.768623 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:55.795001 28358 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:55.804426 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:55.806213   771 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:55.807255   771 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:55.807471 28991 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:18:55.808971 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:55.809929   666 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:55.810359   666 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:55.811203   666 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.812481   666 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.814100   770 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:55.814790   770 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:55.815747   770 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.809475   765 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:55.818694   765 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:55.819072   765 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.824503 28437 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:55.827000 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:55.827832 28437 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:55.834108   750 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:55.841488 28437 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:55.850219 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:55.851014 28983 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:18:55.861168 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.865676 28361 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:55.751116   728 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.801210   565 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:55.776146   648 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:55.802103   720 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:55.860275   688 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.853951   445 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:55.875183   595 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.888837 28983 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
W20260709 12:18:55.920146 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.912253 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:55.929579 28358 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:18:55.902513   720 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:18:55.945490 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 330526us
I20260709 12:18:55.897919   648 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:55.950564 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:55.945209 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 332714us
I20260709 12:18:55.955550 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:55.902545   688 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:55.957955 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:55.923022   721 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:18:55.924268   595 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:55.978930 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.942515   745 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:55.987138 28361 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:55.979892 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:55.935213   458 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:18:55.960738   531 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:55.994261 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:55.995090 28361 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:55.995718 28358 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:18:55.992992 28367 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:55.992905 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:55.997356 28363 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:55.998688 28359 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:18:55.997961 28189 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 9 to 13, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 13 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:55.999722 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:56.000499 28364 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 19: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:55.986181   445 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:55.982818   783 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:56.004096   595 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.004869   720 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.006666   721 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.007339   721 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.009444   728 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:56.015987 28440 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:56.023885 28257 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:56.010299 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:56.000056 28189 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:56.030344 28257 leader_election.cc:400] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18.
I20260709 12:18:56.031531 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:56.035713 28437 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:56.042587 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.046020 28440 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:56.011051   688 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:56.009539   721 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:55.986393   625 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:18:56.052268 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:55.985371   506 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:18:56.029150   531 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:55.992187   698 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:18:56.029330   445 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:56.033187   647 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.012665   627 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:18:56.057278 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:56.065485 28189 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 6 to 9, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 9 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:56.046247   521 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:56.086462 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.087329 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.030687   752 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.040684   695 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
W20260709 12:18:56.098649 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 225567us
I20260709 12:18:56.070075   665 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:18:56.070374   788 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.093775   672 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:56.070637   666 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:18:56.062533   765 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.102588   785 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:18:56.114336 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 82927.7us
I20260709 12:18:56.102824   627 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:56.114951 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:56.115602 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:56.117467 28991 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 19: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:56.103928   648 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.119531 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:56.119679 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:56.121480 28366 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:18:56.123127 28361 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:18:56.089187   556 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.106822   665 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:56.110697   672 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.128743 28189 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 6 to 9, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 9 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:56.111541   728 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:56.111601   745 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:56.131801   648 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:18:56.116526   785 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:56.113808   783 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:56.143224 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.142694 28189 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:18:56.106583   788 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.150978   709 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.152105 28983 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:56.152925   788 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:56.153981   787 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.002s
I20260709 12:18:56.155563   538 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:56.158906   752 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:56.160082   665 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.161882   765 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.147552   728 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.125348   698 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:56.126339   506 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:56.165799 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:56.166661 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.160770 32578 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:56.158475   709 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:56.171447 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:56.178887 28982 raft_consensus.cc:2393] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 9: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:56.174113   728 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:56.174937   647 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:56.175926   556 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.130985   672 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.171550 28440 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:56.146430 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:56.191303 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:56.191634 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:56.194891   707 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:56.195523   707 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.195855   506 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:56.196350   506 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:56.196724   506 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:56.199481 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 53154.6us
I20260709 12:18:56.129877   654 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.205030 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.205873 28287 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:56.176915 32578 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:56.207300 28189 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 6, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:56.210393 28286 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:56.127036   445 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.215787 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.224012 28292 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:56.225991 28360 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:56.222887 28189 catalog_manager.cc:5697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:56.232066 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:18:56.247814 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:56.249428 28432 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:18:56.249892 28432 pending_rounds.cc:85] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 8
I20260709 12:18:56.250119 28432 pending_rounds.cc:107] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:18:56.250389 28432 raft_consensus.cc:2889] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:56.245675 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:56.232631 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:56.223449   783 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.237969 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.263998 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.264096 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:56.265156 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.264458 28189 catalog_manager.cc:5697] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 2 to 6, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 6 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:56.239092 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.239880 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.242920 28982 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:56.239197 28285 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:18:56.254123 28364 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:56.270327 28364 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:18:56.271593 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:56.272238 28982 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:18:56.273545 28285 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:56.274461 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.195905   707 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.276060 28286 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:56.278115 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.278662 28437 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:56.278748 28284 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:56.280164 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:56.270793 28288 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:56.282438 28290 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:18:56.282902 28290 pending_rounds.cc:85] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 6
I20260709 12:18:56.283133 28290 pending_rounds.cc:107] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:18:56.283381 28290 raft_consensus.cc:2889] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:56.258514 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.207212   766 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.206995   788 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:56.262516 28365 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:56.245024 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:56.260748   625 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:18:56.209604   784 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:18:56.261147   634 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:56.242379   666 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:56.284919   795 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 12:18:56.335757 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 52216.7us
I20260709 12:18:56.176592   785 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.277272   757 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:56.222630   799 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 12:18:56.340938 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:56.222985   531 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 12:18:56.322703   765 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:56.267370   695 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 12:18:56.267189   771 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:18:56.277963   750 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:18:56.345587   721 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.285140   797 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:18:56.346854   757 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:56.332150   766 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.342569   726 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 12:18:56.349848   648 consensus_queue.cc:1175] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 15 status { last_received { term: 13 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:56.347820   729 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:18:56.364080   728 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:56.348299   666 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.350582   788 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:18:56.346313   634 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:56.424093 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 178278us
W20260709 12:18:56.431969 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.431389 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.369933   721 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.487317   721 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:56.286329   745 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:56.522197 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 181204us
I20260709 12:18:56.524240   679 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:56.374724   727 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:56.523483 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.374941   757 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:56.386585   800 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 12:18:56.552435 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.388710   796 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:56.563539   796 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:56.386286   810 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:18:56.566288   810 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:56.567905   810 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.381706   654 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:56.478060 28189 catalog_manager.cc:5697] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:56.479930   816 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.381597   728 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.599181   728 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:56.490540 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:56.491125 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:56.606722 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 116503us
I20260709 12:18:56.608263   821 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:56.608968   821 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.609442   821 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.493777   771 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.610785   771 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.494382   634 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:56.620522   458 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:56.328889 32578 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.501083   808 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.501544 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.504838 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.501163 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.505894 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.507721 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.508795   666 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.387334   680 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.629793 28284 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:56.630730   680 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.523622 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:56.558538 28292 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:56.631125 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.632057 28283 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.570065 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.554116 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.583459   816 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.638250 28286 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:56.587448 28174 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 114fa146124747358c997ac1008e1b93 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f526369c0d0743b192be6e205540a059 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:18:56.639077 28174 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 51911.1us
I20260709 12:18:56.589826   470 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:56.589917 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.632632 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.631269 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.632675 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.641216 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.648581 28285 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 12: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:56.642407 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:56.621188   458 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:56.613648   771 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.640048   810 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:56.633209   798 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:18:56.663959 28362 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:18:56.663471   808 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.565572   796 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.667483 28287 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:56.675544   634 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.682039 28292 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.687441 28284 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:56.690474   816 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:56.692297   796 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.694582   757 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.696309   808 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.697113   757 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:56.707193 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 183804us
I20260709 12:18:56.708417 28404 leader_election.cc:400] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 11 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 11. Current term is 12.
W20260709 12:18:56.708653 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.711993 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.713028 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.715337 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:56.715803   506 consensus_meta.cc:287] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.012s
I20260709 12:18:56.716346   506 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.716641   707 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.717805 28291 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:56.689448   470 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.720578 28363 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:18:56.721050   666 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:18:56.721007 28363 pending_rounds.cc:85] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 6
I20260709 12:18:56.738590 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:56.739779   793 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:56.740334   793 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:56.742503 28283 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:56.746119 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:56.746951 28436 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.755116 28290 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:56.721206   470 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:56.729456 28363 pending_rounds.cc:107] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:18:56.764616 28363 raft_consensus.cc:2889] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:56.765810 28286 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:18:56.764115 32578 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:56.769227   819 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.186s	user 0.000s	sys 0.005s
I20260709 12:18:56.788730 28292 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:56.795202   821 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.797115   821 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.806982 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:56.807824 28361 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:56.812956 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 206403us
I20260709 12:18:56.818432 28361 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:18:56.836808   565 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.837596   565 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.839483   458 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:56.840276 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.842577   665 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:56.843322   665 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.844676 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:56.843201   770 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:56.846473   770 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.847467   688 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:56.848081   688 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:56.849045   770 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:56.849344   688 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.850811   785 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.851956   695 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.852700   695 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.855237   695 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:56.836621 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.866590 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:56.868966 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.870522 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.871248 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:56.872246 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.878499 28437 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 9.
I20260709 12:18:56.883008 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:56.884070 28358 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:56.884222 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 176043us
I20260709 12:18:56.886852 28434 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:56.887017 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:56.887972 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:56.888787 28991 raft_consensus.cc:2393] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:56.884075 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:56.893944 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:56.897346   506 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:56.898953   807 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:18:56.900874   824 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:18:56.883244 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.840237   565 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.908252 28436 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
W20260709 12:18:56.906214   625 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.566s	user 0.016s	sys 0.000s
I20260709 12:18:56.909534   625 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:18:56.912844   680 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.909392   757 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:18:56.914273 28438 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:18:56.918221   647 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:56.921294   672 consensus_meta.cc:287] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.720s	user 0.015s	sys 0.004s
I20260709 12:18:56.921927   458 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.923717   707 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.925446 28439 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:56.926246 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.864980   793 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.927026 28434 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:56.927537   793 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:56.934443 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47546.2us
I20260709 12:18:56.934837   634 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:56.936285   729 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.937067   729 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:56.939270 28440 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:56.939926   729 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:56.942312 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:56.944901 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47853.8us
I20260709 12:18:56.962944   634 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:56.963900 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:56.964586 28292 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:56.967736 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.972041   680 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:56.963742   634 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:56.975442 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.975719   766 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:56.976244 28435 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:56.950405   665 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:56.978632 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:56.989171 28440 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:18:56.989697 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:56.990243 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:56.990417 28439 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:57.000270   385 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:57.000689 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:57.000319 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:57.002898 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:57.003069   827 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.003788   827 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.004406   757 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:57.005057   757 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:57.002930 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.006438   757 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.007280 28360 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:57.003931 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:57.002219 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.013640 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.017189 28438 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:57.016987 28440 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:57.017351 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:57.029294 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.009682   625 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:57.030270   625 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:57.031333   625 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.035053 28438 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:18:57.041219 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.043035 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:57.044514 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:57.044514   665 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:57.045843   665 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.046195   665 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.047626 28364 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:56.921942   672 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.050590   672 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.051607 28983 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:57.024636   729 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.053296   729 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:57.054450 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.054996   756 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:57.055675   756 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:57.056073   729 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:57.056597   770 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:18:57.057101   770 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.056718   756 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.057467   770 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:57.060285 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:57.064937 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:57.002712   698 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:18:57.136075 28358 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
W20260709 12:18:57.140708 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.143566 28289 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:18:57.156983 28437 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:56.984862   771 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:18:57.181501 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:57.155618   647 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
W20260709 12:18:57.207669 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49476us
I20260709 12:18:57.168275   766 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:57.209947 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 145104us
I20260709 12:18:57.230777 28367 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:18:57.171962 32578 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:18:57.236423   827 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.216050   766 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:57.248011 28292 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:57.219492   680 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.214737   821 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.250172   695 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:57.278257   728 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.678s	user 0.003s	sys 0.012s
W20260709 12:18:57.339437 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 151046us
I20260709 12:18:57.340682 28440 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:57.341645   784 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:57.320015   821 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.277251   766 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.374687 28438 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:57.375972 28434 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:18:57.249037   729 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:57.385298 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 109384us
W20260709 12:18:57.387454 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 228858us
W20260709 12:18:57.381371   785 consensus_meta.cc:287] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.002s
I20260709 12:18:57.344661   672 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:57.408363 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 382030us
W20260709 12:18:57.410758 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.380118   784 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:57.425374 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:57.436069 28292 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:18:57.437566 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:57.438014 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:57.383783   680 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.386689   728 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.450549 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:57.452306 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:57.453912 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.457799   792 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:57.458360   729 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.458972   729 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:57.461010   458 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.000s
I20260709 12:18:57.461355   729 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:57.461933   796 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.464211 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.464723   680 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.467372 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2360) took 1226 ms. Trace:
W20260709 12:18:57.469880   802 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:18:57.495729   766 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.497195   788 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:57.497622   788 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:57.498553   766 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.498725   788 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.500159   788 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.502517 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:57.453467 28285 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
W20260709 12:18:57.474673 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49446.5us
I20260709 12:18:57.475054   445 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:57.506309   445 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:57.507411   445 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.485127 28434 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:57.386448   695 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.494208 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:57.515921 28432 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.488374 28289 rpcz_store.cc:276] 0709 12:18:56.240611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.264925 (+ 24314us) service_pool.cc:224] Handling call
0709 12:18:57.467140 (+1202215us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54003daed4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:18:57.467292 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482935680}
I20260709 12:18:57.503813 28285 pending_rounds.cc:85] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 7
I20260709 12:18:57.522615 28285 pending_rounds.cc:107] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:18:57.522919 28285 raft_consensus.cc:2889] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:57.528045 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:57.529999 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.530853 28440 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:18:57.535079 28433 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:57.544497 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:57.546574 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:57.549609 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:57.550334 28434 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.551358 28438 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 11: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:57.554639 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:57.445354   672 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:57.560113 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.514679   796 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:57.569906 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 135775us
I20260709 12:18:57.576220   595 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.576668   672 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.576934   595 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:57.584095   665 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.537s	user 0.012s	sys 0.000s
I20260709 12:18:57.584688   665 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.585561   796 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:57.585940 28358 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:18:57.558983   792 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:57.562466 32578 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.575999   821 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:57.575846   837 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
I20260709 12:18:57.597910 28436 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:57.623963 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:57.610751 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:57.611507   784 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.625567   784 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.626470 28956 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:57.627656 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:57.628914   826 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:57.610489 28991 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:18:57.613456 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:57.614961   792 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:57.616742 28432 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:57.618566   665 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.585258   695 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.621975   595 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:57.622799   728 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:57.623140 28982 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:18:57.623289 32578 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.631533 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:18:57.636372 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 99218.3us
I20260709 12:18:57.629361   826 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:57.638538 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:57.638584 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 201128us
I20260709 12:18:57.634265   688 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:18:57.640328   826 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:57.640743   796 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:57.649968   595 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:57.647910   688 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.704056 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:57.706334 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:57.707414 28288 raft_consensus.cc:2410] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 20 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:18:57.688903   688 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:18:57.713935 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 100640us
I20260709 12:18:57.715500 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:57.717260 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:57.719136   793 consensus_meta.cc:287] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.791s	user 0.013s	sys 0.001s
I20260709 12:18:57.719522   595 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:57.778791 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 61671us
I20260709 12:18:57.684373   698 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:18:57.731657   841 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.385s	user 0.002s	sys 0.000s
I20260709 12:18:57.733909   792 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.782604 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.690269   796 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:57.786469 28439 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:57.786684   796 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.788350 28437 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:57.738262 28283 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
W20260709 12:18:57.714013   849 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.084s	user 0.002s	sys 0.000s
W20260709 12:18:57.743556 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.789665 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.791570 28433 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:18:57.745625   843 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.365s	user 0.000s	sys 0.001s
I20260709 12:18:57.709208   556 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.798419   556 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.799239 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2609) took 1064 ms. Trace:
I20260709 12:18:57.799702 28441 rpcz_store.cc:276] 0709 12:18:56.734983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.761277 (+ 26294us) service_pool.cc:224] Handling call
0709 12:18:56.798251 (+ 36974us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:56.940657 (+142406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.940692 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.941371 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.956881 (+ 15510us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:56.956929 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.957014 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:57.744988 (+787974us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:18:57.798808 (+ 53820us) raft_consensus.cc:1685] finished
0709 12:18:57.798833 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.799057 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22364,"raft.queue_time_us":23918,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":15022,"spinlock_wait_cycles":21020544,"thread_start_us":15031,"threads_started":1,"wal-append.queue_time_us":278061}
W20260709 12:18:57.757993 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 73217.3us
W20260709 12:18:57.762544 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 228222us
I20260709 12:18:57.801928   556 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.803479   666 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:57.804644   666 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:18:57.762400 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 233635us
I20260709 12:18:57.806403   685 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:57.806588 28988 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:57.806820   685 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:57.807920   685 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.809264   685 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.762228 32578 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.806432 28332 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:57.810304   728 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:57.811134   765 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:57.811287 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:57.812556   666 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:18:57.814100   783 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:57.814761   783 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.816458   765 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:18:57.816918   765 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.817214   783 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.818185 32578 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:57.819314   853 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:57.768894   625 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.820225   625 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.821906   784 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:18:57.822876   770 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.765s	user 0.008s	sys 0.005s
I20260709 12:18:57.771822   634 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:18:57.773164   836 log.cc:927] Time spent T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.427s	user 0.000s	sys 0.002s
W20260709 12:18:57.777469 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.822696   625 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.726954   720 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:57.779072   595 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.780223   729 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:57.826684 28364 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:57.827317 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:57.822934   784 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.765720 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:57.823553   458 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:57.829698 28257 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:57.829969   458 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:57.805181   757 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:18:57.827464   625 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:57.769600 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2246) took 1266 ms. Trace:
I20260709 12:18:57.831404   625 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.832217 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:57.832849 28358 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:18:57.832899 28359 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:18:57.830838 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:57.830909   458 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.834451 28366 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:18:57.835098   458 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:57.823660   853 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.838009 28290 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.838523 28360 raft_consensus.cc:2410] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:57.839745 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:57.840353 28441 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:57.832031 28285 rpcz_store.cc:276] 0709 12:18:56.503313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.650395 (+147082us) service_pool.cc:224] Handling call
0709 12:18:56.656347 (+  5952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.558515 (+902168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.769303 (+210788us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54003d9fd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:18:57.769496 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749802368}
I20260709 12:18:57.842501 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.842751   784 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.843350 28437 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:57.843935 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:57.844782   770 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.846060 28363 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:18:57.846928 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:57.819043 32578 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.847807   757 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:57.848292 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:57.849090   757 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:57.833633   625 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.851308   688 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.812460 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:57.852931   729 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:57.854700 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:57.856361   729 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:57.857460   757 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:18:57.858219   666 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.858150   757 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.859159   666 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.860947   757 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.861413   666 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.862426   698 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:57.863981   647 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:57.864414   783 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.864360   647 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:57.864786 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:57.853058 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.865291   783 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.856385 32578 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.865722   647 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:57.865489   625 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.854573   853 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.867240   647 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.867786   783 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.869364 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:57.871142 32578 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:57.871971 32578 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:57.872290   445 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:18:57.873836 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44938us
I20260709 12:18:57.933235 28441 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
W20260709 12:18:57.878170   756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.002s	sys 0.002s
I20260709 12:18:57.879837   688 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:57.881135   810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.004s	sys 0.000s
I20260709 12:18:57.882339 28363 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:57.861879 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.888603 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:57.892444 28434 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
W20260709 12:18:57.895291 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:57.898864 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49399.1us
I20260709 12:18:57.958407 28433 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:57.857883 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:57.938758 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:57.848739   770 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:57.957564 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:18:57.935741   756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.002s	sys 0.002s
I20260709 12:18:57.963032 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:57.971935 28437 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:57.959398 28362 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:18:57.965831 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:57.966418 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:57.967957 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.973345 28286 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:18:57.964277 28438 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:57.967024 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:18:57.965674   870 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 12:18:57.957830   810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.005s	sys 0.000s
I20260709 12:18:57.975004 28287 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:18:57.979913 28285 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:18:57.981531 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:18:57.984557 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:57.987105 28360 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:57.993240 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:57.993606 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:18:57.979491   869 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:18:57.984181   720 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:57.988047   695 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.002526   695 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:58.004922   695 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:18:58.010335 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 152560us
I20260709 12:18:57.867223   625 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:57.997619   756 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:18:58.020489   871 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.091s	user 0.000s	sys 0.006s
I20260709 12:18:58.036839 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:58.041474 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:58.042117 28437 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:58.057204 28433 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:18:58.071730 28286 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:18:58.073810 28290 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:58.074518 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:58.033242   793 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.081234 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:58.082386 28435 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:58.016072   859 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 12:18:58.087366 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 54326us
I20260709 12:18:58.086283 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:58.025060   826 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:58.076033   688 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.114210 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:58.118003 28433 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 12:18:58.122363   757 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.123504   858 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:18:58.123210   757 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:58.124512   858 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:18:58.125026   858 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:18:58.126024 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 147554us
I20260709 12:18:58.126921 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:58.127642 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:58.127878   698 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:58.128389   757 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:58.129066 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:58.129310   757 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:58.129686   757 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.130064   757 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.127696 28991 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:58.133945   666 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:58.134503   666 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:58.135313   685 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.135527   666 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.137055   665 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:58.140727   698 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:58.141142   698 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:18:58.142423   879 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:18:58.142196   698 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.143524   698 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:58.145973 28332 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:58.086649   793 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:58.147423   695 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:58.147822   695 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.148147   695 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:58.148675   793 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.149196 28332 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:58.149852 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 133214us
I20260709 12:18:58.151225   685 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.151892 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:58.142472   757 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.150220   385 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:58.153296   385 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.153668   385 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:58.127637 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:58.154019   783 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:18:58.163430   783 consensus_queue.cc:1243] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 9aea2391841b4888b4e195e5a1f4623b log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:18:58.154220   784 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:58.165259 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:58.153715   685 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.166841 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:58.166955 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:58.172838   872 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.003s
I20260709 12:18:58.173754 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:58.182088 28440 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:58.174024   869 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:18:58.174147 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:58.174749 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:58.175812   858 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:58.153313 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:18:58.161345 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:58.188385 28438 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:18:58.191506 28437 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:18:58.193277 28434 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:18:58.202461 28988 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:18:58.203058 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 173842us
W20260709 12:18:58.204700 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.209954 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 150860us
I20260709 12:18:58.160928   788 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:58.155813   757 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.222452 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:58.160418   625 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:58.220564 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:58.194585   665 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:58.224339 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 63077us
I20260709 12:18:58.229357   793 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:58.229846   793 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.230254   793 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:58.230924 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63908.1us
W20260709 12:18:58.232126 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.235340 28432 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 16: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:18:58.236159 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.237061 28360 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:18:58.160959   765 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.123189   688 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:58.259265 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 106041us
I20260709 12:18:58.269292 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:18:58.222656   788 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:58.232237   826 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.235337   665 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.261188 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:58.259505 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:58.271050   648 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:18:58.290813 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:18:58.292668 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 118064us
I20260709 12:18:58.236920   784 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:58.304414 28364 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:58.305305 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:18:58.310361 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 50949us
I20260709 12:18:58.240754   859 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:18:58.311450 28991 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:18:58.314978 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:58.294499   788 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:58.293429   665 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:58.285991   864 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:58.294026   647 raft_consensus.cc:2764] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 12:18:58.335803 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45115.2us
I20260709 12:18:58.312274   648 consensus_queue.cc:1243] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:58.310672   756 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.288540   385 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.320348   808 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:58.351832 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.332147   864 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.341120   445 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.374771   648 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:58.375667 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:58.366526   864 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:58.377699   647 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:58.380038   647 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:58.350266   756 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.360668   808 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:58.384227   808 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.384898   688 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.355424   783 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:58.386008   783 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:58.386428   783 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.387101   385 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.361837   784 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.026s
W20260709 12:18:58.389447 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.385497   756 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.365106 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.402356 28363 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:18:58.403404 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:58.404055 28436 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:58.390295   695 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.405864   445 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.407610   695 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:58.391633 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 86413.5us
I20260709 12:18:58.408124   445 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:58.411057   873 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.219s	user 0.004s	sys 0.000s
I20260709 12:18:58.346799 32578 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:58.405864 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:58.413182 28437 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:58.392812   634 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:58.394306   765 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.415795 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:58.422520 28440 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:58.420284 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:58.423429 28439 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:58.419956 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:58.425956 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.426753 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.403301   810 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:18:58.419464   385 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 12:18:58.442081 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.420271 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:58.417918   634 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:58.444413 28367 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:58.455120   634 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.456542   796 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.457147   796 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.435895   695 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:58.459513   796 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:58.460729   729 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.461324   729 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.462466   765 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:18:58.463094   765 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:58.463649   729 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:58.464039   765 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.426386   720 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:58.396271   793 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:58.433316   810 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:58.483541 28436 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:58.489022   793 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.490453   729 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:58.498478   810 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.504666 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:58.507571 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:58.482574   728 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.482856   796 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.478047   695 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:58.518105 28332 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:58.519685 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.469525   634 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.542140 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.542923 28360 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:18:58.544262 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:58.544623 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:58.545173 28292 raft_consensus.cc:2410] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:18:58.545209 28437 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:58.527871 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:58.547986 28285 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:18:58.551041 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.553113 28435 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:18:58.553509 28435 pending_rounds.cc:85] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 9
I20260709 12:18:58.553756 28435 pending_rounds.cc:107] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:18:58.553988 28435 raft_consensus.cc:2889] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:58.555562 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:58.555792 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:58.556490 28434 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:58.556243 28286 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.551774 28360 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:18:58.552029 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:58.556536 28441 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:58.517629   685 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:58.560338 28364 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:18:58.559962 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.524106   757 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:58.566888 28366 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:58.548691   595 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.527761   810 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:18:58.578341 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:58.547521   784 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.579087 28440 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:18:58.584966 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 209385us
I20260709 12:18:58.581656 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:58.518515   728 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.586524 28983 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:18:58.567523   458 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:58.592891 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:58.525180   796 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.591946 28437 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:18:58.600636 28991 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:58.604632 28362 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:58.607090 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:58.550863 32578 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 12:18:58.613924 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.620640 28286 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:18:58.583352   685 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:58.622542 28987 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:58.586567   750 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:58.592849   884 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:58.659020 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 86346.6us
I20260709 12:18:58.659929 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:58.660825   858 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:58.582239   864 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.623639   506 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:18:58.661278   858 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.661821 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:18:58.665932 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 79745.7us
I20260709 12:18:58.621508   445 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:18:58.594239   784 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.673102 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.589398   595 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:58.678110 28988 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:58.629249   750 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:58.685281 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 108793us
I20260709 12:18:58.616410   698 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:18:58.672277   858 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:58.696810   891 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:18:58.688520 32578 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:18:58.693441 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:58.699879 28364 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 12:18:58.700884   894 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:18:58.697394   783 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.704651   783 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.698980   796 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:58.628899   792 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 12:18:58.692209 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 120492us
I20260709 12:18:58.669541   757 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:58.623430   695 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.681706   788 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:58.722337 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:58.715991 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.757499 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.760443 28364 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:58.831985 28439 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:18:58.689118   750 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:58.887238   910 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.131s
I20260709 12:18:58.858675   784 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:58.850351   899 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:18:58.715556 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:58.713856 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:18:59.024000 28332 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:58.740351 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 56401.8us
I20260709 12:18:58.683173   895 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:58.740063 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:18:59.027876 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 305584us
I20260709 12:18:58.714140   906 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.036340   784 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:58.844142   595 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.947664   458 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:59.031659 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 317918us
I20260709 12:18:58.714139   883 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.340s
I20260709 12:18:58.871325   852 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.938467   634 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:58.970165   647 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:58.971097   853 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.034862 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:58.719156   864 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:58.879547   728 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:58.996233   897 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.835013   665 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:58.831290 28956 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:18:59.042325 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.054649   910 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:59.042634   793 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:18:59.050542   902 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:59.050985 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.079448   770 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:59.079310   625 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.051394 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.050269   729 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:18:59.051013 28435 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:18:59.034641 28983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "114fa146124747358c997ac1008e1b93"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:18:59.053485 28982 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:18:59.034466   898 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:58.773705   808 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.064043 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:59.074849 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:18:59.076009 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.075801 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:59.036808   784 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.075798 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.077987 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:59.076546 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:59.035534   895 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:59.076498 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:59.083523 28363 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:59.084378 28292 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:18:59.091975   901 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:18:58.858170   858 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.095500 28437 raft_consensus.cc:2368] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15.
I20260709 12:18:59.032444 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:59.095016 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 263844us
I20260709 12:18:59.097244 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.105242 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.106211 28287 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:59.091991   810 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:59.111768 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.167340 28436 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:18:59.119189 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.120589 28440 raft_consensus.cc:2368] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:18:59.051193   720 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:59.076766   647 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:18:59.180750   858 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.181442   647 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.182366 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.183027 28283 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:59.092321   852 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.183946 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.126513 28358 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:59.127157 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.128710 28983 raft_consensus.cc:606] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:18:59.130693 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.131763 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.134451 28435 pending_rounds.cc:85] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 7
I20260709 12:18:58.780815   685 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.142068 28289 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:18:59.129902   874 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:18:59.046042   458 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:59.103852   910 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:18:59.145008 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 70223.8us
I20260709 12:18:59.195570 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.136986 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:18:59.147109 28434 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:59.164634 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:59.196412 28286 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:18:59.084089   853 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.146548   796 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:59.168931 28439 raft_consensus.cc:2368] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14.
W20260709 12:18:59.173928 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 97467.4us
I20260709 12:18:59.174645   770 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:59.218605   770 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.101799   897 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.178169 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.178826 28433 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:18:59.178504   720 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.167580   506 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:59.180372   665 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:59.136937   826 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:59.182541   647 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.193926 28289 pending_rounds.cc:85] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 8
I20260709 12:18:59.247879 28289 pending_rounds.cc:107] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:18:59.248179 28289 raft_consensus.cc:2889] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:59.194278 28435 pending_rounds.cc:107] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:18:59.144263   784 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.196182 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:59.196753 28437 raft_consensus.cc:2368] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:18:59.165917 28285 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:18:59.255779 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.196930 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:18:59.197229 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 60304.2us
I20260709 12:18:59.197926 28290 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:18:59.042120   906 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.201414   910 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.204728   898 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:59.205515   728 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:59.149211   895 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.373539 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:59.209028   765 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:59.213378 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.376418 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.376888 28439 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
W20260709 12:18:59.227564 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.117779   757 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.239563   897 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.240231 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:59.379096 28982 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.242022 28365 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:18:59.138288   625 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.185470   852 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.248739 28435 raft_consensus.cc:2889] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:59.211910   904 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:59.252069 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.181397   810 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.356954 28291 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:59.359439 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:59.373000 28364 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:18:59.393688 28285 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.384193 28439 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:18:59.385188 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.386824 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:18:59.377807   884 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.705s	user 0.008s	sys 0.004s
W20260709 12:18:59.245559   916 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:18:59.358688   788 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:59.414144 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 292480us
I20260709 12:18:59.408464 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:59.245306   506 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.388063   904 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.380259   896 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:59.392274   895 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:59.426613 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.409413   902 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.434496 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.415403   884 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:59.417348   765 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:59.451088   765 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.456133   625 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:59.458582   720 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:59.459079   906 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.424443   904 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:59.437762 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 60359.7us
I20260709 12:18:59.440735   506 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:59.446305   922 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:59.447504 28285 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:59.438634 28291 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:18:59.434427   910 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.473237 28291 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:18:59.479334 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.435530   896 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.541545 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.542610 28432 raft_consensus.cc:2410] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:18:59.470847   729 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:59.527388 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:59.547079 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:59.486393   625 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:18:59.541633   698 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.474586   897 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.076s
W20260709 12:18:59.555716 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.466917   902 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:59.558498 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.573272 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:18:59.576349 28432 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:18:59.552520   729 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.581811 28332 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:59.586740 28438 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13. Ops: [11.6-12.7]
I20260709 12:18:59.586910 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.589530 28287 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:59.590580   920 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:59.591041   920 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.591441   920 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.594005   919 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:59.563823   853 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.600018   784 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.607111 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:59.618898 28291 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:59.602675   920 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.562381   698 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:59.632850 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.634481 28982 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:59.640532 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:59.563987   625 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.588850   928 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.651198 28283 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:59.648705 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.652042 28291 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:59.605672   458 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.660645   896 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.663053   902 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:18:59.664260   750 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.634s	user 0.012s	sys 0.000s
I20260709 12:18:59.664738   750 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.667060   750 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.670032 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.670975 28286 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:59.671948 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.670639 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:59.673209 28289 raft_consensus.cc:2410] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:18:59.673049 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.670133 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:59.674860 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:59.675108   729 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:18:59.675498 28432 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:59.676926   784 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.638849   647 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:18:59.677965   647 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.680119   647 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.685273   698 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:59.660807   458 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:59.691358   625 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:18:59.697360 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:59.718641 28438 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.722867   826 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:59.724056   826 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:18:59.718321 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 654331us
I20260709 12:18:59.728464 28438 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:18:59.729997 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:18:59.731736 28438 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:18:59.686717   928 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:18:59.672176   920 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.688793   810 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:18:59.752748 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.753888 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:59.747418 28332 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:18:59.758101 28438 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:18:59.760023 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:18:59.761402 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 195656us
I20260709 12:18:59.762084 28432 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:18:59.762707 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:18:59.771968 28287 raft_consensus.cc:2410] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:18:59.771227 28367 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:18:59.769685 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:59.718721   827 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:59.780993 28289 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:18:59.773295 28438 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:18:59.783119 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.799929 28361 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:18:59.765990   910 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:59.803905 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:18:59.769351   458 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:59.809788 28991 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:18:59.801652 28330 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.813s (SENT) [suppressed 1 similar messages]
I20260709 12:18:59.818091 28330 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:59.822657 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:59.823448 28291 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:59.806773   765 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:18:59.806725   827 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:18:59.825237 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:59.836560   906 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.861706 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:18:59.865360 28291 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:18:59.869865   920 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:59.870460   920 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:59.871495   920 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.846237   458 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:59.874433   458 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:18:59.884745   940 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:18:59.893401 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:59.895326 28291 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.869318   906 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.911406   827 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.916851   901 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:59.917378   901 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:18:59.922251 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.923040 28289 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
W20260709 12:18:59.923647 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.918356   901 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:18:59.937467 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:18:59.940845   895 consensus_meta.cc:287] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.000s
I20260709 12:18:59.959715 28288 raft_consensus.cc:2410] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:18:59.960763 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:18:59.961910 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:59.962713 28291 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:18:59.963647 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:59.972837 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:59.974673   910 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:18:59.978471 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:18:59.986613 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:18:59.987679 28289 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.989457   445 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:18:59.999034 28257 leader_election.cc:400] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15.
I20260709 12:19:00.000783 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:59.974694   698 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:00.002991 28257 leader_election.cc:400] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:19:00.005424 28257 leader_election.cc:400] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14.
I20260709 12:19:00.005930 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:18:59.995465   901 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:59.987720 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 169716us
I20260709 12:18:59.986575 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:00.010313   910 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:00.030635 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.032033   933 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.032680   933 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:00.034143 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:00.034915 28360 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:00.038602 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:00.039932 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:00.040397 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:00.041097 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:00.041885 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:00.042498 28292 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.041095 28283 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:00.049475 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:00.012037   904 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.544s	user 0.012s	sys 0.000s
I20260709 12:19:00.055307 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:00.035141   933 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:00.056411 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:00.056438 28283 raft_consensus.cc:2410] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:00.005029   941 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:00.047497 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:00.086512 28289 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:00.087113 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:00.087857 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:00.071617 28291 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:00.072134 28292 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:00.011443   625 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:00.093021 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:00.094005 28283 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:00.097096 28292 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:00.102788 28283 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:19:00.103490 28283 pending_rounds.cc:85] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 9
I20260709 12:19:00.103845 28283 pending_rounds.cc:107] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:19:00.104336 28283 raft_consensus.cc:2889] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:00.104391 28360 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
W20260709 12:19:00.106483 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.104600 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:00.109112 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:00.108234 28291 raft_consensus.cc:2376] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13. Re-sending same reply.
I20260709 12:19:00.120175 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:00.122920 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.054775   688 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:19:00.095376   792 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:00.099215   884 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:00.098140   634 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:00.107174   729 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:00.126662 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:00.124768 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:00.118304   757 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:00.085134   904 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.094848   950 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:19:00.143993   458 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:00.146268 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:00.147663   458 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:00.149171   947 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:00.122007   445 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:00.150194   445 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.151018 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:00.152429   458 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:00.152915   445 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:00.129825   756 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:00.153978   756 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:00.154417   756 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:00.151587   954 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:19:00.139721   934 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:00.099524   625 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:00.129637   884 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.132351   895 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:00.119441   858 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:19:00.164311 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:00.130546   698 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:00.139104 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:00.175319 28286 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:00.158946   458 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:00.168874   884 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:00.173836   949 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.132968   634 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.135627   729 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.179298   808 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:00.262588   729 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:00.267429 28284 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 12:19:00.268021 28284 pending_rounds.cc:85] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 6
I20260709 12:19:00.268289 28284 pending_rounds.cc:107] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:19:00.268553 28284 raft_consensus.cc:2889] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:00.207042 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:00.169950   770 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:19:00.210455   910 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:00.218947   904 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:00.221987 28285 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:19:00.224669   949 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:00.230324   953 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:19:00.281538   808 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.282295   808 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:00.284926   808 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:00.160476   757 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.246397   458 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:00.181200   934 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:00.201875   698 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.250825   634 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:00.235208   625 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:00.294104 28435 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:19:00.296422 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.297163 28439 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
W20260709 12:19:00.307582 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 100627us
W20260709 12:19:00.314554 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 135637us
I20260709 12:19:00.287106   757 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:00.342941 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:00.359625   756 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:00.367576 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 50795.9us
I20260709 12:19:00.344492 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:00.373217 28366 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:00.325136   941 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:00.379465 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2253) took 1961 ms. Trace:
I20260709 12:19:00.381286 28440 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:00.381788 28440 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:00.329501   935 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:00.383489 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:00.380177 28358 rpcz_store.cc:276] 0709 12:18:58.410666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.418633 (+  7967us) service_pool.cc:224] Handling call
0709 12:18:58.418967 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:59.186076 (+767109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.371300 (+1185224us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b540007a154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:00.371486 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1626789248}
I20260709 12:19:00.336762   919 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:00.384351 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:00.414685 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:00.396835 28440 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:00.404631 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:00.417522 28440 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.420332 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:00.413558 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:00.422152 28991 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 16: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:19:00.425004 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 52015.1us
I20260709 12:19:00.430397 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:00.388105   941 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:19:00.435295   945 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.528s	user 0.004s	sys 0.000s
W20260709 12:19:00.442298 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.446362 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:00.448891   852 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:00.449410   852 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.449908   915 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:00.449810   852 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:00.548400 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.547060 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:00.397135   919 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:00.563957 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:00.545967   920 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:19:00.453444   884 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.566807   884 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:00.568456   895 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:00.397315   935 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.583915   895 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:00.604053 28440 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:19:00.604557 28440 pending_rounds.cc:85] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 8
I20260709 12:19:00.604805 28440 pending_rounds.cc:107] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:19:00.605052 28440 raft_consensus.cc:2889] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:00.594002   698 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:00.630854 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 60240.9us
I20260709 12:19:00.635712 28362 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 12:19:00.637560 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 85410.9us
I20260709 12:19:00.643002 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2267) took 1921 ms. Trace:
W20260709 12:19:00.668701 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.643450 28365 rpcz_store.cc:276] 0709 12:18:58.714470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.715076 (+   606us) service_pool.cc:224] Handling call
0709 12:19:00.635267 (+1920191us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54003babd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:00.635412 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489015168}
I20260709 12:19:00.682585   756 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:00.684306 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:00.686254   941 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:00.685665 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:00.690634 28291 raft_consensus.cc:2376] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13. Re-sending same reply.
I20260709 12:19:00.691246 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:00.691495 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:00.692131 28291 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:00.709199 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 52721.8us
I20260709 12:19:00.715049 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:00.715911 28982 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:19:00.716663   935 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:00.717177 28986 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: [10.10-12.11]
I20260709 12:19:00.703379   729 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:00.721302   729 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:00.722786 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.803385 28291 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:19:00.695211   864 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:00.730558   634 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.800915   934 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.806488 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:00.800989   852 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.723608   729 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:00.823694 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:00.837992 28291 raft_consensus.cc:2376] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14. Re-sending same reply.
W20260709 12:19:00.843603 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 112960us
I20260709 12:19:00.847379 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:00.850944 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:00.855806 28291 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
W20260709 12:19:00.856547 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.820130   974 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:00.875406   974 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.862771 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.863893 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:00.864997 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.866081 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.867178 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.867777 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.867859 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:00.831871   634 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:00.869629   919 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:00.853047   965 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 12:19:00.871429   976 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:19:00.871922 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.831902   934 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:00.847749   729 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.902043 28438 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:00.885419 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:00.884852 28435 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:00.887674 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.889680 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.908608 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:00.909543 28439 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:00.918002 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:00.920120 28436 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:00.877715   974 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:00.922374 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:19:00.921576 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 208840us
I20260709 12:19:00.925881 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:00.929373 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:00.930336 28435 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:00.869004   864 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:01.004858 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 73074.6us
W20260709 12:19:01.004954 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 145949us
I20260709 12:19:01.009187 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:01.013875 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:01.014848 28435 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:01.015244 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:01.016077 28437 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:00.919734   729 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.017689 28291 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
W20260709 12:19:01.018625 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 159136us
I20260709 12:19:01.004484   625 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:01.023277 28437 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:01.019364 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:19:01.025946 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 140627us
I20260709 12:19:01.027915 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:01.033674 28440 raft_consensus.cc:2368] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14.
W20260709 12:19:01.026880   978 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 12:19:01.012724   992 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:01.031935   634 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:01.040824 28438 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:01.030033   874 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:01.077425 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49620us
I20260709 12:19:01.078145 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:01.078275 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:01.078765 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.079175 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:01.078800 28441 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.080977 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.084079 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.074455   625 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.075591   904 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.103892 28441 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:19:01.045571   992 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.083841   874 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:01.081169   997 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:01.087283   808 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:01.086928   895 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:01.112865 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:01.144114 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51504us
I20260709 12:19:01.114836   897 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:01.143038   852 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:01.115374   998 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:01.113794   634 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:01.161867 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 82758.6us
I20260709 12:19:01.109368   995 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:19:01.110353   904 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.167325 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:01.168629 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.168947 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:01.169524 28285 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:19:01.172945 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:01.175832 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.176537 28285 raft_consensus.cc:2376] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14. Re-sending same reply.
I20260709 12:19:01.177474 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:01.178051 28285 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:01.151041   999 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:01.162097   895 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:01.108599   625 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:01.183650 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45520.7us
I20260709 12:19:01.188809 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.195544 28285 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:01.198904 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.204613 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:01.178889   864 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:01.165679   634 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:01.163805   808 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:19:01.212955 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:01.186336   934 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:01.215157 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.300313 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.410168 28285 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:19:01.296694   906 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:01.280030 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2414) took 1221 ms. Trace:
W20260709 12:19:01.260018   990 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:19:01.262912   625 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:01.414849 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 241946us
I20260709 12:19:01.291771 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2356) took 2178 ms. Trace:
I20260709 12:19:01.269436  1001 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:01.336652 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.173532   874 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:01.259409   943 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.373s	user 0.006s	sys 0.003s
I20260709 12:19:01.308259 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.271234   904 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:01.200471   998 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:01.205901   897 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:01.201489   784 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:01.218897   895 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:01.260767   993 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:19:01.441882 28359 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:19:01.443936 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.421262  1001 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:01.445045 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.419911   902 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:01.252529   991 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 12:19:01.217882   634 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.352267 28436 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:19:01.309043   972 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:01.331209   750 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:01.340530 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 779) took 1131 ms. Trace:
I20260709 12:19:01.258796   992 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:01.330520 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 639322us
I20260709 12:19:01.315177 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.481451 28286 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:01.247816  1000 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:01.410260 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63881.6us
I20260709 12:19:01.203475   729 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:01.411623 28291 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:01.418669 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.485976 28285 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:01.445776  1001 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:01.426584 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 213735us
I20260709 12:19:01.417539 28433 rpcz_store.cc:276] 0709 12:18:59.113404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.252781 (+139377us) service_pool.cc:224] Handling call
0709 12:18:59.358951 (+106170us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:59.529027 (+170076us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:59.919674 (+390647us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.990397 (+ 70723us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.994448 (+  4051us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:59.995036 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.999334 (+  4298us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:59.999598 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.999816 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.999895 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.285628 (+1285733us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b18003fbe60. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:19:01.291477 (+  5849us) raft_consensus.cc:1685] finished
0709 12:19:01.291504 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.291660 (+   156us) inbound_call.cc:177] Queueing success response
0709 12:19:01.327392 (+ 35732us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b18003fbe60. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a522 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
Metrics: {"mutex_wait_us":410357,"prepare.queue_time_us":441249,"prepare.run_cpu_time_us":179,"prepare.run_wall_time_us":4361,"raft.queue_time_us":34382,"raft.run_cpu_time_us":504,"raft.run_wall_time_us":29989,"spinlock_wait_cycles":52046976,"thread_start_us":204732,"threads_started":2,"wal-append.queue_time_us":102928}
I20260709 12:19:01.425828   934 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.427459   808 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:01.445873 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.428136   864 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:01.455094 28283 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:01.492816 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2422) took 1050 ms. Trace:
I20260709 12:19:01.457788   988 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:01.457916   928 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:01.464461 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47106.2us
I20260709 12:19:01.436939   895 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.470638 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:01.467545 28436 pending_rounds.cc:85] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 10
I20260709 12:19:01.499794 28436 pending_rounds.cc:107] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:19:01.423435   999 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:19:01.452803   902 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:01.453863   904 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.487521 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2429) took 1043 ms. Trace:
I20260709 12:19:01.411118 28284 rpcz_store.cc:276] 0709 12:19:00.058536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.170032 (+111496us) service_pool.cc:224] Handling call
0709 12:19:00.266956 (+ 96924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.274112 (+  7156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.279780 (+1005668us) spinlock_profiling.cc:243] Waited 96.5 ms on lock 0x7b54003f9650. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:01.279950 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":221977472}
I20260709 12:19:01.489125 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:01.491508 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.493206 28283 rpcz_store.cc:276] 0709 12:19:00.442146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.454305 (+1012159us) service_pool.cc:224] Handling call
0709 12:19:01.454723 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.492526 (+ 37803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.492749 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.472470 28287 rpcz_store.cc:276] 0709 12:19:00.209137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.259515 (+ 50378us) service_pool.cc:224] Handling call
0709 12:19:00.259878 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.324212 (+ 64334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.324247 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:00.335699 (+ 11452us) log.cc:844] Serialized 26 byte log entry
0709 12:19:00.460692 (+124993us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:00.460743 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.340003 (+879260us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b540006fc54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:01.340075 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.340157 (+    82us) raft_consensus.cc:1685] finished
0709 12:19:01.340173 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.340448 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16683,"raft.queue_time_us":12344,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":23916,"spinlock_wait_cycles":133718912,"thread_start_us":122017,"threads_started":1,"wal-append.queue_time_us":243649}
I20260709 12:19:01.497526 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:01.500048 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:01.500082 28436 raft_consensus.cc:2889] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:01.498752 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.503142 28989 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:01.504902 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2408) took 1076 ms. Trace:
I20260709 12:19:01.538509 28358 rpcz_store.cc:276] 0709 12:19:00.428617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.429147 (+   530us) service_pool.cc:224] Handling call
0709 12:19:00.429483 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.429728 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:00.546348 (+116620us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.706998 (+160650us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.503766 (+796768us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.504251 (+   485us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.504303 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:01.504351 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.504471 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.504834 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":818610,"prepare.queue_time_us":120058,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":76,"raft.queue_time_us":699920,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":30723,"thread_start_us":299569,"threads_started":2}
I20260709 12:19:01.541708 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:01.543427 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:01.535476 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47863us
I20260709 12:19:01.501572   895 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:01.502409   859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.636s	user 0.017s	sys 0.014s
I20260709 12:19:01.535838 28285 rpcz_store.cc:276] 0709 12:19:00.444411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.486639 (+1042228us) service_pool.cc:224] Handling call
0709 12:19:01.487036 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.487202 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.487239 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.487286 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:01.487316 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.487368 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.487465 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:01.549196 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 57811.2us
I20260709 12:19:01.547813 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2431) took 1092 ms. Trace:
I20260709 12:19:01.484776   972 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.495079   808 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.562813 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.501376   988 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.565899 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:01.551239   928 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:01.533582  1010 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:01.552881   859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.689s	user 0.018s	sys 0.014s
I20260709 12:19:01.559865 28284 rpcz_store.cc:276] 0709 12:19:00.455019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.547048 (+1092029us) service_pool.cc:224] Handling call
0709 12:19:01.547322 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.547492 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.547515 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.547556 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:01.547587 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.547641 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.547755 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.534694 28982 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:01.577282 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:01.578722 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:01.581027   750 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:19:01.579620   941 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:01.551383   998 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:01.569861   972 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:01.546386   985 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:01.579754 28285 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:01.602361 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2434) took 1054 ms. Trace:
I20260709 12:19:01.492856   934 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.593654 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:01.594141 28284 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:01.597397 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:01.612941 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:01.613986 28441 raft_consensus.cc:2410] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 15 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:19:01.596951 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:01.616611 28436 raft_consensus.cc:2368] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:19:01.598608 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:01.560117   992 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:01.599800 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:01.596844 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:01.619529 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.620465   915 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:01.620926 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:01.621690   969 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:01.574402   729 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:01.621085   915 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:01.622215   969 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:01.602773 28285 rpcz_store.cc:276] 0709 12:19:00.548223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.559431 (+1011208us) service_pool.cc:224] Handling call
0709 12:19:01.559782 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.602004 (+ 42222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.602283 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.546999   904 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.626706 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:01.581061  1010 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:01.570211   973 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:19:01.600324   941 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:01.586897   991 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:01.604166   985 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:01.622097   935 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:01.631619   969 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:01.642763 28982 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:01.623596   901 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.605382   998 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:01.557129  1000 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:19:01.625548   897 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:01.630642   992 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:01.677431 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:01.643843   941 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.635283  1010 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:01.573926   902 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.688268 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:01.649869   985 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.666000   666 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:01.707906 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 114341us
I20260709 12:19:01.650081   991 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.690786   941 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:01.716758 28290 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:19:01.717660 28290 pending_rounds.cc:85] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 11
I20260709 12:19:01.717922 28290 pending_rounds.cc:107] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:19:01.719238 28290 raft_consensus.cc:2889] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:01.694671   695 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:01.721354 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:01.694094  1009 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:19:01.718430 28288 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2254) took 2586 ms (client timeout 1584 ms). Trace:
I20260709 12:19:01.682269   952 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:19:01.680987   901 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.706768   950 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:01.732782   950 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.689561   750 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:19:01.733728   901 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:01.734422   901 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:01.692246   906 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:19:01.735351   901 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:19:01.726718 28288 rpcz_store.cc:269] 0709 12:18:59.126292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.126915 (+   623us) service_pool.cc:224] Handling call
0709 12:19:01.710147 (+2583232us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b54001af254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:01.712765 (+  2618us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1740505472}
I20260709 12:19:01.715934   666 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.726385  1009 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:01.738322  1009 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.739441   666 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:01.735759   901 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:19:01.739756  1017 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:19:01.740999   901 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.742628   901 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.739661  1009 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.741964   950 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:01.746877   445 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:01.747577   445 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.745532   974 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:19:01.696460   897 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:01.731040   695 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:01.734484   999 raft_consensus.cc:2764] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:19:01.755247   445 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:01.832660 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 111405us
I20260709 12:19:01.838294 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.839002 28358 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:01.845314 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:01.833770 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.846854 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.847980 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.849931 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.851281 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2409) took 1941 ms. Trace:
I20260709 12:19:01.852922 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.851711 28292 rpcz_store.cc:276] 0709 12:18:59.909639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.126801 (+217162us) service_pool.cc:224] Handling call
0709 12:19:00.133669 (+  6868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.207733 (+ 74064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.207774 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.208010 (+   236us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:00.208049 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.851005 (+1642956us) spinlock_profiling.cc:243] Waited 54 ms on lock 0x7b54002422d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:01.851060 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.851210 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124209536}
I20260709 12:19:01.857025 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:01.857478 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2428) took 1413 ms. Trace:
I20260709 12:19:01.708069   915 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.857926 28291 rpcz_store.cc:276] 0709 12:19:00.444089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.486034 (+1041945us) service_pool.cc:224] Handling call
0709 12:19:01.486390 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:01.486624 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.486649 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:01.487255 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.628687 (+141432us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:01.841795 (+213108us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.857118 (+ 15323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.857207 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.857255 (+    48us) raft_consensus.cc:1685] finished
0709 12:19:01.857269 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.857418 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7422,"thread_start_us":216934,"threads_started":2,"wal-append.queue_time_us":149296}
I20260709 12:19:01.859586 28288 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:01.870108 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 787) took 1024 ms. Trace:
I20260709 12:19:01.870533 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 786) took 1025 ms. Trace:
I20260709 12:19:01.870478 28288 rpcz_store.cc:276] 0709 12:19:00.845315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.868742 (+1023427us) service_pool.cc:224] Handling call
0709 12:19:01.869448 (+   706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.869748 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.869781 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.869826 (+    45us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:01.869857 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.869914 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.870035 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.871742 28364 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:01.867168 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 170622us
I20260709 12:19:01.873054 28257 leader_election.cc:400] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
I20260709 12:19:01.830120   808 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:01.788447   988 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:01.870924 28291 rpcz_store.cc:276] 0709 12:19:00.844960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.865976 (+1021016us) service_pool.cc:224] Handling call
0709 12:19:01.869993 (+  4017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.870225 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.870253 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.870295 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:01.870325 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.870377 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.870479 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.830484   928 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:01.882815 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.886706 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 789) took 1040 ms. Trace:
I20260709 12:19:01.853024   765 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:01.786974   634 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.839395  1009 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:01.878216 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 788) took 1032 ms. Trace:
I20260709 12:19:01.878885 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:01.886240 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.843466  1000 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:01.829248   997 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:01.877040 28292 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:19:01.877396 28257 leader_election.cc:403] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:19:01.888702 28291 rpcz_store.cc:276] 0709 12:19:00.846366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.884362 (+1037996us) service_pool.cc:224] Handling call
0709 12:19:01.885724 (+  1362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.886393 (+   669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.886418 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.886460 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:01.886495 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.886548 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.886650 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.878645 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:01.897173 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2268) took 1199 ms. Trace:
I20260709 12:19:01.893762 28288 rpcz_store.cc:276] 0709 12:19:00.845595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.876367 (+1030772us) service_pool.cc:224] Handling call
0709 12:19:01.877436 (+  1069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.877875 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.877902 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.877947 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:01.877984 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.878038 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.878156 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8576}
I20260709 12:19:01.897593 28292 rpcz_store.cc:276] 0709 12:19:00.697349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.855937 (+1158588us) service_pool.cc:224] Handling call
0709 12:19:01.876792 (+ 20855us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:01.896871 (+ 20079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.897105 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.899447 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 791) took 1041 ms. Trace:
I20260709 12:19:01.900914 28288 rpcz_store.cc:276] 0709 12:19:00.858149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.898585 (+1040436us) service_pool.cc:224] Handling call
0709 12:19:01.898915 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.899108 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.899135 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.899180 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:01.899212 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.899264 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.899375 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.874017   920 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:01.912230 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.914115 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.889003   934 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:01.900406  1009 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.779222   998 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.939620 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:01.937420 28360 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:01.925309 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 790) took 1077 ms. Trace:
W20260709 12:19:01.950551 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 77649.3us
I20260709 12:19:01.964501 28440 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:19:01.925052 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.965363 28291 rpcz_store.cc:276] 0709 12:19:00.847424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.898804 (+1051380us) service_pool.cc:224] Handling call
0709 12:19:01.924755 (+ 25951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.924946 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.924975 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:01.925021 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:01.925051 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.925105 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.925243 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:01.958083 28987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 714) took 1289 ms. Trace:
I20260709 12:19:01.973560 28257 leader_election.cc:400] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14.
I20260709 12:19:01.974960 28257 leader_election.cc:403] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:19:01.902588   808 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:01.933127  1012 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 12:19:01.954448  1024 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:19:01.951813   445 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 12. Current term is 13.
W20260709 12:19:01.938763  1021 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
W20260709 12:19:01.953544  1022 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:19:01.952342   897 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:01.974798 28987 rpcz_store.cc:276] 0709 12:19:00.668266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.668568 (+   302us) service_pool.cc:224] Handling call
0709 12:19:00.668950 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:00.877961 (+209011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.877997 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:00.878042 (+    45us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:00.878092 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.957800 (+1079708us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54002ab554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:01.957863 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.957998 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480148736}
I20260709 12:19:01.938207   925 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.953091  1011 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.023339  1034 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:01.938042  1023 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 12:19:01.936416   987 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:01.953485   625 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:02.005441 28291 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:19:01.999101 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:01.934487   669 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:01.895283   765 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.953702   928 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:01.936254   991 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:01.986812   666 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:02.029577 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 159983us
I20260709 12:19:02.077876 28404 leader_election.cc:400] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:19:02.078265 28404 leader_election.cc:403] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:01.976209   729 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:01.903261  1000 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.038043 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:01.941865   920 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.976022   949 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 12:19:02.049927 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 76515.1us
W20260709 12:19:01.935385  1018 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:19:01.917246   997 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:01.900802   695 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:02.063026 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66294.3us
I20260709 12:19:01.996431   808 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.037528   984 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:02.067196 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2436) took 1426 ms. Trace:
I20260709 12:19:02.104718 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:19:02.110562 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 158215us
I20260709 12:19:02.072767   996 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:02.115943 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2432) took 1658 ms. Trace:
I20260709 12:19:02.055608   925 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.120188 28283 rpcz_store.cc:276] 0709 12:19:00.457057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.546579 (+1089522us) service_pool.cc:224] Handling call
0709 12:19:01.546990 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:01.549802 (+  2812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.549834 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:01.550441 (+   607us) log.cc:844] Serialized 52 byte log entry
0709 12:19:01.632519 (+ 82078us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:01.632563 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.632634 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.952826 (+320192us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:02.111211 (+158385us) raft_consensus.cc:1685] finished
0709 12:19:02.111234 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.115864 (+  4630us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2334,"raft.queue_time_us":3868,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":8257,"spinlock_wait_cycles":57732992,"thread_start_us":61895,"threads_started":1,"wal-append.queue_time_us":136820}
I20260709 12:19:02.071113  1029 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14.
I20260709 12:19:02.114964 28284 rpcz_store.cc:276] 0709 12:19:00.641028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.610880 (+969852us) service_pool.cc:224] Handling call
0709 12:19:01.611192 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.611431 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:01.872140 (+260709us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.066804 (+194664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.066862 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:02.066908 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.066980 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.067130 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49762,"prepare.queue_time_us":322008,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":26087,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3966,"thread_start_us":404819,"threads_started":2}
I20260709 12:19:02.126346 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.077582   999 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:02.057703  1011 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.070824   625 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:02.064990  1034 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.143366 28361 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:19:02.119369   996 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:02.104334   666 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.160146   908 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:02.066285   987 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:02.162904  1041 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:19:02.162406  1010 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:02.157377   968 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 12:19:02.163956  1040 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:19:02.165328 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2437) took 1495 ms. Trace:
I20260709 12:19:02.207991 28285 rpcz_store.cc:276] 0709 12:19:00.670186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.626124 (+955938us) service_pool.cc:224] Handling call
0709 12:19:01.626740 (+   616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:01.770167 (+143427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.770216 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:01.778159 (+  7943us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.922910 (+144751us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:01.922979 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.923108 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.165041 (+241933us) raft_consensus.cc:1685] finished
0709 12:19:02.165079 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.165246 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8364,"raft.queue_time_us":99418,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":152,"thread_start_us":206822,"threads_started":2,"wal-append.queue_time_us":326037}
I20260709 12:19:02.168079  1000 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:02.192965   978 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:02.080216   928 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:02.195034   908 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:02.207376  1010 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.198101 28367 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 10.
I20260709 12:19:02.073572   669 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.109948   729 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.195721 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 816) took 1177 ms. Trace:
I20260709 12:19:02.207053  1050 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:02.214216  1000 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:02.215006   920 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.122000   984 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:02.207752   969 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.002s
I20260709 12:19:02.215662   908 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:02.145107   999 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:02.218878   949 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.205516   987 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:02.210054   634 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:02.217610  1001 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.720s	user 0.013s	sys 0.000s
I20260709 12:19:02.240301 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2386) took 2266 ms. Trace:
I20260709 12:19:02.199568   985 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.220088 28435 rpcz_store.cc:276] 0709 12:19:01.018106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.018689 (+   583us) service_pool.cc:224] Handling call
0709 12:19:01.019045 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:01.037725 (+ 18680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.037786 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:01.040259 (+  2473us) log.cc:844] Serialized 52 byte log entry
0709 12:19:01.546228 (+505969us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:01.573396 (+ 27168us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.574244 (+   848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.574319 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.734127 (+159808us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b1800297700. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:19:02.190676 (+456549us) raft_consensus.cc:1685] finished
0709 12:19:02.195440 (+  4764us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.195640 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":538,"prepare.queue_time_us":26921,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":73884,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":7323,"spinlock_wait_cycles":16193280,"thread_start_us":421256,"threads_started":2,"wal-append.queue_time_us":537955}
I20260709 12:19:02.240775 28362 rpcz_store.cc:276] 0709 12:18:59.971955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.972118 (+   163us) service_pool.cc:224] Handling call
0709 12:19:00.635421 (+663303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:00.638506 (+  3085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.238206 (+1599700us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b54003babd0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:02.238388 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1524322560}
I20260709 12:19:02.216090   996 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.219821  1052 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:19:02.236078   729 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.255815 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:02.236347   984 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:02.225292 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 120670us
I20260709 12:19:02.206565   625 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:02.239898 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 115707us
I20260709 12:19:02.237001   765 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:02.237178  1036 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:19:02.258847 28404 leader_election.cc:400] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13.
I20260709 12:19:02.260646 28436 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:19:02.263592 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.264921 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.274559 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:02.274816 28441 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:02.270044   928 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:02.275102 28436 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:19:02.276868 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:02.231776  1000 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.277906 28432 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:02.280994 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2433) took 1736 ms. Trace:
I20260709 12:19:02.237015   969 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.253528   634 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:02.261397   901 consensus_meta.cc:287] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
I20260709 12:19:02.284873 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:02.242141   949 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.295456 28436 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.281450 28287 rpcz_store.cc:276] 0709 12:19:00.544881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.547434 (+1002553us) service_pool.cc:224] Handling call
0709 12:19:01.552856 (+  5422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:01.761313 (+208457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.761352 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:01.761975 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.973673 (+211698us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:01.973712 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.271831 (+298119us) spinlock_profiling.cc:243] Waited 94.9 ms on lock 0x7b54002409d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:02.271888 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.271938 (+    50us) raft_consensus.cc:1685] finished
0709 12:19:02.271952 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.280915 (+  8963us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98475,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"spinlock_wait_cycles":218299648,"thread_start_us":323515,"threads_started":2,"wal-append.queue_time_us":287290}
I20260709 12:19:02.303905 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:02.305083 28441 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:19:02.244833  1001 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.230546  1050 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.298272 28437 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:02.313180 28438 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:19:02.314896 28438 pending_rounds.cc:85] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 10
I20260709 12:19:02.315194 28438 pending_rounds.cc:107] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:19:02.315428 28438 raft_consensus.cc:2889] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:02.316380 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.294368   808 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.331861 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.292659   925 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.333585 28432 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:02.316630   969 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.341812 28436 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:02.344866  1011 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.349917   634 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.350556   634 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.362094 28983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37070 (request call id 8) took 3355 ms. Trace:
I20260709 12:19:02.362605 28983 rpcz_store.cc:276] 0709 12:18:59.012676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.019058 (+  6382us) service_pool.cc:224] Handling call
0709 12:19:00.693279 (+1674221us) spinlock_profiling.cc:243] Waited 7.06 ms on lock 0x7b540071ee20. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be804520 00007f85be881652 00007f85c384f60a 00007f85bcdd57d4 00007f85bcdd5762 00007f85bcdd5691 00007f85bcdd560c 00007f85bcdd48c2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:02.361863 (+1668584us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b4c00055f80. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bcb53a20 00007f85bcb64a29 00007f85bcb2a644 00007f85bcb51f2d 00007f85bcb51b93 00007f85be881c92 00007f85be88184a 00007f85c384f60a 00007f85bcdd57d4 00007f85bcdd5762 00007f85bcdd5691
0709 12:19:02.361987 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19354368}
I20260709 12:19:02.334251  1059 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:02.363515  1059 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:02.363940  1059 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:02.344620   941 consensus_meta.cc:287] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.000s
I20260709 12:19:02.367854   941 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.368350 28988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:40336 (request call id 8) took 2648 ms. Trace:
I20260709 12:19:02.368796 28988 rpcz_store.cc:276] 0709 12:18:59.720163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.723564 (+  3401us) service_pool.cc:224] Handling call
0709 12:19:02.367875 (+2644311us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b540071e654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88d735 00007f85be8859fa 00007f85c384d4c9 00007f85bcdc9c54 00007f85bcdc9be2 00007f85bcdc9b11 00007f85bcdc9a8c 00007f85bcdc8d42 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:02.368263 (+   388us) inbound_call.cc:177] Queueing failure response
Metrics: {"spinlock_wait_cycles":6075160960}
I20260709 12:19:02.373235   902 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:02.334820   895 consensus_meta.cc:287] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.781s	user 0.014s	sys 0.000s
I20260709 12:19:02.374168   895 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.377851 28437 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
W20260709 12:19:02.379706   951 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 2.175s	user 0.009s	sys 0.000s
I20260709 12:19:02.381263   669 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.381830  1009 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.382648   666 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.384618   949 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.386788 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 820) took 1348 ms. Trace:
I20260709 12:19:02.387192 28439 rpcz_store.cc:276] 0709 12:19:01.038043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.038180 (+   137us) service_pool.cc:224] Handling call
0709 12:19:01.319286 (+281106us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:01.319758 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.319787 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:01.320378 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.502820 (+182442us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:01.502870 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.502943 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.695324 (+192381us) raft_consensus.cc:1685] finished
0709 12:19:01.695354 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.386470 (+691116us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5400347950. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:02.386720 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1345,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":123,"spinlock_wait_cycles":645472128,"thread_start_us":180451,"threads_started":1,"wal-append.queue_time_us":265146}
I20260709 12:19:02.390439   999 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:02.338703   901 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:02.391320   901 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:02.320742   695 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:02.392130   695 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.393206  1001 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.394286 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 783) took 1990 ms. Trace:
I20260709 12:19:02.322825   729 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:02.391713   901 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:02.319056  1053 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:02.396732  1000 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.394449   985 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.398164   729 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.399420 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 664) took 2661 ms. Trace:
I20260709 12:19:02.399864 28363 rpcz_store.cc:276] 0709 12:18:59.737783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.738341 (+   558us) service_pool.cc:224] Handling call
0709 12:18:59.764057 (+ 25716us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:59.772182 (+  8125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:59.772215 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:59.780135 (+  7920us) log.cc:844] Serialized 52 byte log entry
0709 12:19:00.059520 (+279385us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:00.174921 (+115401us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.320465 (+145544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.320566 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.379003 (+2058437us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:02.381912 (+  2909us) raft_consensus.cc:1685] finished
0709 12:19:02.382370 (+   458us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.399328 (+ 16958us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74194,"prepare.queue_time_us":260059,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":9009,"raft.queue_time_us":35309,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":7994,"spinlock_wait_cycles":4071808,"thread_start_us":432675,"threads_started":3,"wal-append.queue_time_us":320645}
I20260709 12:19:02.394733 28290 rpcz_store.cc:276] 0709 12:19:00.404070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.404251 (+   181us) service_pool.cc:224] Handling call
0709 12:19:01.710376 (+1306125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.719755 (+  9379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.394068 (+674313us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54001af250. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:02.394218 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3003008768}
I20260709 12:19:02.405527 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:02.406348 28983 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:19:02.407554 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:02.410107  1058 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:02.410130 28986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:19:02.411952 28986 raft_consensus.cc:606] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:02.415257  1050 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.411521  1052 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:02.415616 28986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:19:02.416595  1052 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:02.417598  1052 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.418680  1052 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:02.413231 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 75975.7us
I20260709 12:19:02.425601 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:02.425662 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 83563.2us
W20260709 12:19:02.431936 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.433537 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.435603 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:02.435899 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.445235 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:02.446925 28332 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:02.448151 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.444252 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:02.450728 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:02.443249 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:02.451474 28366 raft_consensus.cc:2393] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:19:02.454269  1019 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.674s	user 0.000s	sys 0.003s
I20260709 12:19:02.455534 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2426) took 2012 ms. Trace:
I20260709 12:19:02.455917 28286 rpcz_store.cc:276] 0709 12:19:00.443229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.482435 (+1039206us) service_pool.cc:224] Handling call
0709 12:19:01.482882 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:01.492178 (+  9296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.492213 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:01.496497 (+  4284us) log.cc:844] Serialized 26 byte log entry
0709 12:19:01.603635 (+107138us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:01.831747 (+228112us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.832029 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.832118 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.419697 (+587579us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:02.455219 (+ 35522us) raft_consensus.cc:1685] finished
0709 12:19:02.455334 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.455466 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36920,"prepare.queue_time_us":329377,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":65348,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":11363,"spinlock_wait_cycles":6759168,"thread_start_us":296683,"threads_started":2,"wal-append.queue_time_us":213029}
I20260709 12:19:02.462447  1050 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:02.462983  1050 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:02.463377  1050 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.465246 28283 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:19:02.465638 28283 pending_rounds.cc:85] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 6
I20260709 12:19:02.465902 28283 pending_rounds.cc:107] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:19:02.466127 28283 raft_consensus.cc:2889] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:02.471050   666 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:02.471822   666 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:02.474102 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.475538 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.475879   949 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:02.476285   949 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:02.476620 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.478467 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:02.479622 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.480351   920 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:02.480793   920 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:02.481231 28363 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:02.480624 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.413496   996 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:02.400121   695 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.469311  1059 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.476643   949 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:02.494160 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.493362 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:19:02.496618 28174 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 112978us
I20260709 12:19:02.481251   920 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.504781 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:02.506424 28363 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:02.510514 28988 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:02.508349  1053 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:02.498816   941 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.507926 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.483888   901 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.492718   984 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:19:02.492388   685 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:19:02.492861  1054 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:19:02.515688  1000 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:02.501204   919 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:19:02.497363   935 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:19:02.540594  1059 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.551062   666 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:02.551678   666 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:02.552747   666 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.546375   901 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.546800 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 794) took 1093 ms. Trace:
W20260709 12:19:02.549103 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 430464us
I20260709 12:19:02.555697 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:02.554322   666 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.560642 28288 rpcz_store.cc:276] 0709 12:19:01.453553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.902411 (+448858us) service_pool.cc:224] Handling call
0709 12:19:01.903277 (+   866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:01.903479 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:02.057891 (+154412us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.067981 (+ 10090us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.365271 (+297290us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b18000fdfc0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:19:02.546263 (+180992us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b18000fdfc0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:19:02.546436 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.546487 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:02.546552 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.546627 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.546738 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11725,"prepare.queue_time_us":32027,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":491248,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":207,"spinlock_wait_cycles":8043776,"thread_start_us":151690,"threads_started":1}
I20260709 12:19:02.569454 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.570706 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:02.573947 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.575383 28287 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:19:02.576135 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.576844 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.577190 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.577826 28286 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:02.577921 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.578964 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:02.579595 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.580675 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.582582 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.581651 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.576390 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.586905 28287 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:02.589905 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.587018  1059 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:19:02.592885 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:02.596549 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:02.597851 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.601121 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:02.601161 28290 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:02.604159 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.604966 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.605261 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.605968 28290 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:02.606364 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:02.606814 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.608686 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.609316 28290 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:02.610164 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.613188 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:02.613409 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.614172 28288 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:02.616554 28288 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:19:02.618592 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:02.571533  1050 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:02.623221 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:02.624493 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.626339 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.627637 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:02.627995 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:02.631443 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:02.636799 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.637457 28287 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:19:02.650615 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.651336 28439 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:02.652339 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.653841 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:02.654436 28435 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.654734 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.655637 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.656253 28439 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:02.660012 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:02.602876   685 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:19:02.632678   999 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:02.670290 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 107985us
I20260709 12:19:02.661338 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.672043 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:02.673162 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:02.662194 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:02.604393   901 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:19:02.670239 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.645381   859 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:02.706758 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:02.708498 28287 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:02.708536 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:02.709625 28286 raft_consensus.cc:2410] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:02.710024 28287 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:19:02.711206 28358 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:02.704658   996 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.719188   997 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.721284 28330 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:02.722270   935 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:02.723048  1050 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:02.675112  1059 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:02.724129   934 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.724710   934 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:02.726933   934 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:02.728094 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 56178.7us
I20260709 12:19:02.729959   698 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:02.730770   698 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:02.731099   999 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:02.731591   999 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:02.731942   999 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:02.734052 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.737036 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.737980 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.743134 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.743537 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:02.743830 28361 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:02.661643  1001 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:19:02.748574 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:02.748767 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:02.708745   904 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:02.769236 28255 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:02.775437 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51844us
I20260709 12:19:02.777232 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:02.778321 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:19:02.782536 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 151294us
I20260709 12:19:02.782557 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:02.793116 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:02.794541 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:02.795172 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:02.798844 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:02.781231   685 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:02.794459   992 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.051s	user 0.024s	sys 0.015s
I20260709 12:19:02.792868   934 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:02.789085  1054 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:02.792001  1011 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:02.814291 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 168881us
I20260709 12:19:02.826356 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:02.816280 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:02.834784 28435 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:02.835958   949 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:02.825452   992 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.082s	user 0.025s	sys 0.016s
I20260709 12:19:02.814715   997 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.824525  1002 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:02.849373 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 797) took 1094 ms. Trace:
I20260709 12:19:02.811997   987 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:02.816253   685 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.811322   985 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 12:19:02.862855 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 189764us
I20260709 12:19:02.862694 28285 rpcz_store.cc:276] 0709 12:19:01.754803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.209837 (+455034us) service_pool.cc:224] Handling call
0709 12:19:02.216744 (+  6907us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:02.589675 (+372931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.589709 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:02.590344 (+   635us) log.cc:844] Serialized 52 byte log entry
0709 12:19:02.750129 (+159785us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:02.797822 (+ 47693us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.812312 (+ 14490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.812401 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.849115 (+ 36714us) raft_consensus.cc:1685] finished
0709 12:19:02.849140 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.849302 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":447716,"prepare.queue_time_us":51085,"prepare.run_wall_time_us":50,"raft.queue_time_us":395251,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":5531,"spinlock_wait_cycles":7049472,"thread_start_us":142562,"threads_started":1,"wal-append.queue_time_us":253672}
I20260709 12:19:02.869874 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:02.872850 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:02.882248 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:02.884135 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:02.872844 28983 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:02.862102   906 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:02.862277  1002 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.901857 28435 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:02.871212   935 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 12:19:02.908480 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 58502.5us
I20260709 12:19:02.867916   685 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:02.911319 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:02.864785   808 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:02.913127 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:02.932421 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:02.929958 28432 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:02.886463   901 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:02.911609   906 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.938642 28441 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:19:02.894675  1052 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:02.940411 28987 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:02.914222  1002 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.942147 28441 pending_rounds.cc:85] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 6
I20260709 12:19:02.944756 28441 pending_rounds.cc:107] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:19:02.945047 28441 raft_consensus.cc:2889] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:02.946513 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:02.867422   997 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:02.871347  1068 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:19:02.957756 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:03.012934 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:03.006855 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:02.929716   808 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.006414 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:03.010985 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 64611us
W20260709 12:19:03.044692 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.022310 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 89964.8us
I20260709 12:19:03.056769 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:02.931334  1050 raft_consensus.cc:2764] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:19:03.051476   987 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:19:03.008018   999 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.054715   901 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:03.072875  1054 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:03.052664 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:03.064548   949 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:02.943305   906 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.941512  1059 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:03.052479  1071 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:19:03.082020 28984 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:03.082988 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:02.941666   859 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.138s
I20260709 12:19:03.009897   669 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:03.075139   808 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:03.078228  1054 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:19:03.095408   920 consensus_meta.cc:287] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.000s
I20260709 12:19:03.099279   920 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.100675 28364 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:03.102839   920 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:03.096817   998 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.013s	sys 0.004s
I20260709 12:19:03.108408  1055 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:03.114646 28367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56626 (request call id 684) took 1228 ms. Trace:
I20260709 12:19:03.116834 28359 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:03.117638 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 171199us
I20260709 12:19:03.118620  1011 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.118854   729 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:03.120260   685 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.120292   997 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:03.122152 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:03.123217 28288 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:03.132264 28984 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:03.129220 28367 rpcz_store.cc:276] 0709 12:19:01.885778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.885973 (+   195us) service_pool.cc:224] Handling call
0709 12:19:03.114441 (+1228468us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b540007a654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:03.114578 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699072896}
W20260709 12:19:03.127298   998 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.013s	sys 0.004s
I20260709 12:19:03.137943  1005 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:03.139420   685 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:03.140434  1092 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:19:03.152057  1010 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:03.179302  1052 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:03.179706  1052 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:03.155735  1026 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:03.157070  1034 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:03.179941  1050 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:03.179625 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 122954us
I20260709 12:19:03.160457  1005 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.181533 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:03.181803   884 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:03.182817  1005 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:03.164304 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2412) took 1261 ms. Trace:
I20260709 12:19:03.183569  1084 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:03.184175  1084 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:03.183609 28365 rpcz_store.cc:276] 0709 12:19:01.902626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.902778 (+   152us) service_pool.cc:224] Handling call
0709 12:19:01.903090 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.110950 (+207860us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.110987 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.111035 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:02.111070 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.164007 (+1052937us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400135454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:03.164062 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.164226 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477526912}
I20260709 12:19:03.166510   895 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:03.171856   634 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:03.176136 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:03.176728   934 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.186546   934 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.186853 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:03.188184 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:03.188700   934 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:03.177183   784 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.189606   784 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:03.190105  1026 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.192310  1026 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:03.192420   784 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:03.192886  1026 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:03.178025 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 55975.4us
I20260709 12:19:03.182751 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.194411 28365 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:03.168041  1000 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:03.195101  1000 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:03.195463   634 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:03.177855   902 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:03.185910  1084 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.195946  1000 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.197052  1000 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.095176   669 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:03.194659 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:03.199059 28257 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:03.197693  1054 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:03.159473  1057 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.797s	user 0.005s	sys 0.000s
I20260709 12:19:03.189364 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:03.196724   902 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.211951 28441 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:03.232398 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:03.232975 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:03.212843 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.233584 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:03.235400 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 46111.9us
I20260709 12:19:03.227268 28290 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:03.236282 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:03.237258 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.237381 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.238125 28437 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:03.238473 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.239014 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.240104 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.241596 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:03.229689 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 792) took 2198 ms. Trace:
I20260709 12:19:03.242516 28292 rpcz_store.cc:276] 0709 12:19:01.030930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.917767 (+886837us) service_pool.cc:224] Handling call
0709 12:19:01.967152 (+ 49385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.194528 (+227376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.194574 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.194621 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:02.194657 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.229383 (+1034726us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54000f2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:03.229448 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.229606 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522404224}
W20260709 12:19:03.243819 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.234261 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:03.247215 28432 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:19:03.189759   895 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.253684 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.251163 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.260087 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2447) took 1586 ms. Trace:
I20260709 12:19:03.260564 28283 rpcz_store.cc:276] 0709 12:19:01.673407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.152431 (+479024us) service_pool.cc:224] Handling call
0709 12:19:02.152759 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.466542 (+313783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.259834 (+793292us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b540005a0d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:03.260012 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717800192}
I20260709 12:19:03.261600 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.268743 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.269162 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:03.180986  1052 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.249490   902 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:03.294241 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:03.295658 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 61461.6us
I20260709 12:19:03.294977 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:03.277951 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.266155  1005 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:03.312497 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:03.255506   998 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:03.266721   729 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:03.324173 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 46341us
I20260709 12:19:03.280620  1059 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.254345  1026 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:03.314957  1048 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:03.293987  1010 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:03.349795 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:03.350108 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:03.279649   941 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:03.281047  1050 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:03.338080 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:03.342864 28361 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:03.341023 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.313182 28441 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:03.270954  1054 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:03.321727  1052 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:03.358243 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.331678   901 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:03.333786   729 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.333402   998 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.369995 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:03.374225 28362 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
W20260709 12:19:03.371436 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.377099 28366 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:03.378459 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:03.336920  1005 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:03.462148  1005 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.381628 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:03.467046 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:03.467754 28292 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:03.383106 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:03.482523 28360 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:03.387717 28185 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 11 to 15, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 15 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:03.390308 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:03.392261 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 78237.1us
I20260709 12:19:03.393064 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2416) took 1483 ms. Trace:
I20260709 12:19:03.393586   908 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:03.396245  1042 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:03.319461  1011 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:19:03.430423 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:03.432350   904 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:03.437319  1034 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:03.438542 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 51413us
I20260709 12:19:03.512652 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:03.513475 28367 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:03.440452   968 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:03.347388   859 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.364887   884 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:03.370795  1079 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 12:19:03.488130 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:03.489292 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:03.347390   688 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.496446 28359 rpcz_store.cc:276] 0709 12:19:01.909608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.910441 (+   833us) service_pool.cc:224] Handling call
0709 12:19:03.116615 (+1206174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:03.129357 (+ 12742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.392790 (+263433us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540007a650. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:03.392978 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2773182208}
I20260709 12:19:03.496227   784 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:03.385768   998 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:03.484591   729 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:03.538255 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.512737 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:03.567479 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 54833.4us
I20260709 12:19:03.499511   908 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:03.481609  1026 consensus_queue.cc:1059] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.150s
I20260709 12:19:03.516953   968 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:03.499691   901 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:19:03.583168 28292 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:03.537065   884 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:03.266604  1095 consensus_queue.cc:1175] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f526369c0d0743b192be6e205540a059" responder_term: 20 status { last_received { term: 19 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:03.598201 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:03.523262  1068 consensus_queue.cc:1059] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:19:03.523367   750 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:19:03.674489  1068 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:03.674891  1068 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:03.632179 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.570751   688 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:03.675806  1068 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:03.677191  1068 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:03.678463   688 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:03.633054 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.633468   908 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:03.634176  1099 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:03.635466   698 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:03.579716   784 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:03.521222   904 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:03.639158   968 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:03.643193   906 consensus_meta.cc:287] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.000s
I20260709 12:19:03.522455  1034 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:03.628046   884 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:03.625240  1002 consensus_meta.cc:287] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.615s	user 0.011s	sys 0.000s
I20260709 12:19:03.651790   729 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.578675   941 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:03.660372   808 consensus_meta.cc:287] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.003s
I20260709 12:19:03.720618   808 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.722174 28332 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.722963   808 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:03.723100   750 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.724248 28332 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.725142   750 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:03.726696   999 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:03.673362   669 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:03.681854 28988 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
W20260709 12:19:03.682794 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 103603us
I20260709 12:19:03.683074 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:03.686685 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.691730 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.683843   908 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:03.701964  1009 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:03.719997   941 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:03.725602 28332 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.728566 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.730230 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:03.730953 28437 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:03.736706 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.738420 28438 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:19:03.739301 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:03.740700 28332 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.741958 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.743424 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:03.742626 28437 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:03.744477   984 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.752513   784 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:03.752918 28441 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:03.713748   906 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.744495   669 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:03.756714   906 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:03.757992 28361 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:03.760016 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:03.760828 28437 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:03.775764 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:03.731429  1009 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:03.731489  1058 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.735004   941 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:03.784405 28287 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.008s	sys 0.005s
I20260709 12:19:03.794217 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:03.751048   750 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:03.814121 28287 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.008s	sys 0.005s
W20260709 12:19:03.839033 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 95680.1us
W20260709 12:19:03.904273 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 94053.5us
I20260709 12:19:03.728088  1002 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.916757 28437 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:03.918500 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:19:03.771010   892 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.414s	user 0.045s	sys 0.070s
I20260709 12:19:03.920687 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:03.779289   808 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:03.804401  1000 consensus_meta.cc:287] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.593s	user 0.005s	sys 0.007s
I20260709 12:19:03.778504   934 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:03.953110 28437 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:03.954650 28364 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:03.957397 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.902475  1099 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.188s	user 0.006s	sys 0.006s
I20260709 12:19:03.815433  1052 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:03.961596 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:03.944453   892 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.587s	user 0.046s	sys 0.071s
I20260709 12:19:03.906522   445 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:03.903481   984 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:03.921325  1102 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.170s	user 0.009s	sys 0.005s
I20260709 12:19:03.979818 28290 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:19:03.985625 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.974395  1077 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:19:03.994668 28986 raft_consensus.cc:606] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:03.948408   928 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:04.007746 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 220113us
I20260709 12:19:04.007920   928 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.008877 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:04.010617   928 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.010848 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:04.011658  1050 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:04.012202 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2434) took 1131 ms. Trace:
I20260709 12:19:04.013015   765 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:04.013880   765 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:04.014371   765 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:04.015652 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:04.016356 28434 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:03.987139  1081 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.017725  1081 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:03.950309   634 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.019997   634 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.022378 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.023650  1034 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.023936 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.024622   904 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:03.916545  1079 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:19:04.030166 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.032830 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.036731 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:03.980768   902 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.659s	user 0.003s	sys 0.007s
I20260709 12:19:04.046105   902 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.046280  1009 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.047647   920 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.048637   902 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:04.048413   920 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:03.906575  1054 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 12:19:04.049759  1054 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:03.955513   784 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.049949  1002 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:03.967929   997 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.050892 28292 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:19:04.050774   997 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:03.905097  1058 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.053143   997 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:03.951309  1053 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.057220 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.017196   858 raft_consensus.cc:993] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:03.969031   925 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.015334   765 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:04.006834 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 102710us
I20260709 12:19:04.063198 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:04.012588 28358 rpcz_store.cc:276] 0709 12:19:02.880566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.880720 (+   154us) service_pool.cc:224] Handling call
0709 12:19:02.881058 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.904502 (+ 23444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.904534 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:03.251019 (+346485us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b2c0000e678. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:19:03.251572 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 12:19:03.703298 (+451726us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:03.703336 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.705130 (+  1794us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.011880 (+306750us) raft_consensus.cc:1685] finished
0709 12:19:04.011901 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.012129 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23057,"raft.queue_time_us":23462,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":17320832,"thread_start_us":390949,"threads_started":1,"wal-append.queue_time_us":481807}
I20260709 12:19:04.004724  1084 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:03.951035  1025 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.059409 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:04.075203 28440 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.011s	sys 0.016s
W20260709 12:19:04.075946 28440 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.011s	sys 0.016s
I20260709 12:19:04.069010 28432 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:04.005337  1000 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.022341   634 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.079409 28332 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:04.080295 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.968370   808 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.054917  1054 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:03.988260  1052 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.162365  1052 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.164949 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.165932 28984 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:04.082935  1084 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.167790 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.168223  1084 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.072276  1053 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.171696  1053 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.078401  1025 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.175257  1025 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.083511   925 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.012213  1050 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.183722  1050 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.186097   984 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.186856   984 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.189283   984 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.050500   784 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.194474   729 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.180308   925 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.206919 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.207155 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.208549 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.210701 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.211263 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:04.211766 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.215353 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.157567   920 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.084549   906 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.131s
W20260709 12:19:04.231855 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.155630  1000 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.213902  1002 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.240274 28364 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:04.053300  1058 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.020640  1081 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.082381   669 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.082618   750 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:04.262423 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 183128us
I20260709 12:19:04.280826 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:04.087342  1112 raft_consensus.cc:1081] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:04.252764   920 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.257244   729 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:04.303839 28286 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.002s	sys 0.049s
W20260709 12:19:04.312830 28286 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.002s	sys 0.050s
W20260709 12:19:04.304642 28289 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.010s	sys 0.041s
W20260709 12:19:04.313463 28289 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.010s	sys 0.041s
I20260709 12:19:04.317770 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:04.107583   998 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.252717   904 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.324738   984 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.326431   919 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:04.343575 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2753) took 1084 ms. Trace:
I20260709 12:19:04.261090  1002 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.329723   968 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.339123   904 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:04.356338  1113 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:04.344765 28441 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:04.236524  1054 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.365196  1000 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:04.324741  1108 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:19:04.366832 28436 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: [16.7-16.7]
I20260709 12:19:04.349534 28440 rpcz_store.cc:276] 0709 12:19:03.259459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.271268 (+ 11809us) service_pool.cc:224] Handling call
0709 12:19:03.271686 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:03.275125 (+  3439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:03.275164 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:03.275726 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 12:19:04.076644 (+800918us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:04.076700 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.076790 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.343281 (+266491us) raft_consensus.cc:1685] finished
0709 12:19:04.343307 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.343486 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125874,"raft.queue_time_us":3894,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":71887,"thread_start_us":737395,"threads_started":1,"wal-append.queue_time_us":946826}
I20260709 12:19:04.311650  1081 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:04.334136  1029 raft_consensus.cc:3055] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:04.383270 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:04.352605  1068 consensus_meta.cc:287] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.675s	user 0.014s	sys 0.000s
I20260709 12:19:04.315570   920 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.387880   904 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:04.388464   904 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.389297  1002 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:04.390688   784 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.316155   729 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.393190   729 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.394145   808 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:04.338724  1116 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:04.394811 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2480) took 1087 ms. Trace:
I20260709 12:19:04.395578  1010 raft_consensus.cc:993] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:04.395177 28292 rpcz_store.cc:276] 0709 12:19:03.307630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.584085 (+276455us) service_pool.cc:224] Handling call
0709 12:19:03.584497 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.051307 (+466810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.394495 (+343188us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b540009e2d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:04.394740 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072000768}
I20260709 12:19:04.396651 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:04.397771 28292 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:04.311949   750 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:04.402324 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.382153   908 consensus_meta.cc:287] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.001s
I20260709 12:19:04.389182   920 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.384616  1058 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.383838  1029 raft_consensus.cc:740] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:19:04.434957 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 424165us
I20260709 12:19:04.409973 28365 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:04.404585   729 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:04.412645 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 95006.2us
I20260709 12:19:04.441061 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:04.446380 28257 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:19:04.447999 28438 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.005s	sys 0.017s
W20260709 12:19:04.415067   884 consensus_meta.cc:287] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.684s	user 0.012s	sys 0.001s
I20260709 12:19:04.399030   941 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.386482  1081 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:04.455049 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.458051 28438 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.005s	sys 0.018s
W20260709 12:19:04.464336 28255 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.006s	sys 0.015s
W20260709 12:19:04.466028 28255 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.006s	sys 0.015s
W20260709 12:19:04.468950 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 664442us
I20260709 12:19:04.486095 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.496181 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2479) took 1188 ms. Trace:
I20260709 12:19:04.496408 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.496672 28290 rpcz_store.cc:276] 0709 12:19:03.307379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.519804 (+212425us) service_pool.cc:224] Handling call
0709 12:19:03.540608 (+ 20804us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.003836 (+463228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.495855 (+492019us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54003f9154. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:04.496102 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009119488}
I20260709 12:19:04.449560  1026 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:04.454629   935 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:04.462857   784 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:04.501713 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:04.510562 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.511135   784 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.511341 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.491591   634 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.516712   784 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.517107   634 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.520149   634 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.521809 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.523135 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.524577 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.526098 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:04.485397   625 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:19:04.528457   625 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:04.528904   625 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:04.529345 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:04.529845   625 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.531271   625 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.531759  1081 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:04.534776  1106 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.296s	user 0.003s	sys 0.000s
I20260709 12:19:04.535154   941 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:04.535830   941 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:04.524932 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.501291  1084 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:04.525456 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:04.528319 28258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.012s	sys 0.008s
W20260709 12:19:04.538297 28258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.012s	sys 0.009s
I20260709 12:19:04.444334   729 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.524534 28359 raft_consensus.cc:1240] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:04.491814  1000 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:04.536782 28363 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:04.537833 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.515043 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2277) took 1940 ms. Trace:
I20260709 12:19:04.445540  1058 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.501128  1052 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.464463   884 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.495003  1025 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.565630   908 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.566167 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.566350   908 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.566754 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.568648   908 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.517024   750 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.559917  1052 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.571425  1052 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.571666 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.572557 28360 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:04.547519 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:04.573006 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2751) took 1314 ms. Trace:
I20260709 12:19:04.537216  1084 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.573428 28436 rpcz_store.cc:276] 0709 12:19:03.258920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.264099 (+  5179us) service_pool.cc:224] Handling call
0709 12:19:03.264441 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.379895 (+1115454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.572675 (+192780us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002e45d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:04.572933 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2534542080}
I20260709 12:19:04.574878 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:04.575507  1084 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.510715   935 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:04.576251 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:04.576946 28436 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:04.576721   935 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.578241   935 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.580669 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 808) took 1373 ms. Trace:
I20260709 12:19:04.565835   729 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.582206 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.583925 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.587160 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.562223  1058 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.591913 28366 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:04.595470   998 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.597438  1112 raft_consensus.cc:1081] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:04.566368  1025 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.547855 28291 rpcz_store.cc:276] 0709 12:19:02.574411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.577895 (+  3484us) service_pool.cc:224] Handling call
0709 12:19:02.578186 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.578387 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:02.780227 (+201840us) log.cc:844] Serialized 14 byte log entry
0709 12:19:03.756840 (+976613us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b1800396340. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85c227f3c7 00007f85be8919a7 00007f85be8a10bf 00007f85be8a1038
0709 12:19:04.250119 (+493279us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.514684 (+264565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.514742 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:04.514791 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.514859 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.514975 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74224,"prepare.queue_time_us":937732,"prepare.run_cpu_time_us":165,"prepare.run_wall_time_us":16204,"raft.queue_time_us":43383,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":16844,"spinlock_wait_cycles":2721536,"thread_start_us":1017171,"threads_started":4,"wal-append.queue_time_us":239480,"wal-append.run_cpu_time_us":1168,"wal-append.run_wall_time_us":1033828}
I20260709 12:19:04.603188  1025 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.603472 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.605051 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.566089 28983 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:04.582391  1011 raft_consensus.cc:3055] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:04.613294 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.614936 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:04.615988 28440 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:19:04.618019 28362 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:04.616979 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.618992 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.621343 28364 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:04.623499 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.623667 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.624300 28364 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:04.625389 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.626032 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:04.627761 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:04.631721 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:04.634099 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 807) took 1431 ms. Trace:
I20260709 12:19:04.636650 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.581141 28283 rpcz_store.cc:276] 0709 12:19:03.206986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.272063 (+ 65077us) service_pool.cc:224] Handling call
0709 12:19:03.272571 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:03.513227 (+240656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:03.513260 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:03.513809 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:19:03.922256 (+408447us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:03.987432 (+ 65176us) log.cc:844] Serialized 14 byte log entry
0709 12:19:03.987667 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.580224 (+592557us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54001af254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:04.580277 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.580334 (+    57us) raft_consensus.cc:1685] finished
0709 12:19:04.580440 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.580600 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22748,"prepare.queue_time_us":97092,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":33327,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":4898,"spinlock_wait_cycles":552548736,"thread_start_us":403184,"threads_started":1,"wal-append.queue_time_us":746861}
I20260709 12:19:04.638124 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:04.639256 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.634542 28287 rpcz_store.cc:276] 0709 12:19:03.202485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.202729 (+   244us) service_pool.cc:224] Handling call
0709 12:19:03.208056 (+  5327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:03.208439 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:03.208469 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:03.208980 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:19:03.903248 (+694268us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:03.903294 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.903376 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.534111 (+630735us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:04.628646 (+ 94535us) raft_consensus.cc:1685] finished
0709 12:19:04.628673 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.634015 (+  5342us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19019,"raft.queue_time_us":2328,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":13805,"spinlock_wait_cycles":32961280,"thread_start_us":693865,"threads_started":1,"wal-append.queue_time_us":1014772}
I20260709 12:19:04.637588 28360 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:04.642130 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:04.643014 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:04.643975 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:04.651948 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:04.652868 28283 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.618969  1011 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:04.667124 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:04.667306 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:04.669013 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.674341 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2470) took 1741 ms. Trace:
I20260709 12:19:04.674942 28284 rpcz_store.cc:276] 0709 12:19:02.933149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.933283 (+   134us) service_pool.cc:224] Handling call
0709 12:19:02.933606 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.933868 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:04.227833 (+1293965us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.673953 (+446120us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.674013 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:04.674063 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.674140 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.674276 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":879086,"prepare.queue_time_us":844375,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":90780,"thread_start_us":821590,"threads_started":2}
I20260709 12:19:04.676772 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.680068 28284 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:04.638001   952 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:19:04.547421  1000 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.649811  1025 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:04.689710 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:04.671054 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2472) took 1735 ms. Trace:
I20260709 12:19:04.684520 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.690618 28360 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:04.687795 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2471) took 1753 ms. Trace:
I20260709 12:19:04.654652  1052 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:04.702867 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:04.655243   902 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.698340 28285 rpcz_store.cc:276] 0709 12:19:02.934818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.935001 (+   183us) service_pool.cc:224] Handling call
0709 12:19:02.945140 (+ 10139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.945349 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:04.261062 (+1315713us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.687373 (+426311us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.687433 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:04.687496 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.687575 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.687721 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":812901,"prepare.queue_time_us":1056090,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":920838,"threads_started":2}
I20260709 12:19:04.645025 28440 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:53280
I20260709 12:19:04.701139 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:04.709956 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.711862 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2473) took 1776 ms. Trace:
I20260709 12:19:04.671797  1042 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.715421 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:04.662772 28436 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:53280
I20260709 12:19:04.713483 28289 rpcz_store.cc:276] 0709 12:19:02.935494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.935897 (+   403us) service_pool.cc:224] Handling call
0709 12:19:02.936158 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.937497 (+  1339us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:04.313919 (+1376422us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.710093 (+396174us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.711493 (+  1400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.711551 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:04.711600 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.711675 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.711799 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":839162,"prepare.queue_time_us":1056606,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"thread_start_us":933497,"threads_started":2}
I20260709 12:19:04.716523 28436 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:04.716928 28436 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.706032 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.665439   941 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:04.671211  1129 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.682148  1116 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.717828 28287 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:04.700131 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:04.689471   928 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.725695 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.711781 28440 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:04.720964 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.728524 28291 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:04.728852 28440 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.698915  1025 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.712433  1125 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.734828 28283 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:04.712958   750 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:04.739902 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.739845 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.719484   695 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:04.714145   808 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.720305   998 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.743062 28283 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:04.744570 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:04.747855 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:04.742389 28284 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:04.741287   750 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:04.722693  1131 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.720434  1127 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:04.800001  1127 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.803372 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2417) took 1007 ms. Trace:
I20260709 12:19:04.803869 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2762) took 1446 ms. Trace:
I20260709 12:19:04.763509 28440 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.806128 28440 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.807320 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:04.808511 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:04.810256  1125 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.766263  1052 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.803879 28441 rpcz_store.cc:276] 0709 12:19:03.796041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.360777 (+564736us) service_pool.cc:224] Handling call
0709 12:19:04.361129 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.361494 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.361523 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.366082 (+  4559us) log.cc:844] Serialized 26 byte log entry
0709 12:19:04.638848 (+272766us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:04.638890 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.638960 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.803138 (+164178us) raft_consensus.cc:1685] finished
0709 12:19:04.803159 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.803293 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4454,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":272394,"threads_started":1,"wal-append.queue_time_us":423042}
I20260709 12:19:04.813024 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:04.724949  1116 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.819818 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2757) took 1515 ms. Trace:
I20260709 12:19:04.804425 28437 rpcz_store.cc:276] 0709 12:19:03.357160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.959437 (+602277us) service_pool.cc:224] Handling call
0709 12:19:03.959725 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.173442 (+213717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.173480 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.173523 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:04.173555 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.803641 (+630086us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400185954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:04.803690 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.803808 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491020160}
I20260709 12:19:04.735981   928 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.499866  1029 consensus_queue.cc:260] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.779031  1010 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:04.782305   969 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:04.727087  1129 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.722190  1042 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.722813   941 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.785535 28332 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:04.737042  1025 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.750766   695 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.786710   915 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.786422  1130 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:04.736660  1011 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.763463   968 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:19:04.794982  1000 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.730026   884 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:04.764770   808 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.713924   902 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.803372 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2763) took 1440 ms. Trace:
I20260709 12:19:04.812714  1131 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.809639 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:04.840898 28440 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:04.768216 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:04.819447 28441 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:04.738402  1058 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.839723 28434 rpcz_store.cc:276] 0709 12:19:03.362611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.017158 (+654547us) service_pool.cc:224] Handling call
0709 12:19:04.017465 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.057076 (+ 39611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.057111 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.057154 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:04.057183 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.803141 (+745958us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b54003ca854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:04.803198 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.803302 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90498944}
I20260709 12:19:04.828783  1010 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.844756 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:04.832593   695 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.763099 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.820324 28433 rpcz_store.cc:276] 0709 12:19:03.304576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.313394 (+  8818us) service_pool.cc:224] Handling call
0709 12:19:03.335431 (+ 22037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:03.739064 (+403633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:03.739096 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:03.739140 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:03.739170 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.819493 (+1080323us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b5400414954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:04.819553 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.819741 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24004,"raft.queue_time_us":5464,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":12397,"spinlock_wait_cycles":927397248}
I20260709 12:19:04.856195 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2771) took 1226 ms. Trace:
I20260709 12:19:04.827072 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:04.865106 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.831218 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:04.832082 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46652.2us
I20260709 12:19:04.832046 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:04.826215  1132 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:19:04.819957   729 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.834192   908 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:19:04.847931 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 79838.9us
I20260709 12:19:04.880882 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:04.822584 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:04.847070  1010 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.826174 28359 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:04.860352 28435 rpcz_store.cc:276] 0709 12:19:03.630005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.163313 (+533308us) service_pool.cc:224] Handling call
0709 12:19:04.163565 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.169700 (+  6135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.169756 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.169804 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:04.169837 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.855778 (+685941us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b5400184f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:04.855853 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.856034 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13596544}
I20260709 12:19:04.869372 28332 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:04.870052 28283 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:04.846855  1011 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:04.823925  1052 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.887542 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:04.884268 28436 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.836092   915 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.853993  1000 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:04.918270 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 349620us
I20260709 12:19:04.924240 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.928663 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:04.879572  1123 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:04.925580 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:04.930176 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:04.935530 28436 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.935812 28362 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 11: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:04.937179 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:04.871196 28257 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:04.934034 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:04.947664 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 77097.4us
I20260709 12:19:04.905102   998 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:04.846016   784 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.949877 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:04.851627  1058 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.953591 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2300) took 1578 ms. Trace:
I20260709 12:19:04.961915 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:04.962309 28361 rpcz_store.cc:276] 0709 12:19:03.375518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.379553 (+  4035us) service_pool.cc:224] Handling call
0709 12:19:03.379853 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:03.758489 (+378636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.953349 (+1194860us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54002f3ad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:04.953523 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":869192448}
I20260709 12:19:04.954062 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:04.966498 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 193018us
I20260709 12:19:04.924510  1127 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:04.970651 28441 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:19:04.942476  1081 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:04.973933  1081 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.974419  1081 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.889281   729 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:04.966614   445 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.982242   445 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.983062   915 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:04.949880  1000 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:04.983771  1000 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:04.984355   941 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:04.984973   941 consensus_queue.cc:1059] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:04.942834  1123 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.985857  1123 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:04.986244   941 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:04.936486  1084 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:04.952728   904 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:04.947800   634 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:04.957011   998 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:04.994355   729 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:05.081486 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:05.083559 28991 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:05.066792  1127 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:04.950678  1126 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.063788 28330 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:05.066726 28433 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:05.091946 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:05.051856 28290 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:04.985452  1131 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:05.055958  1084 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:05.083912 28991 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:05.072579   998 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.987221  1148 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:05.068962   634 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:05.070951   934 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:05.127606 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 349149us
W20260709 12:19:05.129566 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:05.130752 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 169010us
W20260709 12:19:05.131510 28404 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.711s (SENT) [suppressed 11 similar messages]
W20260709 12:19:05.111152 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.067490   904 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:05.101240   625 consensus_meta.cc:287] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.569s	user 0.013s	sys 0.000s
W20260709 12:19:05.200297 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 70255.9us
I20260709 12:19:05.103113  1129 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:05.117455  1134 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 12:19:05.100458   669 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.132858   634 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.104846   915 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:05.177320  1149 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:05.208087 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 76333.9us
I20260709 12:19:05.124513 28991 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.227741 28991 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:05.212653  1152 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:19:05.213364  1143 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:19:05.214078  1052 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:05.160866   902 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:05.220865 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.103039  1084 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.224359 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:05.225761 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:05.316617 28984 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:05.201267   904 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:05.227767 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:05.233199 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:05.195924   934 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:05.350270 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 122595us
I20260709 12:19:05.364267 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:05.362847 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.317044 28984 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:05.368618 28984 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.371034 28984 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:05.372502 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.321219  1131 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.366909 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 412925us
I20260709 12:19:05.388501 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:05.320991   729 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:05.351532   941 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:05.461469 28258 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.26.123.62:36149, user_credentials={real_user=slave}} blocked reactor thread for 87017.2us
I20260709 12:19:05.364436  1158 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.476253 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 87885.9us
I20260709 12:19:05.364750   902 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:05.370399   935 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:05.367120   915 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:05.466253  1143 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:19:05.244733  1149 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.493971   669 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.381440  1000 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:05.462062  1058 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:05.478574  1146 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.419s	user 0.000s	sys 0.001s
I20260709 12:19:05.467635  1125 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:05.483038  1126 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:05.516387 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.490420   948 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:05.482489  1159 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:05.516826  1125 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.524712  1167 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:05.476617   729 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.479803   941 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.498822   902 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:05.586443   729 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:05.527057   948 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.591076  1149 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:05.591760  1126 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:05.592984   948 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:05.592857  1126 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.468456  1131 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.595573  1131 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:05.584091   941 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:05.513056  1058 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:05.610663 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:05.612030 28983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 571) took 1837 ms. Trace:
I20260709 12:19:05.612494 28983 rpcz_store.cc:276] 0709 12:19:03.774418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.776743 (+  2325us) service_pool.cc:224] Handling call
0709 12:19:03.777057 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.606734 (+829677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.611755 (+1005021us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b5400762d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:05.611954 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1814209408}
I20260709 12:19:05.498255   915 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.495707   935 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.483275  1158 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:05.623035 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.596801  1058 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.628561 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:05.591120  1125 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:05.616463   695 consensus_meta.cc:287] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.002s
I20260709 12:19:05.512356   669 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.630642 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2312) took 1019 ms. Trace:
I20260709 12:19:05.640672 28285 rpcz_store.cc:276] 0709 12:19:04.611156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.611483 (+1000327us) service_pool.cc:224] Handling call
0709 12:19:05.630566 (+ 19083us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:05.646131 28404 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.035s	sys 0.029s
W20260709 12:19:05.647836 28404 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.035s	sys 0.029s
I20260709 12:19:05.619731   948 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.637466 28288 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:05.649101 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37070 (request call id 10) took 3233 ms. Trace:
I20260709 12:19:05.653234 28986 rpcz_store.cc:276] 0709 12:19:02.415279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.415446 (+   167us) service_pool.cc:224] Handling call
0709 12:19:04.588371 (+2172925us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b4c00055340. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bcb53a20 00007f85bcb64a29 00007f85bcb2a644 00007f85bcb51f2d 00007f85bcb51b93 00007f85be881c92 00007f85be88184a 00007f85c384f60a 00007f85bcdd57d4 00007f85bcdd5762 00007f85bcdd5691
0709 12:19:05.648885 (+1060514us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54004c18d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be881853 00007f85c384f60a 00007f85bcdd57d4 00007f85bcdd5762 00007f85bcdd5691 00007f85bcdd560c 00007f85bcdd48c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:05.648989 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3676466816}
I20260709 12:19:05.623823   784 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:05.629388  1143 log.cc:927] Time spent T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:19:05.658790 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:05.660779 28432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2747) took 2396 ms (client timeout 1747 ms). Trace:
W20260709 12:19:05.662245 28432 rpcz_store.cc:269] 0709 12:19:03.257551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.260147 (+  2596us) service_pool.cc:224] Handling call
0709 12:19:05.648932 (+2388785us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54002e2cd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:05.649032 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1512354048}
I20260709 12:19:05.654443   948 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:05.666057  1156 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:19:05.667711   948 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:05.546113  1159 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.676100   968 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:05.682103  1160 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:19:05.658049   934 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:05.684507   998 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.000s
I20260709 12:19:05.685030 28984 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:05.693012   935 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:05.698727 28991 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:05.691792   915 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:05.698894   688 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.700701   688 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.691744  1029 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.705869   688 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:05.598856  1167 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.707312 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2759) took 2384 ms. Trace:
W20260709 12:19:05.649885 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 518373us
I20260709 12:19:05.708125 28438 rpcz_store.cc:276] 0709 12:19:03.332740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.821229 (+488489us) service_pool.cc:224] Handling call
0709 12:19:03.821693 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:03.822228 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:04.461058 (+638830us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.819865 (+358807us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.820483 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:19:04.820638 (+   155us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:04.820872 (+   234us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.826606 (+  5734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.826698 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:05.707086 (+880388us) raft_consensus.cc:1685] finished
0709 12:19:05.707110 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.707251 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40723,"prepare.queue_time_us":151816,"prepare.run_cpu_time_us":163,"prepare.run_wall_time_us":10221,"raft.queue_time_us":86670,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":45496,"thread_start_us":834987,"threads_started":2,"wal-append.queue_time_us":482044}
I20260709 12:19:05.709575 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:05.712236 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:05.650631   695 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.705643   669 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:05.701488  1025 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.863s	user 0.008s	sys 0.005s
I20260709 12:19:05.718106 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:05.691777  1058 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:05.707315  1081 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.732s	user 0.013s	sys 0.000s
I20260709 12:19:05.716243  1125 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.738157  1081 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.738992  1125 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.741044  1081 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:05.685034   998 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.713308   999 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.725942   729 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:05.731371   695 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:05.743605   998 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:05.744370 28441 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:05.745141  1131 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:05.745579  1131 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.745965  1131 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:05.615603  1127 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:05.754359 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:05.751526  1167 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:05.758148  1025 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:05.758919   445 consensus_meta.cc:287] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.003s
W20260709 12:19:05.820798 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 466937us
I20260709 12:19:05.706103  1158 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:05.715080  1148 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
W20260709 12:19:05.725642  1011 consensus_meta.cc:287] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.817s	user 0.009s	sys 0.000s
W20260709 12:19:05.752084  1010 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.834s	user 0.009s	sys 0.004s
I20260709 12:19:05.826831 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:05.829068 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:05.741104  1125 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:05.831547 28364 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:05.831734 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 354665us
I20260709 12:19:05.829597 28284 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:05.832906 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2305) took 1234 ms. Trace:
I20260709 12:19:05.833171 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:05.833233 28284 rpcz_store.cc:276] 0709 12:19:04.598877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.826535 (+1227658us) service_pool.cc:224] Handling call
0709 12:19:05.832834 (+  6299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.834021 28358 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
W20260709 12:19:05.835621 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.714965   634 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:05.836694 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:05.837354 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:05.858763 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:05.854216 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:05.861195 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:05.821233   729 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:05.805549   998 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:05.863322 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:05.826972   695 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.017940 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.030972 28359 raft_consensus.cc:2393] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:05.903798   934 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.032127   934 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.033221   808 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:05.945204 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.034606   934 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.034600 28362 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 11.
I20260709 12:19:06.035763   695 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.904675  1167 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:19:05.952322 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:05.942902  1159 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:05.950359 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:05.948076 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:05.898684  1084 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:05.900799 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:05.905581   998 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.964812 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:05.989723  1171 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:05.913246 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.938298  1096 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.904531   688 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:06.057284   688 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.057777   688 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:06.056865  1131 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.060436 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.043839   908 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:06.061805 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:06.062999 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.063401  1096 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.064075 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:06.064509 28358 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:06.065322 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.040162   695 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:06.066162 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.067013 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.067679 28361 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:05.935281   634 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:05.942409  1025 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:05.855321   669 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.066028  1131 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.071583   669 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.000912 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:05.929359 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2425) took 1278 ms. Trace:
I20260709 12:19:06.073019 28433 rpcz_store.cc:276] 0709 12:19:04.651069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.886550 (+235481us) service_pool.cc:224] Handling call
0709 12:19:04.921828 (+ 35278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.091189 (+169361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.929063 (+837874us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540026c0d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:05.929280 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300932864}
W20260709 12:19:06.044704   904 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.679s	user 0.010s	sys 0.002s
I20260709 12:19:05.841493  1010 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:06.078601  1157 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.456s	user 0.004s	sys 0.000s
I20260709 12:19:05.949885 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:05.914958   784 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:06.082878   784 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.084149  1010 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.082661 28434 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:05.829439  1148 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:05.946889 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:05.956015 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.000007 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.912949 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2764) took 2427 ms. Trace:
I20260709 12:19:06.088883 28439 rpcz_store.cc:276] 0709 12:19:03.485187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.050292 (+565105us) service_pool.cc:224] Handling call
0709 12:19:04.050679 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.163149 (+112470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.163181 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.163224 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:04.163253 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.912663 (+1749410us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540026cad4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:05.912722 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.912887 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258193664}
W20260709 12:19:05.868289 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.939057   750 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.918001  1127 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.073822  1171 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.091013  1053 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.091559  1053 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:05.823227   445 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.027783 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2309) took 1427 ms. Trace:
I20260709 12:19:06.026927 28432 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:05.900632 28289 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:19:06.046473 28360 raft_consensus.cc:2410] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:06.047601 28287 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:06.096079 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2496) took 1423 ms. Trace:
I20260709 12:19:06.047796 28436 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
W20260709 12:19:05.999544  1123 consensus_meta.cc:287] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.013s	user 0.012s	sys 0.000s
I20260709 12:19:06.052251 28359 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:06.054937  1011 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.055363  1173 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.070979  1096 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:06.073653 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 159415us
I20260709 12:19:06.048489   808 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.045552  1170 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.074766 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:05.912032  1166 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.085497   784 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:05.937956 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 814) took 1374 ms. Trace:
I20260709 12:19:06.097003 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:06.099428  1068 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:06.097571 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:06.098213 28284 rpcz_store.cc:276] 0709 12:19:04.599885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.853917 (+1254032us) service_pool.cc:224] Handling call
0709 12:19:06.027493 (+173576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.107681 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:06.122557 28360 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:06.123574 28434 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:06.110724 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:06.123489 28365 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2312) took 1952 ms (client timeout 1819 ms). Trace:
I20260709 12:19:06.056376  1084 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:06.124243 28365 rpcz_store.cc:269] 0709 12:19:04.167319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.167503 (+   184us) service_pool.cc:224] Handling call
0709 12:19:06.119853 (+1952350us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54001d8b54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:06.119979 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555034368}
I20260709 12:19:06.126458  1084 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.126777   445 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.127663   669 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.115031  1011 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.112588  1128 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:06.113685   998 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.113852 28433 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:06.090806   934 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.062150   908 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.113102  1170 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.139585  1170 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.139699   908 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.137877   934 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.133160   695 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:06.141304 28360 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:06.116039  1173 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.141790   695 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.142699   934 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.143728  1173 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.143990   695 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.076510   904 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.105901  1123 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.095410  1131 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.096438 28287 rpcz_store.cc:276] 0709 12:19:04.672620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.947790 (+1275170us) service_pool.cc:224] Handling call
0709 12:19:06.096012 (+148222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.119655 28290 rpcz_store.cc:276] 0709 12:19:04.563806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.586880 (+ 23074us) service_pool.cc:224] Handling call
0709 12:19:05.937705 (+1350825us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54002413d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:05.937888 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066390912}
I20260709 12:19:06.144771  1053 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.116462 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:06.181285 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65297.8us
I20260709 12:19:06.133288  1096 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:06.122021  1068 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:19:06.144968 28257 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:59778 dropped due to backpressure: service queue is full with 50 items
I20260709 12:19:06.149302 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.134195 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.258330 28284 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 9.
I20260709 12:19:06.258823 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2306) took 1659 ms. Trace:
I20260709 12:19:06.259104 28284 rpcz_store.cc:276] 0709 12:19:04.599118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.133888 (+1534770us) service_pool.cc:224] Handling call
0709 12:19:06.258764 (+124876us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.256671 28292 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:19:06.257557 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.258147 28287 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:06.261013 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2502) took 1572 ms. Trace:
I20260709 12:19:06.261483 28287 rpcz_store.cc:276] 0709 12:19:04.688499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.149000 (+1460501us) service_pool.cc:224] Handling call
0709 12:19:06.260947 (+111947us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.262681 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.264786 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.265161 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:06.115860   808 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.266654 28284 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:06.136610  1171 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:06.264043 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 152562us
I20260709 12:19:06.268769 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:06.133569  1128 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:06.271976  1123 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.272773  1068 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.146097   883 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:06.274928 28285 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:06.279148 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2297) took 1682 ms. Trace:
I20260709 12:19:06.281724 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:06.281898 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.118898  1166 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.256065   908 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:06.260954   998 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:06.285725 28284 rpcz_store.cc:276] 0709 12:19:04.596604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.264509 (+1667905us) service_pool.cc:224] Handling call
0709 12:19:06.279085 (+ 14576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.286334 28290 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:06.286985 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:06.288942   634 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:06.290962   729 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:06.146382  1131 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.267989   695 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:06.291005   634 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:06.294961   634 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.276906  1025 consensus_queue.cc:1059] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:19:06.296528  1025 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:06.296924  1025 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:06.297827  1025 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.299075  1025 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.283525  1053 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:06.280556  1058 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:19:06.302134  1053 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:06.302552  1058 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:06.302614  1053 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:06.303069  1058 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:06.303998  1058 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.305220  1058 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:06.280747  1167 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.306203  1167 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.265575  1096 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.294312 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:06.310307  1096 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.313331   908 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.292794  1068 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:06.315626   908 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.312785   904 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.313107  1053 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.323557 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:06.326982 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:06.284878   883 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:06.329025 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:06.331521 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:06.330827   901 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:06.325855   998 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:06.332641   998 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:06.333043   998 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.333285   808 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.329924  1084 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:06.335338  1178 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:06.335668 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 863) took 1115 ms. Trace:
I20260709 12:19:06.336134 28441 rpcz_store.cc:276] 0709 12:19:05.220078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.220224 (+   146us) service_pool.cc:224] Handling call
0709 12:19:05.220549 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.744811 (+524262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.335399 (+590588us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b5400415354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:06.335595 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204207488}
I20260709 12:19:06.337199  1131 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:06.340039 28292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2314) took 1644 ms (client timeout 1951 ms). Trace:
I20260709 12:19:06.330909   729 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:06.331410  1128 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.342742  1128 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:06.343025 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.340570 28292 rpcz_store.cc:269] 0709 12:19:04.695658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.927107 (+231449us) service_pool.cc:224] Handling call
0709 12:19:06.339796 (+1412689us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003db3d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:06.339960 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2986040576}
I20260709 12:19:06.345826   729 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:06.346949   729 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:06.347402   729 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:06.257054 28257 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:59778 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2319)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2327)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2326)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2325)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2323)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2333)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2330)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2331)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2511)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2332)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2335)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2328)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2296)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 812)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 815)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2494)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2495)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2334)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2505)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2508)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2509)
I20260709 12:19:06.353852   883 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.355142   883 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.690750 28286 rpcz_store.cc:276] 0709 12:19:02.935156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.935318 (+   162us) service_pool.cc:224] Handling call
0709 12:19:02.935616 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.935824 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:04.340535 (+1404711us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.670471 (+329936us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.670547 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:04.670604 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.670689 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.670965 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":864974,"prepare.queue_time_us":1009185,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":44445,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":7412,"thread_start_us":841570,"threads_started":2}
I20260709 12:19:06.363970 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:06.364353 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.364893 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:06.365118 28360 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:06.366848 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:06.367471 28984 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:19:06.351388 28257 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:59778 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2319)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2327)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2326)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2325)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2323)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2333)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2330)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2331)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2511)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2332)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2514)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2517)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2335)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2328)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2296)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 812)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 815)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2494)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2495)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2334)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2505)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2508)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2509)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2515)
I20260709 12:19:06.369105 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.370648 28440 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:06.372802 28439 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:06.373422 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.372694 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.374579 28435 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:19:06.375473 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.376044 28440 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:19:06.378206 28433 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:06.380077 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.380525 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.381819 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.382438 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.383244 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.384744 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.384833 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:06.385565 28365 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:06.387898 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.389254 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.389912  1131 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:06.390559 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.390674 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:06.391351 28990 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:06.392179 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:06.392946 28434 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:06.393311 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:06.395170  1096 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:06.395578  1042 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.396332 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.390465  1131 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:06.336700  1166 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.391309 28987 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:06.386096 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:06.400051 28365 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:06.408787   634 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.398236   729 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.417021   729 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.418947  1084 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:06.418668   634 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.419589  1084 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:06.410466  1009 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:06.398224  1162 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.772s	user 0.003s	sys 0.006s
I20260709 12:19:06.413842 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:06.325007   695 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.421814 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:06.411226   669 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.424798 28985 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:19:06.424966  1116 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:06.428546 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.444693 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.445770 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.446594 28359 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:06.446403  1131 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.447468 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.448091 28359 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:06.449013 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.444439   669 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.450508 28364 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:19:06.451282 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.378019   892 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:06.438110 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.395583  1096 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.438992 28985 raft_consensus.cc:606] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:06.454185 28985 raft_consensus.cc:614] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:06.448709  1084 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.450917 28364 pending_rounds.cc:85] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 8
I20260709 12:19:06.455404 28364 pending_rounds.cc:107] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:19:06.455663 28364 raft_consensus.cc:2889] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:06.430298 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:06.465723 28365 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:06.463409   634 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:06.466979 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 53991.4us
I20260709 12:19:06.451717   952 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:06.438817  1050 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:06.448299   968 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:06.439059  1011 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.472121 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:06.423482   688 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:06.473423 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.474486 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.476778 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.477845 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.476842 28984 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:06.481171 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:06.466598  1166 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.478818 28367 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:19:06.474902 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44701.9us
I20260709 12:19:06.463924  1096 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:06.484740 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:06.484935 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:06.484246 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.471464   952 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.491122 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:06.491784 28991 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 21: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:06.489023 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.492902 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:19:06.493359 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.491911 28990 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:06.473999  1050 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.478094   901 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:06.477898   968 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.494206   952 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.469300   892 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.500018 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.471882   634 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.497352 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:06.498607 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:06.504268 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 113591us
I20260709 12:19:06.499194 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 699) took 1825 ms. Trace:
I20260709 12:19:06.501468 28367 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:06.508872 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:06.450917 28258 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39592 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2333)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2330)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2331)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2511)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2332)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2514)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2335)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2328)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2296)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 812)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 815)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2494)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2495)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2334)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2505)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2508)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2509)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2515)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2518)
I20260709 12:19:06.490829  1011 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:06.512840 28258 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:39592 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2333)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2330)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2331)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2511)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2332)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2514)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2335)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2328)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2296)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 812)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 815)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2494)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2495)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2334)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2505)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2508)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2509)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2515)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2518)
I20260709 12:19:06.494973  1042 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:06.518246 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.519613 28990 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:19:06.520262 28990 raft_consensus.cc:606] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:06.517680 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.524359 28987 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:06.498047   901 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.531746   901 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:06.532030 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:06.533030  1181 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:06.533913  1180 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:06.534922   565 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:06.535305   565 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:06.536407   565 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.538146 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.539218   892 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.538156   904 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:06.534395  1181 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:06.540417  1181 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.540308   904 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.540896  1181 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.539486 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.542577 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.545519   565 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:06.502509  1053 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.546866  1053 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:06.498237   729 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:06.498054  1050 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.538575 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.535214  1180 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:06.550516  1180 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:06.551628  1180 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.552206 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.553277  1180 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.533810  1011 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.555249   634 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.556133   928 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.556700  1011 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.497898  1005 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:06.486346  1166 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.559571  1042 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.561935  1042 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:06.564316 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.564837 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.550047 28367 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:06.566401   669 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.533486 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:06.513499  1054 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:06.570042   858 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:06.570533   858 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.570916   858 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.545756   904 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.509459 28366 rpcz_store.cc:276] 0709 12:19:04.674079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.696668 (+ 22589us) service_pool.cc:224] Handling call
0709 12:19:04.697258 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.703898 (+  6640us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:05.055466 (+351568us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.658235 (+602769us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:05.673340 (+ 15105us) log.cc:844] Serialized 494 byte log entry
0709 12:19:05.673521 (+   181us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:05.673557 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.673632 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:06.361412 (+687780us) spinlock_profiling.cc:243] Waited 61.9 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:06.497757 (+136345us) raft_consensus.cc:1685] finished
0709 12:19:06.498966 (+  1209us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.499123 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8498,"prepare.queue_time_us":386471,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":5027,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":10484,"spinlock_wait_cycles":142263552,"thread_start_us":953597,"threads_started":3,"wal-append.queue_time_us":466199}
I20260709 12:19:06.575579 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.612476   948 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:06.604961 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:06.581972 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:06.518373 28258 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:39592 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2333)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2330)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2331)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2511)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2332)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2514)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2335)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2328)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2296)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 812)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 815)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2494)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2495)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2334)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2505)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2508)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2509)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2515)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2518)
I20260709 12:19:06.619124 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.645098 28956 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:06.649515 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.648610 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.656924 28366 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:06.657231 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.574817   695 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.613948   792 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:06.614269   904 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.674791 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:06.657229   892 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:06.656706   565 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:06.677544   688 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.685209  1005 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:06.706779 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.708918 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:06.685451 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 188180us
I20260709 12:19:06.684535  1011 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:19:06.673771   792 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:06.710484 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:06.711367 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:06.715364 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:06.717197 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.722322 28982 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
W20260709 12:19:06.728655 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.728816 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:06.733559 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.734249 28283 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:06.701993  1005 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.679563   904 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:06.740665 28288 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2294) took 2226 ms (client timeout 1711 ms). Trace:
I20260709 12:19:06.724114   808 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:06.738225 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:06.741084 28288 rpcz_store.cc:269] 0709 12:19:04.514202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.547234 (+ 33032us) service_pool.cc:224] Handling call
0709 12:19:06.740447 (+2193213us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54000b1dd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:06.740574 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1452576128}
I20260709 12:19:06.739032 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:06.712621   695 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.725497  1131 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:06.742753 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.747864 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.725945  1048 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:06.686328   998 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.751144 28434 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:06.751353 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.751564  1048 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.751989  1048 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.756081  1136 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:06.756580  1136 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:06.733238  1042 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:06.757481  1136 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.757975  1042 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:06.758409  1042 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.758567  1136 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.760599   901 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:06.762470 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.763795 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.765044 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:06.765547 28438 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:06.771268   904 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.771930   904 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:06.774192  1166 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.776057   998 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:06.777530 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.777825  1184 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.744325 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:06.778365  1184 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.778927 28283 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:06.779036 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:06.740061  1005 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:06.781948  1052 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.782469  1052 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.745091   808 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.786005   808 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:06.749832   695 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:06.790728 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.772420   904 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.792173 28287 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.795691  1096 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.798326  1096 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:06.800540  1142 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.694s	user 0.017s	sys 0.020s
W20260709 12:19:06.808303 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 147734us
W20260709 12:19:06.813125 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.814145 28283 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:06.814819 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:06.815210 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:06.819993   858 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:06.822293 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.824162  1184 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.823357  1080 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:06.478585 28257 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:59778 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2333)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2330)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2331)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 821)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2511)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2332)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2514)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2335)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2328)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 822)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2491)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 809)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 810)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 811)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2296)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 812)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 813)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 815)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 816)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 817)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2494)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2495)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 818)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 820)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2334)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2505)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2508)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2509)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2515)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2518)
I20260709 12:19:06.825280 28363 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:06.825984   908 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:06.806685   792 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.831972  1009 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.832299   968 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.834453 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:06.835495   695 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.840869 28362 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:06.841439   915 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.841979   915 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.844609 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.845238 28432 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:06.849349 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.851596 28439 raft_consensus.cc:2410] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:06.853629 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.855561 28433 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:06.859786 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:06.859753 28441 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:06.873466 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.874511 28441 raft_consensus.cc:2410] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:19:06.906355 28432 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 19: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:19:06.926590 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 66928.1us
I20260709 12:19:06.928237 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:06.931102 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.932142 28441 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:06.933501 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:06.907374 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.935501 28432 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:06.935599 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.936861 28433 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:06.937785 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:19:06.938522 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.939184 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:06.940625 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.942029 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.942585 28441 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:06.942720 28432 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:19:06.943926 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.807916  1181 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.854519  1123 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.947921 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:06.905686  1131 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:06.950176 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.933420  1084 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:06.954980 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:06.934268   998 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.925832  1025 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.845580  1050 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.939777  1184 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.905275   952 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.942261   729 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:06.941187  1053 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:06.853856  1009 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.963454   968 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.965396   904 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.965310   858 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.958109   634 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:06.973662  1184 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.989771   915 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.976765  1050 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.994760   634 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:06.970496   998 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.995916  1009 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.995810   634 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.996891   998 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.974165  1005 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.976635  1167 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.968806  1025 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.996803  1184 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.971982   729 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:06.981703  1181 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:06.982321  1042 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:06.985140  1052 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:07.000074   729 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:06.961894  1131 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:06.997008   808 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.002111  1131 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.989682   952 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:07.003124   808 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:06.961931  1058 consensus_meta.cc:287] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.654s	user 0.013s	sys 0.000s
I20260709 12:19:06.999338  1005 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:06.999603  1025 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:06.959023  1123 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.006186  1042 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:07.006837  1123 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:07.017987  1163 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.387s	user 0.010s	sys 0.000s
W20260709 12:19:07.049502 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 142144us
W20260709 12:19:07.051460 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 335998us
I20260709 12:19:07.052431 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:07.060696 28288 raft_consensus.cc:2393] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:07.061340 28283 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
W20260709 12:19:07.081146 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 262472us
I20260709 12:19:06.965821 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:06.961081 28954 leader_election.cc:336] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:39592 dropped due to backpressure: service queue is full with 50 items
W20260709 12:19:07.097018 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.131637  1080 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:40336 (request call id 9) took 4585 ms. Trace:
I20260709 12:19:07.134383   808 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:07.138393   904 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:07.139112   904 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:07.139585 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 203026us
I20260709 12:19:07.140002 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.141106 28361 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:07.143539 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.143831 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:07.126654 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2779) took 2420 ms. Trace:
W20260709 12:19:07.147488 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.145943 28437 rpcz_store.cc:276] 0709 12:19:04.706094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.921261 (+215167us) service_pool.cc:224] Handling call
0709 12:19:04.921678 (+   417us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:04.922214 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.922244 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:04.955682 (+ 33438us) log.cc:844] Serialized 52 byte log entry
0709 12:19:05.346083 (+390401us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:05.346127 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.346216 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.017264 (+1671048us) spinlock_profiling.cc:243] Waited 45.5 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:07.125887 (+108623us) raft_consensus.cc:1685] finished
0709 12:19:07.125939 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.126432 (+   493us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":11265,"spinlock_wait_cycles":104561408,"thread_start_us":272616,"threads_started":1,"wal-append.queue_time_us":456861}
I20260709 12:19:07.134595 28441 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:07.139608   904 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:07.157574 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 199476us
I20260709 12:19:07.163425 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:07.164574 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:07.166491 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:07.167507 28285 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2501) took 2485 ms (client timeout 2008 ms). Trace:
I20260709 12:19:07.168399 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:07.170975 28359 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:07.173370 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:07.166414 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.174711 28982 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:07.175346 28364 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:07.146049 28433 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:07.149713 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.003819  1048 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.187714 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.188932 28433 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:07.190132 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.190889 28437 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:07.132380  1080 rpcz_store.cc:276] 0709 12:19:02.545663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.545840 (+   177us) service_pool.cc:224] Handling call
0709 12:19:02.901221 (+355381us) log.cc:844] Serialized 494 byte log entry
0709 12:19:03.172972 (+271751us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b540071e654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88d735 00007f85be8859fa 00007f85c384d4c9 00007f85bcdc9c54 00007f85bcdc9be2 00007f85bcdc9b11 00007f85bcdc9a8c 00007f85bcdc8d42 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:07.131227 (+3958255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2013,"raft.run_cpu_time_us":1150,"raft.run_wall_time_us":1168,"spinlock_wait_cycles":743703808}
W20260709 12:19:07.198691 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 55086.2us
I20260709 12:19:07.200093 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:07.201197 28982 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:07.202203  1149 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:07.204370  1048 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.163678 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:07.205385  1166 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:07.170892 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:07.206831 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 114fa146124747358c997ac1008e1b93 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 12:19:07.207909 28990 raft_consensus.cc:2368] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:19:07.208739 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:07.209827 28984 raft_consensus.cc:2368] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:19:07.211918 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.212863 28361 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:07.191118 28364 raft_consensus.cc:2368] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:19:07.215874 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.217041 28433 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:07.218271 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.219887 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.221040 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.222101 28436 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:07.222626 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.223107 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.223495 28364 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:07.205799 28439 raft_consensus.cc:2368] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14.
W20260709 12:19:07.180625 28285 rpcz_store.cc:269] 0709 12:19:04.682218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.641719 (+959501us) service_pool.cc:224] Handling call
0709 12:19:07.166897 (+1525178us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b5400059bd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:07.167202 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1392768384}
I20260709 12:19:07.223872 28436 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:07.229573 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.229780 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.231046 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.234119 28433 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:19:07.236742 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.237176 28433 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: [16.7-16.7]
I20260709 12:19:07.228782 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.228782 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:07.237673 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.254822 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:07.231046 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.246369 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2327) took 1299 ms. Trace:
W20260709 12:19:07.247713 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 195662us
I20260709 12:19:07.245139 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2331) took 1217 ms. Trace:
I20260709 12:19:07.263787 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2507) took 1127 ms. Trace:
I20260709 12:19:07.264098 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.265430 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.265828 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.266541 28286 raft_consensus.cc:2368] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15.
I20260709 12:19:07.267160 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2325) took 1192 ms. Trace:
I20260709 12:19:07.267556 28286 rpcz_store.cc:276] 0709 12:19:06.075130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.264726 (+1189596us) service_pool.cc:224] Handling call
0709 12:19:07.267058 (+  2332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.268567 28288 rpcz_store.cc:276] 0709 12:19:06.136736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.747581 (+610845us) service_pool.cc:224] Handling call
0709 12:19:07.263508 (+515927us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540009f1d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:07.263694 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711218432}
W20260709 12:19:07.270501 28289 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 819) took 2327 ms (client timeout 1537 ms). Trace:
I20260709 12:19:07.271292 28364 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:07.270977 28289 rpcz_store.cc:269] 0709 12:19:04.943350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.610334 (+666984us) service_pool.cc:224] Handling call
0709 12:19:07.270143 (+1659809us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b540009ecd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:07.270315 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512928000}
I20260709 12:19:07.279198 28433 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.257544 28358 rpcz_store.cc:276] 0709 12:19:05.947091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.950117 (+  3026us) service_pool.cc:224] Handling call
0709 12:19:07.245881 (+1295764us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b5400126e54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:07.246115 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39610752}
I20260709 12:19:07.280937 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.270889 28434 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.260658 28362 rpcz_store.cc:276] 0709 12:19:06.045293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.050442 (+  5149us) service_pool.cc:224] Handling call
0709 12:19:07.244623 (+1194181us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b54003bb5d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:07.244871 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1781938944}
I20260709 12:19:07.283139 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.283710 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.282425 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.285172 28436 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:07.286504 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 57085.7us
I20260709 12:19:07.288247 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:07.289127 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.291288 28185 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.292357 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:07.294600 28287 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.500s	user 0.004s	sys 0.021s
I20260709 12:19:07.289978 28439 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:07.295241 28287 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:07.294915 28359 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:07.297109 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.298787 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.300623 28359 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:07.302896 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.303961 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2327) took 1217 ms. Trace:
I20260709 12:19:07.304332 28291 rpcz_store.cc:276] 0709 12:19:06.086321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.302298 (+1215977us) service_pool.cc:224] Handling call
0709 12:19:07.303868 (+  1570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.296066 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2330) took 1199 ms. Trace:
I20260709 12:19:07.306528 28287 rpcz_store.cc:276] 0709 12:19:06.097022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.764787 (+667765us) service_pool.cc:224] Handling call
0709 12:19:07.295958 (+531171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.308427 28982 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.309899 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:07.310716 28984 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:07.311371 28436 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:07.312263 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:07.313139 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2434) took 1452 ms. Trace:
I20260709 12:19:07.313580  1195 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.314419 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:07.315012 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:07.315886   808 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:07.316412   808 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:07.316874   808 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.316077 28991 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:19:07.317833 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.318754 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.319986   792 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:07.314319 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:19:07.322105 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.323908 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:07.325184 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:07.326128 28290 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:07.325395  1111 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:07.306519 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.313584 28438 rpcz_store.cc:276] 0709 12:19:05.860767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.860930 (+   163us) service_pool.cc:224] Handling call
0709 12:19:07.312784 (+1451854us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54000d8c54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:07.312926 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2078137088}
I20260709 12:19:07.244390  1048 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:07.348925 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.352557 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:07.306212 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:07.358254 28404 leader_election.cc:400] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:19:07.358777 28404 leader_election.cc:403] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:19:07.359782 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.360803 28436 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:07.362237 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.362680  1191 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:07.363153  1191 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.363520  1191 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:07.365923 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.366618 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 75859.8us
I20260709 12:19:07.367619 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:07.370113   928 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.372339 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.373165 28438 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:07.373878 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.374632 28439 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:07.376503  1048 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.377956 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.378751 28434 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:07.379063 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:07.379305  1048 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:07.380132  1188 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:07.380585  1188 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:07.381031  1188 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.383819 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.384557 28284 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:07.385002 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2326) took 1308 ms. Trace:
I20260709 12:19:07.380638 28285 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:07.389122 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2322) took 1332 ms. Trace:
I20260709 12:19:07.389467 28285 rpcz_store.cc:276] 0709 12:19:06.056714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.361868 (+1305154us) service_pool.cc:224] Handling call
0709 12:19:07.389033 (+ 27165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.390691 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:07.391395 28285 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.362138 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.394636 28433 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:07.396155 28433 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:07.380125 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.360906 28990 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:07.385286 28284 rpcz_store.cc:276] 0709 12:19:06.082138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.383341 (+1301203us) service_pool.cc:224] Handling call
0709 12:19:07.384947 (+  1606us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:07.449791 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.456674 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:07.401201   792 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:19:07.474344 28289 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:07.416564 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.474898 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2318) took 1422 ms. Trace:
I20260709 12:19:07.475286 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.399255   729 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.448415 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.413183 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.476565 28441 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:19:07.447592 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:07.417477 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.447216  1068 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:19:07.478302 28286 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:19:07.478895 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2321) took 1425 ms. Trace:
W20260709 12:19:07.427063 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 57776.6us
I20260709 12:19:07.398471  1166 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.413717 28185 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.480031  1166 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.481331 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:07.435065 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:07.475663 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2511) took 1331 ms. Trace:
I20260709 12:19:07.446373 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:07.422592 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 71656.6us
I20260709 12:19:07.453030 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:07.470366 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.400054  1195 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.449352 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:07.447461 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.412230 28367 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:07.410815   750 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:07.485946 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.486706 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.486847 28288 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:07.487583 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2331) took 1388 ms. Trace:
I20260709 12:19:07.482404 28283 rpcz_store.cc:276] 0709 12:19:06.144371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.416219 (+1271848us) service_pool.cc:224] Handling call
0709 12:19:07.475571 (+ 59352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.398757   901 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:07.422881   634 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:19:07.481618   729 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:07.489274   904 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.490202 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 821) took 1821 ms. Trace:
I20260709 12:19:07.484580  1111 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:07.491771  1111 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:07.491899   904 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:07.446221 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 67330.5us
I20260709 12:19:07.492018  1154 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.493059  1111 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.493264  1154 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.489506 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.475368 28289 rpcz_store.cc:276] 0709 12:19:06.052905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.456151 (+1403246us) service_pool.cc:224] Handling call
0709 12:19:07.474829 (+ 18678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.494624 28283 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:07.495146 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.484580  1195 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:07.496131  1154 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.496692 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:07.483544 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.484419 28291 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:07.493978 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:07.497941 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2320) took 1444 ms. Trace:
I20260709 12:19:07.490810 28436 raft_consensus.cc:2393] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:07.497524  1154 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:19:07.499253 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.499720  1184 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:07.500171  1184 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:07.498415 28291 rpcz_store.cc:276] 0709 12:19:06.053289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.448954 (+1395665us) service_pool.cc:224] Handling call
0709 12:19:07.497875 (+ 48921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.500806  1184 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.500159  1172 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:07.499311  1048 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.485031 28292 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:19:07.501685  1172 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:19:07.502277 28292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2506) took 1369 ms (client timeout 1816 ms). Trace:
W20260709 12:19:07.502801 28292 rpcz_store.cc:269] 0709 12:19:06.132466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.380403 (+1247937us) service_pool.cc:224] Handling call
0709 12:19:07.502220 (+121817us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.502460  1048 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:07.503966 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.500864  1052 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:07.504680  1127 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:07.504952  1052 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.505669  1052 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:07.506882   948 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:07.501600 28287 rpcz_store.cc:276] 0709 12:19:05.668480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.412825 (+1744345us) service_pool.cc:224] Handling call
0709 12:19:07.490106 (+ 77281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.508940 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.509341   948 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.503357  1172 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.505098  1127 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:07.474728   792 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:07.500705 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.468999  1191 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.447269 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.405324 28984 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
W20260709 12:19:07.459308 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.509692 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2523) took 1082 ms. Trace:
I20260709 12:19:07.523353 28287 rpcz_store.cc:276] 0709 12:19:06.427845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.508570 (+1080725us) service_pool.cc:224] Handling call
0709 12:19:07.509638 (+  1068us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:07.523468 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 76347.2us
I20260709 12:19:07.501474 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2521) took 1079 ms. Trace:
I20260709 12:19:07.495139 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2512) took 1145 ms. Trace:
I20260709 12:19:07.520906 28364 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:19:07.488547 28288 rpcz_store.cc:276] 0709 12:19:06.099350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.447960 (+1348610us) service_pool.cc:224] Handling call
0709 12:19:07.487510 (+ 39550us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.479226 28286 rpcz_store.cc:276] 0709 12:19:06.053499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.417115 (+1363616us) service_pool.cc:224] Handling call
0709 12:19:07.478823 (+ 61708us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.512588 28289 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.512265 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:07.526587 28284 rpcz_store.cc:276] 0709 12:19:06.422376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.474919 (+1052543us) service_pool.cc:224] Handling call
0709 12:19:07.501410 (+ 26491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.527230 28283 rpcz_store.cc:276] 0709 12:19:06.349514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.489146 (+1139632us) service_pool.cc:224] Handling call
0709 12:19:07.495056 (+  5910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.505007 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2524) took 1067 ms. Trace:
I20260709 12:19:07.533824 28292 rpcz_store.cc:276] 0709 12:19:06.437264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.503631 (+1066367us) service_pool.cc:224] Handling call
0709 12:19:07.504950 (+  1319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.482930  1166 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:07.479230 28189 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 219) took 42214 ms (client timeout 15000 ms). Trace:
I20260709 12:19:07.510027   948 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.537623 28285 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:07.511188  1172 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.510094  1181 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:07.523084   729 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:07.511557 28290 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:07.543089 28255 leader_election.cc:400] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:19:07.543956 28255 leader_election.cc:403] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:19:07.546279 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2332) took 1437 ms. Trace:
I20260709 12:19:07.546764 28285 rpcz_store.cc:276] 0709 12:19:06.103458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.390266 (+1286808us) service_pool.cc:224] Handling call
0709 12:19:07.541049 (+150783us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.481115 28185 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 4 to 14. New cstate: current_term: 14 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.530526 28291 raft_consensus.cc:2376] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11. Re-sending same reply.
I20260709 12:19:07.511982   792 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:07.552273 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:07.547469 28290 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2319) took 1490 ms (client timeout 1777 ms). Trace:
I20260709 12:19:07.546362   928 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:07.555962   765 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:07.556557   765 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:07.557049   765 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.558121   948 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:07.537961 28189 rpcz_store.cc:269] 0709 12:18:25.265337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.265747 (+   410us) service_pool.cc:224] Handling call
0709 12:19:07.478981 (+42213234us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:56.993532 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:57.041342 (+ 47810us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:57.041405 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:18:57.041442 (+    37us) tablet.cc:662] Decoding operations
0709 12:18:57.042583 (+  1141us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:57.042706 (+   123us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:57.042735 (+    29us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:18:57.043574 (+   839us) tablet.cc:701] Row locks acquired
0709 12:18:57.043640 (+    66us) write_op.cc:260] PREPARE: finished
0709 12:18:57.050035 (+  6395us) write_op.cc:270] Start()
0709 12:18:57.145064 (+ 95029us) write_op.cc:276] Timestamp: P: 1783599537043752 usec, L: 0
0709 12:18:57.145395 (+   331us) op_driver.cc:348] REPLICATION: starting
0709 12:18:57.148115 (+  2720us) log.cc:844] Serialized 3609 byte log entry
0709 12:18:58.078226 (+930111us) op_driver.cc:464] REPLICATION: finished
0709 12:18:58.362174 (+283948us) write_op.cc:301] APPLY: starting
0709 12:18:58.417189 (+ 55015us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:58.496588 (+ 79399us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:58.507606 (+ 11018us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:59.219883 (+712277us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b1800323fe0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be5c60ec 00007f85be612c0a 00007f85be6124f7 00007f85be612490 00007f85be5c9f48 00007f85c22de1bb 00007f85c23255d4 00007f85c21d18da 00007f85c21d43cb
0709 12:18:59.221293 (+  1410us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:59.221618 (+   325us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 12:18:59.221685 (+    67us) write_op.cc:312] APPLY: finished
0709 12:18:59.425052 (+203367us) log.cc:844] Serialized 89 byte log entry
0709 12:18:59.561130 (+136078us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:59.578139 (+ 17009us) write_op.cc:454] Released schema lock
0709 12:18:59.583888 (+  5749us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:59.584658 (+   770us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:59.584726 (+    68us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1161519,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":252322,"apply.run_cpu_time_us":35446,"apply.run_wall_time_us":1236692,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41729,"num_ops":9,"prepare.queue_time_us":266937,"prepare.run_cpu_time_us":46801,"prepare.run_wall_time_us":338331,"raft.queue_time_us":282406,"raft.run_cpu_time_us":12804,"raft.run_wall_time_us":126510,"replication_time_us":930316,"spinlock_wait_cycles":2874752,"thread_start_us":643346,"threads_started":5,"wal-append.queue_time_us":579858,"wal-append.run_cpu_time_us":24895,"wal-append.run_wall_time_us":2514532}]]}
W20260709 12:19:07.554178 28290 rpcz_store.cc:269] 0709 12:19:06.053067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.324544 (+1271477us) service_pool.cc:224] Handling call
0709 12:19:07.543568 (+219024us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.562764 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:07.509896  1068 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.475832  1188 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:07.566816 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 46894.1us
I20260709 12:19:07.546689  1181 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:07.546586   729 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.566141 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:07.512161  1127 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.511629  1184 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.572633 28287 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:07.574580 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.575412 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.581007 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2335) took 1464 ms. Trace:
I20260709 12:19:07.581434 28287 rpcz_store.cc:276] 0709 12:19:06.116847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.527487 (+1410640us) service_pool.cc:224] Handling call
0709 12:19:07.580936 (+ 53449us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.571635   904 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:07.572297   729 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:07.588627 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.580106  1166 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:07.598659 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:07.599001 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.598578   908 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:07.600157 28288 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:07.599715   908 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.601022   908 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:07.580125   808 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.603963   808 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.610003   901 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:07.610678 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:07.584836   948 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.600692 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2328) took 1506 ms. Trace:
I20260709 12:19:07.614383 28288 rpcz_store.cc:276] 0709 12:19:06.094557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.562318 (+1467761us) service_pool.cc:224] Handling call
0709 12:19:07.600628 (+ 38310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.599761 28359 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.599092 28332 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:19:07.617019 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 65633.9us
I20260709 12:19:07.599002  1188 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.621551 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.582300  1068 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.627688  1136 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:07.628329  1136 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.628782  1136 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.628072 28185 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 8 to 14, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 14 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.629233 28332 leader_election.cc:400] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14.
I20260709 12:19:07.629849 28332 leader_election.cc:403] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:19:07.630894 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.631784 28288 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:19:07.632246 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:07.633407 28288 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.633569   915 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.634130  1181 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.635134 28291 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:07.638113 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.641577  1136 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.643421   792 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.657763   792 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.658565 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 810) took 3106 ms. Trace:
I20260709 12:19:07.658931 28288 rpcz_store.cc:276] 0709 12:19:04.543567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.632787 (+3089220us) service_pool.cc:224] Handling call
0709 12:19:07.633203 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.649692 (+ 16489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.649917 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.661310 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 811) took 3115 ms. Trace:
I20260709 12:19:07.630366 28185 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.626863 28292 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:07.641539 28286 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.647925 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:07.655510  1191 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:07.630272  1068 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.667773 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:07.668949 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2492) took 3388 ms. Trace:
I20260709 12:19:07.669360 28286 rpcz_store.cc:276] 0709 12:19:04.280154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.640919 (+3360765us) service_pool.cc:224] Handling call
0709 12:19:07.641324 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.668671 (+ 27347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.668884 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.671082 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2295) took 3149 ms. Trace:
I20260709 12:19:07.671494 28291 rpcz_store.cc:276] 0709 12:19:04.514514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.571222 (+3056708us) service_pool.cc:224] Handling call
0709 12:19:07.611838 (+ 40616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.663458 (+ 51620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.663681 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.656158 28956 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:07.656752  1136 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.661634 28288 rpcz_store.cc:276] 0709 12:19:04.545510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.660287 (+3114777us) service_pool.cc:224] Handling call
0709 12:19:07.660681 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.660938 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.660960 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:07.661000 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:07.661026 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.661131 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.661235 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.662520 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:07.673661 28285 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:07.674083 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 822) took 1561 ms. Trace:
I20260709 12:19:07.666973 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.674649 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 809) took 3131 ms. Trace:
I20260709 12:19:07.674403 28285 rpcz_store.cc:276] 0709 12:19:06.112520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.614630 (+1502110us) service_pool.cc:224] Handling call
0709 12:19:07.674018 (+ 59388us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:07.666889 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 305502us
I20260709 12:19:07.667029  1188 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:07.677562 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 142079us
I20260709 12:19:07.676690 28283 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:19:07.678958 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2522) took 1252 ms. Trace:
I20260709 12:19:07.679288 28283 rpcz_store.cc:276] 0709 12:19:06.426729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.614975 (+1188246us) service_pool.cc:224] Handling call
0709 12:19:07.678884 (+ 63909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.675021 28287 rpcz_store.cc:276] 0709 12:19:04.543258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.582232 (+3038974us) service_pool.cc:224] Handling call
0709 12:19:07.619557 (+ 37325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.667257 (+ 47700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.667286 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:07.674391 (+  7105us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:07.674435 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.674495 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.674595 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.682336 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:07.620138  1052 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:07.690759 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.610679   908 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.687863 28360 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:07.682919  1127 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:07.697288 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.697271 28291 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.698482 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 813) took 3148 ms. Trace:
I20260709 12:19:07.698823 28291 rpcz_store.cc:276] 0709 12:19:04.549675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.677224 (+3127549us) service_pool.cc:224] Handling call
0709 12:19:07.697060 (+ 19836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.698214 (+  1154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.698424 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.700196 28291 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:07.700270 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.700968 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 817) took 3135 ms. Trace:
I20260709 12:19:07.701325 28291 rpcz_store.cc:276] 0709 12:19:04.565047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.699610 (+3134563us) service_pool.cc:224] Handling call
0709 12:19:07.699982 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.700694 (+   712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.700886 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.687793   948 raft_consensus.cc:2764] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:19:07.705961   925 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.706508   925 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.706964 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:07.707172   935 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.707772   935 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.708411  1136 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:07.708910  1136 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:07.688613   928 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.711119 28286 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.706889 28285 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:19:07.712366 28292 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:19:07.693349  1181 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:07.714289 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 816) took 3149 ms. Trace:
I20260709 12:19:07.716004 28287 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:19:07.716249 28285 rpcz_store.cc:276] 0709 12:19:04.564766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.689943 (+3125177us) service_pool.cc:224] Handling call
0709 12:19:07.706451 (+ 16508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:07.713941 (+  7490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.714231 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.717412 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2494) took 3045 ms. Trace:
I20260709 12:19:07.719101 28288 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.718299 28185 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.719761 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2495) took 3047 ms. Trace:
I20260709 12:19:07.720116 28288 rpcz_store.cc:276] 0709 12:19:04.672333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.710381 (+3038048us) service_pool.cc:224] Handling call
0709 12:19:07.718893 (+  8512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.719494 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.719686 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:07.722568 28360 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2343) took 1340 ms (client timeout 1687 ms). Trace:
I20260709 12:19:07.708426   968 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.723403   968 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.717895 28287 rpcz_store.cc:276] 0709 12:19:04.672016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.709414 (+3037398us) service_pool.cc:224] Handling call
0709 12:19:07.715688 (+  6274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.716390 (+   702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.717165 (+   775us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b540018f450. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:07.717301 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13435136}
I20260709 12:19:07.726234 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2296) took 3170 ms. Trace:
I20260709 12:19:07.726707 28292 rpcz_store.cc:276] 0709 12:19:04.545564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.665445 (+3119881us) service_pool.cc:224] Handling call
0709 12:19:07.687399 (+ 21954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:07.715469 (+ 28070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.715708 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:07.722996 28360 rpcz_store.cc:269] 0709 12:19:06.382057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.382192 (+   135us) service_pool.cc:224] Handling call
0709 12:19:07.722347 (+1340155us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003d2f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:07.722492 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2985540864}
I20260709 12:19:07.729573 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2499) took 3056 ms. Trace:
I20260709 12:19:07.727475 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 812) took 3170 ms. Trace:
I20260709 12:19:07.730039 28288 rpcz_store.cc:276] 0709 12:19:04.673392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.720966 (+3047574us) service_pool.cc:224] Handling call
0709 12:19:07.721427 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.721708 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.724898 (+  3190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:07.729308 (+  4410us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:07.729350 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.729414 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.729515 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.731500 28286 rpcz_store.cc:276] 0709 12:19:04.547065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.670186 (+3123121us) service_pool.cc:224] Handling call
0709 12:19:07.710951 (+ 40765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.716809 (+  5858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.717001 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.737242   908 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.694169  1188 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.717433   565 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:07.717308   935 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:07.746769 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 91826.9us
W20260709 12:19:07.749782 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.750592 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.750662   925 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.738298 28185 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 6, term changed from 11 to 14, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to <none>, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1), VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2), VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3), VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 14 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:07.753115 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:07.754863 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 224209us
I20260709 12:19:07.757118 28292 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:07.738246 28291 raft_consensus.cc:1240] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.758249 28289 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
W20260709 12:19:07.759055 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 76816.2us
I20260709 12:19:07.760583 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.762382   698 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:07.765868 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.780331 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2514) took 1430 ms. Trace:
I20260709 12:19:07.780776 28289 rpcz_store.cc:276] 0709 12:19:06.350159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.496346 (+1146187us) service_pool.cc:224] Handling call
0709 12:19:07.780266 (+283920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.769263   904 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:19:07.753325 28185 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.769718   765 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.770403 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:07.771044   948 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.771780 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 818) took 2836 ms. Trace:
I20260709 12:19:07.785204 28359 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:07.785348 28288 rpcz_store.cc:276] 0709 12:19:04.935461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.738240 (+2802779us) service_pool.cc:224] Handling call
0709 12:19:07.771194 (+ 32954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.771383 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.771409 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:07.771480 (+    71us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:07.771529 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.771586 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.771705 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.775377  1188 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:07.776925 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.743379  1184 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:07.779604 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2503) took 3085 ms. Trace:
I20260709 12:19:07.729425   928 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.783595 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2299) took 3186 ms. Trace:
I20260709 12:19:07.792673 28291 rpcz_store.cc:276] 0709 12:19:04.597273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.702107 (+3104834us) service_pool.cc:224] Handling call
0709 12:19:07.702464 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.737371 (+ 34907us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b54005120a0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7fedc0 00007f85be8891c0 00007f85be88a231 00007f85be886d8f 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c
0709 12:19:07.783333 (+ 45962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.783533 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23499648}
W20260709 12:19:07.779263 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 145961us
I20260709 12:19:07.789026 28283 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.790156 28292 rpcz_store.cc:276] 0709 12:19:04.693909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.731232 (+3037323us) service_pool.cc:224] Handling call
0709 12:19:07.756900 (+ 25668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.778704 (+ 21804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.779554 (+   850us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.782351 28404 leader_election.cc:400] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
W20260709 12:19:07.798430 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 179125us
I20260709 12:19:07.799693 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:07.783908   915 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14.
I20260709 12:19:07.804328 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 815) took 3240 ms. Trace:
I20260709 12:19:07.804814 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2504) took 2974 ms. Trace:
I20260709 12:19:07.805212 28285 rpcz_store.cc:276] 0709 12:19:04.830430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.732714 (+2902284us) service_pool.cc:224] Handling call
0709 12:19:07.777729 (+ 45015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.789634 (+ 11905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.789660 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:07.792851 (+  3191us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:07.804582 (+ 11731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.804652 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.804758 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.789755  1191 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:07.804751 28283 rpcz_store.cc:276] 0709 12:19:04.564120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.679978 (+3115858us) service_pool.cc:224] Handling call
0709 12:19:07.784462 (+104484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.796651 (+ 12189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.804091 (+  7440us) spinlock_profiling.cc:243] Waited 91.9 ms on lock 0x7b54000028d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:07.804252 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211326336}
I20260709 12:19:07.807504 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2334) took 1693 ms. Trace:
I20260709 12:19:07.739662  1136 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.809407 28291 rpcz_store.cc:276] 0709 12:19:06.114424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.800994 (+1686570us) service_pool.cc:224] Handling call
0709 12:19:07.801390 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.807177 (+  5787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.807203 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:07.807245 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:07.807279 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.807331 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.807441 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.817900  1112 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:07.818356  1112 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.818712  1112 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.813851 28287 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:19:07.814358 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:07.823274  1188 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:07.823897 28283 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:07.824594 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2518) took 1455 ms. Trace:
I20260709 12:19:07.824951 28283 rpcz_store.cc:276] 0709 12:19:06.369420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.822901 (+1453481us) service_pool.cc:224] Handling call
0709 12:19:07.823682 (+   781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.824320 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.824525 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.824016 28332 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:19:07.826790 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 118890us
I20260709 12:19:07.828118 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2500) took 3154 ms. Trace:
I20260709 12:19:07.801566  1048 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:07.828490 28287 rpcz_store.cc:276] 0709 12:19:04.673668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.725562 (+3051894us) service_pool.cc:224] Handling call
0709 12:19:07.736353 (+ 10791us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:07.822160 (+ 85807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.827869 (+  5709us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b54003f9654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:07.828061 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177586944}
I20260709 12:19:07.830470 28292 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:07.830709 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 820) took 2622 ms. Trace:
W20260709 12:19:07.831599 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 50912.5us
I20260709 12:19:07.832827   968 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.830508  1188 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.834786 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2509) took 1690 ms. Trace:
I20260709 12:19:07.835078 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:07.833518   928 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.835161 28292 rpcz_store.cc:276] 0709 12:19:06.143847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.803263 (+1659416us) service_pool.cc:224] Handling call
0709 12:19:07.828669 (+ 25406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.834519 (+  5850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.834728 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.809139  1191 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:07.835711 28283 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:07.841681 28332 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.831147 28286 rpcz_store.cc:276] 0709 12:19:05.208811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.779456 (+2570645us) service_pool.cc:224] Handling call
0709 12:19:07.810609 (+ 31153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.827936 (+ 17327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.827966 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:07.830437 (+  2471us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:07.830479 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.830555 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.830652 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.838505 28289 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:07.833923 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:07.845103 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2505) took 1722 ms. Trace:
I20260709 12:19:07.845463 28289 rpcz_store.cc:276] 0709 12:19:06.121805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.814843 (+1693038us) service_pool.cc:224] Handling call
0709 12:19:07.831275 (+ 16432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.844028 (+ 12753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.844237 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.828872   948 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.846781 28185 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 11 to 14, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 14 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:07.848595   948 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.839496   915 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:07.849853   915 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:07.850282   915 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.850713   765 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.851262 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.841811  1154 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:07.852638  1154 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.852977  1154 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:07.843011  1052 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:07.858588 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.831013  1048 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:07.861867 28984 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:07.848508 28185 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 8 to 14, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 14 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:07.849620 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.865307 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.867164 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:07.870970 28365 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.867304 28332 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.872043  1188 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.873209 28332 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.873544   935 raft_consensus.cc:2764] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:07.874553 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:07.875196   968 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:07.875622   968 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.875957   968 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.876390  1068 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:07.878082 28364 raft_consensus.cc:2368] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:07.839993  1111 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:19:07.863914  1048 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.864900   750 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:07.882843   750 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.883198   750 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:07.877562  1068 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:07.885852  1068 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:07.869421  1052 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.886735  1068 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.886706  1052 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.888151  1068 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:07.840657  1184 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:19:07.889760  1052 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:07.872973 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 70791.3us
I20260709 12:19:07.891093 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:07.892501 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:07.893313   904 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:07.894941 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:07.895155 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:07.894764 28185 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.895732 28438 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.897153 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.897290   792 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:07.843634  1191 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.897801 28435 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:07.899850 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.900689 28441 raft_consensus.cc:2410] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:07.902693  1202 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:19:07.903595 28284 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:19:07.912830  1166 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:07.905290  1204 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:19:07.905678 28438 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:07.906937   750 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.908506 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:07.950532 28441 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:07.909920 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:07.913838 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.914151   904 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:07.904526 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2508) took 1766 ms. Trace:
I20260709 12:19:07.915247   729 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.897725   792 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:07.876119  1203 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:07.918052 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:07.952857  1203 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:07.953368  1203 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.953460 28432 raft_consensus.cc:2410] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:07.922545 28359 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:07.924433 28365 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:07.927229 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:07.957953 28435 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:07.959733 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.961328  1172 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:07.928597 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 94746.1us
I20260709 12:19:07.963079 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:07.933199 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.964471 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.952092 28288 rpcz_store.cc:276] 0709 12:19:06.143530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.805352 (+1661822us) service_pool.cc:224] Handling call
0709 12:19:07.836877 (+ 31525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.904182 (+ 67305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.904212 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:07.904258 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:07.904292 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.904362 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.904468 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:07.936187  1205 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:07.980305 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.927172 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.929621  1188 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:07.967787   968 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.864511  1112 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:07.970516 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 99675.7us
I20260709 12:19:07.975210   915 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:07.896667 28185 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 7, term changed from 7 to 14, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4), f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:19:07.976635 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 81793.2us
I20260709 12:19:07.978880 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:07.927136  1154 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.980332 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.007702 28255 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:08.002225 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:07.985792 28441 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:07.978801   952 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:19:08.009119 28435 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:07.990319 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2491) took 3710 ms. Trace:
I20260709 12:19:08.009690 28290 rpcz_store.cc:276] 0709 12:19:04.279898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.565824 (+3285926us) service_pool.cc:224] Handling call
0709 12:19:07.581025 (+ 15201us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:07.585397 (+  4372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:07.585438 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:07.611871 (+ 26433us) log.cc:844] Serialized 26 byte log entry
0709 12:19:07.698922 (+ 87051us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:07.698962 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.699031 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.990065 (+291034us) raft_consensus.cc:1685] finished
0709 12:19:07.990086 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.990225 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":15358,"thread_start_us":63176,"threads_started":1,"wal-append.queue_time_us":109316}
I20260709 12:19:07.990737   792 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.993392   729 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:07.993808   948 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:07.994372  1112 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.995074   750 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:07.995608 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:07.995848  1195 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:07.997664 28438 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:07.987720  1205 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:07.985929  1188 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:08.031296  1188 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.031579   948 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:08.000234  1111 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:19:08.033535 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 54750.8us
I20260709 12:19:08.021899 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.022492 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.023327 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:08.024480 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 75914.9us
I20260709 12:19:08.025039  1080 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:07.983937  1172 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.000146   968 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.029422 28367 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:19:08.027114 28185 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:07.990396  1191 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.025493 28366 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:08.038837 28364 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:08.039770 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:08.047224 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:08.041290 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:08.048013 28288 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.015715   925 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:08.040874 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:19:08.040637 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.047376 28290 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
W20260709 12:19:08.050110 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.029469  1127 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.048666 28361 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:08.055718 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:08.049203 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:08.063380 28360 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:08.032014   948 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:08.038079   698 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:08.083921 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.048637   925 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.046619   952 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:19:08.086519 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 52127.7us
I20260709 12:19:08.091822  1195 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:08.092798  1127 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:08.082504 28185 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:08.054780  1052 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.108029  1052 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.108872   915 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.095851   792 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.110241  1052 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:08.098992 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:08.102011 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.103148 28185 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 10 to 13, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 13 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:08.107023  1205 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:08.117093 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 75156.6us
I20260709 12:19:08.119563 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:08.120256 28284 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.123621 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:08.124464 28358 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:08.087296  1184 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:08.096383   925 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.130887 28290 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.122298  1052 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:08.149834 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 106473us
I20260709 12:19:08.154822  1181 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:08.150300  1052 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.096223   698 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.150802  1203 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.162827   948 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.153216  1184 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.123109  1127 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.152115  1215 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:08.166162  1195 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.130954  1080 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:08.167899 28332 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.172115 28288 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:08.188825 28363 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.189282   698 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.189653   808 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:08.191294   565 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:08.191792   565 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:08.192884   565 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.175343 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:08.130338  1111 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:19:08.179545 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:08.170192  1181 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.216547  1181 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.217329 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:08.216596   698 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:19:08.219223  1191 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:08.198186  1215 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:08.204509 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.219802   698 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:08.220212   698 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.220432  1215 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.221103  1184 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:08.195164  1068 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.222070  1068 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:08.223014  1068 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.225850   698 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.191805  1222 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.226480  1222 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.198797   948 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.216485  1172 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:08.229755 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 61793.2us
I20260709 12:19:08.231375   952 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:08.231782   952 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:08.232156   952 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.215332  1166 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:19:08.217466 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 129426us
I20260709 12:19:08.215631   928 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:08.180135   904 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:08.237782   904 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.238087   928 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:08.238497   928 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:08.239317   928 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.240469   928 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.176322  1048 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.243666 28285 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.234905  1166 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:08.244865  1166 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:08.245096  1080 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.245827  1166 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.208989   765 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:19:08.247344 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44379.4us
I20260709 12:19:08.247589   765 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:08.248509   765 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.223912 28366 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:08.250154 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2515) took 1899 ms. Trace:
I20260709 12:19:08.174265 28284 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:08.208269  1154 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.228845  1052 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:08.261240 28404 leader_election.cc:400] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:08.261601 28404 leader_election.cc:403] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:19:08.250551 28285 rpcz_store.cc:276] 0709 12:19:06.350438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.813520 (+1463082us) service_pool.cc:224] Handling call
0709 12:19:07.842996 (+ 29476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.244040 (+401044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.245634 (+  1594us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54003ab954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:08.250090 (+  4456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":920995072}
I20260709 12:19:08.211802  1203 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.255349 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:08.260887 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 218613us
W20260709 12:19:08.268890 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.266808 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.268021 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:08.259842 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.262229  1166 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.272018 28984 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:19:08.272644   935 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.273502 28990 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:19:08.273229   935 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.274590 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:08.275290 28436 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:08.275835  1184 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:08.276383  1184 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:08.268419   765 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.279337 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.279953 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:08.282063 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:08.290086 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:08.290866 28435 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:08.288193 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.295151 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:08.295764 28438 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.300658 28990 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.277841  1052 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:08.276068   935 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:08.314167 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.313974 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:08.313854 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:08.322190 28437 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.317626 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:08.322471 28439 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 20: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:08.317180 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:08.278280  1048 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.291167  1205 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:08.293484  1127 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:08.333032 28433 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:08.321599 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:08.289742   968 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:08.304358  1191 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:08.338568 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:08.342314 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:08.346405 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.346657 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.347080 28362 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:19:08.277356  1184 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.320642  1222 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.352378 28434 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:08.356467 28367 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:08.304225   729 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:19:08.335659  1127 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:08.365988 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 98098.8us
I20260709 12:19:08.316924   925 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.339670  1154 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.364614 28438 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:08.366979 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:08.365023 28366 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:08.343007  1205 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:08.369705 28984 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:08.393453 28288 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:08.394582 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:08.396239 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59120.8us
I20260709 12:19:08.404470 28438 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:08.368657  1127 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.405802 28360 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 18: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:08.352118  1223 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:08.406884 28956 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:08.418471 28363 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:08.418823  1201 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 12:19:08.419888  1201 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:08.442999 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.405220  1154 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:08.454797  1080 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:08.402287  1052 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:19:08.411453  1203 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.462536  1224 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:08.478854 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.454593  1221 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.420444  1201 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.452706  1223 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:08.412220  1188 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:08.489579 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 82246.3us
I20260709 12:19:08.474053   968 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:08.486502  1203 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.507895  1223 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.508958  1181 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.509470  1205 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.498704  1154 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:08.495079  1226 log.cc:927] Time spent T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:19:08.465325  1218 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:08.499927 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.488731  1224 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.478137  1080 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.494987  1235 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:19:08.529085 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.530285 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:08.531175  1221 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.514686  1218 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.536981 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:08.532172  1080 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.531605  1235 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:19:08.549633  1228 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:19:08.551991  1242 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:08.552417  1242 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:08.553373  1242 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.555634  1181 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.534134  1048 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:19:08.559068 28437 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:08.562155   968 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.563273  1224 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.545717  1238 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:08.576572 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.537997  1221 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:08.533167  1191 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:08.574260 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:08.574976   928 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:08.540375  1218 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.577514 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:08.568331 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:08.574995 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.580467  1048 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:08.605584  1048 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:08.582705 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:08.582973  1188 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:08.584993 28330 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.008s (SENT)
I20260709 12:19:08.612203 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:08.586833 28359 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:08.616568 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 289165us
W20260709 12:19:08.592252 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.598554 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:19:08.590121 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.597935 28285 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.622726 28437 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:08.622867 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.579047  1181 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
W20260709 12:19:08.628400 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.631726 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 100057us
I20260709 12:19:08.598352   928 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:08.636010 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.640127 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:08.637027   928 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.645431 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.606601  1048 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.649525  1048 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.651093  1235 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.652484  1235 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.653936  1221 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.654915   904 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.656114   928 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.662317 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:08.666580  1201 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:08.671613 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:08.663602 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:08.678673 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.679469 28437 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:08.680089  1233 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.579634  1238 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.689108 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.728159 28359 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:08.689842 28332 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:19:08.693425 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 108475us
I20260709 12:19:08.693912 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:08.673939   952 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.627333   698 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.700908  1203 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:08.704146   565 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:08.698107  1195 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:08.741074  1195 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:08.741480  1195 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.714293  1224 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:19:08.705957 28361 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:08.706512   904 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.743672   904 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:08.707363   928 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.744076  1166 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:08.746438   750 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:08.746975   750 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:08.717782  1242 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.718667 28990 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:19:08.718364  1191 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:08.720539 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.765997 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.722540 28367 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:08.722952 28285 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
W20260709 12:19:08.724577 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.733145 28364 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:08.734578   952 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:08.737006  1238 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:08.747891   750 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.727752  1201 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.740402 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:08.742601 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48775.7us
I20260709 12:19:08.780339 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:08.764808 28984 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:08.772460 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:08.773922 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.739598   698 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.781540 28360 raft_consensus.cc:1813] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:08.721776  1127 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.776486 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:08.777421 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:08.780601 28363 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.784071 28360 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:19:08.782426 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.750072 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.784137 28991 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:08.767251  1052 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:08.784955 28361 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:08.788352 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:08.789546 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.793671 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.794226  1238 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:08.794710  1238 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.795068  1238 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.800232  1240 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:08.800853  1240 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:08.790490   925 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.801800  1240 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:08.805627 28404 leader_election.cc:336] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.505s (SENT)
I20260709 12:19:08.787674  1201 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.812307  1201 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:08.818673 28363 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:08.786175  1154 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.823418 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.824369 28363 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:08.820240  1154 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:08.829296  1154 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.791069  1052 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.830152  1127 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.831946  1052 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:08.834082 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45835us
I20260709 12:19:08.834951 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:08.822381 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.834573  1166 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:19:08.838829 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 106383us
W20260709 12:19:08.838743 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 62389.4us
I20260709 12:19:08.841385 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:08.847332 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:08.850215 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:08.850924 28285 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:19:08.852836 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:08.854245 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:08.855000 28288 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:19:08.844398  1127 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:08.858213  1127 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:08.858592  1127 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:08.859833 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.860347 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:08.835382  1195 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.861306   698 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:08.861656  1231 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:08.862349  1231 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:08.844475   935 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:08.863090   935 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.863339  1231 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.864379   935 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.861913   698 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:08.865720  1218 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.869385 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:08.838462  1166 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:08.871058  1166 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.871697   908 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:08.872277   908 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:08.872937  1227 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:08.873457  1227 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:08.873926  1227 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.876157 28362 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:19:08.878623   698 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.870044 28989 raft_consensus.cc:1813] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:08.882117 28989 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 12:19:08.858639  1251 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:08.886545  1181 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:19:08.890311 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44551.5us
I20260709 12:19:08.893713 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:08.894495 28440 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:08.898197 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.905261 28367 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.908408 28285 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:08.873165   908 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:08.921985 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:08.841903  1052 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:08.924856 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:08.927639  1184 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:08.927443  1112 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:19:08.965111  1234 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:08.965646  1222 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.928228   792 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.966213  1222 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.966920   792 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:08.927484  1224 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:08.968562  1222 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.869935  1218 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:08.985250 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:08.985766 28989 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:08.918020  1166 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.958128 28364 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:08.959458 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.957371 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:08.970502 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47509.1us
I20260709 12:19:08.991629 28284 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:08.970083 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:08.974941 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:08.993484 28283 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:08.929073  1172 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:08.986315 28289 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.005187 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:08.924228 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:08.980027 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:08.995249 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:09.008608 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:09.009486 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 85320.9us
I20260709 12:19:08.957612  1052 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:09.014300 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.924528  1201 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:19:09.025461 28435 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:09.004633  1224 consensus_queue.cc:1243] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:09.019438   952 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:09.048283 28404 leader_election.cc:336] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.777s (SENT)
I20260709 12:19:09.021370  1052 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.036017 28440 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:09.042085 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.052009 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:09.042795  1258 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.200s	user 0.002s	sys 0.000s
I20260709 12:19:09.021075  1195 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.020438   792 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:09.045382  1201 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:09.030864 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:09.056993 28364 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:09.026023  1223 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:09.069711 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 56399.7us
I20260709 12:19:09.074853 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:09.082135 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:09.048026   952 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.093650 28435 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:09.075807  1223 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.098637 28283 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:09.110124  1223 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.072366  1201 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:09.112341 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:09.090198   952 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.131440  1201 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.132824  1201 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.134439 28284 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:09.135614 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:09.136854 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 107589us
I20260709 12:19:09.094429  1195 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:09.113832 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:09.141959 28364 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:09.089098  1181 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:09.109217 28285 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:09.090440   948 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:09.145358   948 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:19:09.139918 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.146469   948 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:09.147719  1260 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:19:09.141208 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:09.151999 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.152318 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.133399  1127 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.153602 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.153936 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.154070 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.154634 28290 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:09.155293 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.155846 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.156205 28287 raft_consensus.cc:2410] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:19:09.156884  1181 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.157310  1181 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.160460 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.161285 28292 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:19:09.143996 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.162757 28288 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:09.118093 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 105329us
W20260709 12:19:09.182945 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.184378 28286 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.185778 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.188493 28441 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
W20260709 12:19:09.190564 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.191120  1227 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:09.195895  1235 consensus_meta.cc:287] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.006s
I20260709 12:19:09.196805 28437 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:19:09.197150 28437 pending_rounds.cc:85] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 11
I20260709 12:19:09.197345 28437 pending_rounds.cc:107] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:19:09.197616 28437 raft_consensus.cc:2889] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:09.117010   935 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.204727   935 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:09.206952 28404 leader_election.cc:336] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.951s (SENT)
I20260709 12:19:09.207543 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:09.209173 28367 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:09.212769  1191 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:09.214771  1191 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:09.215965 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.216614 28436 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:09.222911 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.110438  1212 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 12:19:09.177150  1080 consensus_meta.cc:287] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.600s	user 0.013s	sys 0.000s
I20260709 12:19:09.139652   928 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:09.244535 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:09.245306 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.247145 28440 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:19:09.247531 28440 pending_rounds.cc:85] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 12
I20260709 12:19:09.247756 28440 pending_rounds.cc:107] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:19:09.247989 28440 raft_consensus.cc:2889] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:09.248225 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:09.248860 28991 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:09.230903 28436 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:09.227207 28292 raft_consensus.cc:2393] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.237896 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.238184 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.251422 28289 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.248989 28286 raft_consensus.cc:2368] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18.
I20260709 12:19:09.193457  1227 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.254088  1264 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:19:09.258409 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.258282  1238 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:09.259285 28284 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15.
I20260709 12:19:09.260728  1238 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:09.265309 28991 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:09.267910 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:09.268733 28363 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.242172 28364 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:09.223740  1195 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:09.268147 28287 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:09.271055 28290 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:09.273880 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.281226 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.284058 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.284503 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.286370 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.290812  1267 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.097s	user 0.003s	sys 0.000s
I20260709 12:19:09.297843 28436 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:09.250643   928 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:09.303594 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.304394 28288 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:09.315951 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.316795 28290 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:09.331930 28288 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:09.248580  1080 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.333323 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.335389 28286 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.339097  1195 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.340185 28284 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:09.342010   904 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:09.342417   904 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:09.343240   904 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:09.344466   904 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.345204  1080 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.346093  1112 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:09.347581  1112 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.348073  1112 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.352218   928 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:09.353842 28435 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:19:09.354293 28435 pending_rounds.cc:85] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 8
I20260709 12:19:09.354521 28435 pending_rounds.cc:107] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:19:09.354768 28435 raft_consensus.cc:2889] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:09.355827  1223 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.357163 28989 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:19:09.318697  1181 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.362608  1127 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:09.364055 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.364545 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.364697 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.150s (SENT)
I20260709 12:19:09.365101 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:09.366529  1201 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:09.368201 28363 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:09.364008  1112 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.371949 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.372830 28291 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.373706 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.374236 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.374375 28291 raft_consensus.cc:2376] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15. Re-sending same reply.
I20260709 12:19:09.375125 28289 raft_consensus.cc:2410] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:19:09.375272 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.375901 28291 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:09.375983 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.377130 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.377699 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.380430 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.381162 28292 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.382036 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.383015 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.383562 28292 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.384299 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.384907 28283 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:09.378558 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.386129 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.386785 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.387061 28284 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 18 because replica has last-logged OpId of term: 16 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 12:19:09.365103 28982 raft_consensus.cc:2368] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:09.386721  1222 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:09.387895  1222 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:09.387984 28285 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.388296 28284 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:09.388872  1222 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.389853 28284 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:09.390192   698 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:09.390751   698 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:09.392885   698 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.389971  1222 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.393359  1242 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:09.395879 28982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 762) took 1127 ms. Trace:
I20260709 12:19:09.396274 28982 rpcz_store.cc:276] 0709 12:19:08.268312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.268431 (+   119us) service_pool.cc:224] Handling call
0709 12:19:08.568183 (+299752us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762e9c 00007f85bb762a5b 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:19:09.395642 (+827459us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54007413d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:09.395789 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1767457664}
I20260709 12:19:09.398038 28363 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:09.404592 28286 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:09.377166 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.405584 28289 raft_consensus.cc:2376] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16. Re-sending same reply.
I20260709 12:19:09.405599 28285 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:09.409024 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.409902 28287 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:09.404515 28283 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:09.359820  1181 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.415745 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.419238 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.425683 28440 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.426882 28435 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:09.428092 28290 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:09.425377 28288 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:19:09.425315 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 43672.9us
I20260709 12:19:09.431860 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:09.430102 28330 leader_election.cc:400] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:09.433930 28330 leader_election.cc:403] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:09.435331 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.436900 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.437040 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.438153 28437 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:09.438524 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.437770 28287 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:19:09.438699 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.439446 28283 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:09.440618 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:09.410844  1218 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.445398 28365 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:19:09.449206 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.241s (SENT)
I20260709 12:19:09.449384 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:09.449895 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:09.449539 28332 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:09.459399 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:09.460116 28358 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.465082  1201 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:09.465919  1205 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:09.470503 28358 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:09.411060  1242 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:19:09.471768 28358 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:09.472266 28362 raft_consensus.cc:2368] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16.
W20260709 12:19:09.474587 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.479147 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:09.480412 28364 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:09.420143  1112 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:19:09.421347   792 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:19:09.499755 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.487248 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.480777  1223 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:09.478652  1215 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:09.487362  1205 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:09.493409  1216 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:09.483430  1181 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:19:09.496308 28440 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
W20260709 12:19:09.517948 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48265.7us
I20260709 12:19:09.518390 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:09.502557 28437 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:09.505904 28287 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.511132 28292 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:09.440536  1068 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:19:09.526696 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.505908  1195 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.452227  1218 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.551985  1068 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:09.552491  1068 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:09.553201  1218 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.553357  1068 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.537364   928 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:19:09.554850  1068 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.556382 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.557168 28292 raft_consensus.cc:2376] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15. Re-sending same reply.
I20260709 12:19:09.537266  1264 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:19:09.559029  1205 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:09.560272  1205 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.565622  1276 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:09.566275  1276 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:09.567402  1276 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.569052   948 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:09.516897  1221 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.583489 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.583635  1195 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:09.590969  1272 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:19:09.591639  1195 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:09.513969   565 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:19:09.602928   952 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.606245   952 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:09.607040  1221 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.607393 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:09.608194 28982 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.615849 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.616680 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.618748 28982 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:09.620549 28287 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:19:09.614012  1052 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 12:19:09.622751 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.569594   948 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:09.619199 28292 raft_consensus.cc:2376] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16. Re-sending same reply.
I20260709 12:19:09.621052 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:09.636283 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.637130 28292 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
W20260709 12:19:09.641920 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 210080us
I20260709 12:19:09.645229 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.609690  1221 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:09.646427 28292 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.625491  1195 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.647337 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.654703  1195 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.656522 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:09.657074  1195 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:09.658017 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:09.658838 28989 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 18: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:09.658640 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:09.659410 28984 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:09.663241  1234 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.663820  1234 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:09.633886   968 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:09.666183  1234 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.667053 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.667500 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.668239 28285 raft_consensus.cc:2368] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:09.668466 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.668960 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.666932 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.669845 28291 raft_consensus.cc:2376] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14. Re-sending same reply.
I20260709 12:19:09.669766 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.670500 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.669251 28283 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.670809 28286 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.671283 28290 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:09.671594 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.633930  1052 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:09.659178 28954 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:09.670799 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.670085 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.681870 28289 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:09.673826 28292 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:09.674431 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.682442 28291 raft_consensus.cc:2376] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20. Re-sending same reply.
I20260709 12:19:09.683284 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:09.683494 28287 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:09.686231 28283 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:09.686906 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.689232 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:09.704032 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:09.704919 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.705149 28984 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:19:09.705539 28291 raft_consensus.cc:2376] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14. Re-sending same reply.
I20260709 12:19:09.706902 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:09.708127  1221 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:09.709954  1221 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.710354  1221 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:09.711905  1052 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:09.713511 28404 leader_election.cc:336] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.960s (SENT)
I20260709 12:19:09.703974  1172 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:09.704349  1112 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:09.718124  1112 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.718523  1112 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:09.717943 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.719233 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:09.649791   792 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:19:09.692728   968 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:19:09.727711 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 209428us
I20260709 12:19:09.650007   565 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:19:09.733604 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:09.734258 28359 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.709120  1195 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:09.732437 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.736972 28283 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 18 because replica has last-logged OpId of term: 16 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
W20260709 12:19:09.738904 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.729099  1223 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:09.725121  1172 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:09.747658 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.737906  1195 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.750908 28290 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:09.748507  1223 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:09.752624  1195 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.757907 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:09.748318 28283 raft_consensus.cc:2376] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16. Re-sending same reply.
I20260709 12:19:09.746670   968 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.766427 28287 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
W20260709 12:19:09.771665 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.776573 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.730847 28984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "189fdd6055df4826a42af9c225596e34"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:19:09.799928 28359 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:09.728463   948 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.814728   928 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.823138 28255 leader_election.cc:400] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16.
I20260709 12:19:09.823612 28255 leader_election.cc:403] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 12:19:09.823143  1278 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.135s	user 0.000s	sys 0.004s
I20260709 12:19:09.824939  1271 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16.
I20260709 12:19:09.823781  1172 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:09.836735 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.836735 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.846673 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.847301 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.849272 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:09.850237 28437 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:09.851068 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.851145 28435 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:09.851665 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.859009 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:09.859815 28436 raft_consensus.cc:2368] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17.
I20260709 12:19:09.860688 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.861322 28438 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.865482  1181 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.866153  1181 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.815374   928 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:09.867585  1286 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:19:09.868449  1181 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:09.873342 28435 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:09.889577 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:09.891626 28435 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:09.894276 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.896205 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:09.896731 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:09.896961 28288 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:09.889333 28438 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:09.814543  1223 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.904582 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:09.908190 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:09.911957 28436 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:09.915303   928 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:09.923661  1112 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.924978 28984 raft_consensus.cc:606] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:09.925402 28984 raft_consensus.cc:614] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:09.926546  1221 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:09.928790  1221 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:09.929778 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.934540 28363 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:09.937143 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.939541 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:09.941254  1233 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:09.942639   908 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:09.943761   908 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:09.943895  1233 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:09.944473  1233 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:09.947047 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:09.947762 28437 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:09.949433 28437 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:09.952409  1222 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:09.958711 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:09.959543 28991 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:09.969000 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:09.978042 28436 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:09.983597 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.984823 28435 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:09.944049  1276 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.987030 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:09.946779   908 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:09.956447  1154 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:09.998821 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:10.005250 28991 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:19:10.008193 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:10.011798 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:10.019727 28436 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:10.001056 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:10.023773 28432 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 18: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:10.025902 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:10.026750 28359 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:10.028450 28436 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:10.028360 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:10.029498 28434 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:10.033121 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.945551  1233 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:10.037014 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.032356 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:09.997464  1154 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:10.050695 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:10.047479 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.040630 28439 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.999184  1212 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:19:10.028148   928 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:10.073217 28359 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:10.075134 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:10.052976  1154 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:10.078045 28433 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:19:10.078620 28433 pending_rounds.cc:85] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 9
I20260709 12:19:10.078868 28433 pending_rounds.cc:107] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:19:10.079107 28433 raft_consensus.cc:2889] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:10.095430 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:10.096217 28359 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:10.038980  1205 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:19:10.112059 28433 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:10.088097  1112 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:10.116618 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.105964  1233 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:10.057626   908 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:10.120383 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:10.088599   928 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:10.124214 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:10.104013   904 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:19:10.145869 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:10.159780 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.160712 28435 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:10.164462 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:10.168366 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:10.169183 28435 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:19:10.147277 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:10.175441 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:10.176422 28359 raft_consensus.cc:2410] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:19:10.182184 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:10.182946 28435 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 18: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:10.185846 28363 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:10.186892 28435 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
W20260709 12:19:10.177472  1288 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.371s	user 0.004s	sys 0.000s
I20260709 12:19:10.190182 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:10.194856 28361 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:19:10.197185 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:10.197600  1294 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:19:10.209429 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:10.210297 28435 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:19:10.212817 28366 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:10.188800  1295 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 12:19:10.188499  1302 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:19:10.186010  1221 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:10.219282 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:10.220079 28436 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:10.215111 28439 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:10.222434 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:10.229063 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.235476 28286 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:19:10.229880 28404 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.716s (SENT) [suppressed 68 similar messages]
I20260709 12:19:10.249539 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:10.274884 28436 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:10.281893 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:10.277406  1052 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:10.297493 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 118231us
I20260709 12:19:10.221325   928 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:10.306180 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.295125  1080 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:10.315817 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.315960 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.288278  1305 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:19:10.302970   904 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:10.351722 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 160988us
W20260709 12:19:10.348899  1195 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.004s
I20260709 12:19:10.317217  1080 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:10.350343  1285 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.696s	user 0.005s	sys 0.000s
I20260709 12:19:10.362583 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:10.372067 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.373191 28292 raft_consensus.cc:2410] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:19:10.376207 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:10.379281 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:10.381968   935 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:10.381865 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:10.382776 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.389833 28284 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:19:10.391710 28291 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:10.393317 28283 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 20: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:19:10.393824 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.385632  1241 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:10.399097  1241 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:10.365280  1195 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:10.401548 28289 raft_consensus.cc:2368] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:10.364207  1080 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:10.403301  1306 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:10.403744  1306 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.404063  1306 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.406908 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:10.409379 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:10.417621 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64191.6us
I20260709 12:19:10.407856 28286 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:10.392477   935 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.423504 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:10.424187 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.360658   904 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:10.363359  1234 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:19:10.425992 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.398397  1052 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:10.429304 28404 leader_election.cc:400] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15.
I20260709 12:19:10.427974 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:10.433598 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:10.433872 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:10.434340 28284 raft_consensus.cc:2368] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:10.409384  1312 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.438133 28285 raft_consensus.cc:2410] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:19:10.438329  1312 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:10.439460 28291 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:10.440948  1312 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:10.442416  1308 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:19:10.444990 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 908) took 1044 ms. Trace:
I20260709 12:19:10.445433 28441 rpcz_store.cc:276] 0709 12:19:09.400941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.401437 (+   496us) service_pool.cc:224] Handling call
0709 12:19:09.409061 (+  7624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.409428 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:09.409460 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:09.409940 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.534484 (+124544us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:09.535025 (+   541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.535107 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.224961 (+689854us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:10.444652 (+219691us) raft_consensus.cc:1685] finished
0709 12:19:10.444678 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.444914 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4123,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":4621,"spinlock_wait_cycles":61833472,"thread_start_us":92606,"threads_started":1,"wal-append.queue_time_us":216501}
I20260709 12:19:10.448680 28366 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:10.432047   935 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:10.455850 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.459885 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.460415 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.460680 28283 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:10.474891 28290 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.482718 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:10.483523 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 104357us
I20260709 12:19:10.486094 28285 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:10.507472 28982 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:10.489944 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:10.510455 28284 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 20: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:10.453896  1195 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:10.493999 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64879.6us
I20260709 12:19:10.506999 28290 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:19:10.515970 28283 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:10.519877 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:10.480489  1201 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:19:10.525147 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:10.539827 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.540637 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:10.541384 28367 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:10.541399 28991 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:10.548657 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:10.522603   948 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:10.565428 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.568310 28285 raft_consensus.cc:2368] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:10.551575 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:10.564731 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:10.548272  1319 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:10.527076  1307 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.573889 28991 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:10.576071 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:10.573211 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:10.577152 28360 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 16
W20260709 12:19:10.594851 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 161021us
I20260709 12:19:10.609704 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:10.623828 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 148818us
I20260709 12:19:10.609328 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:10.634274 28285 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:10.572911  1315 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:10.636205 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.584497   948 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:19:10.643801 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47004.1us
W20260709 12:19:10.630479  1297 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:19:10.644572 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.598493   904 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:10.645105 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:10.515026  1154 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:10.649605 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 73631.5us
I20260709 12:19:10.633653  1319 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:10.684468 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:10.688889 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:10.576231  1318 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
W20260709 12:19:10.715109 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 87478.1us
I20260709 12:19:10.725343 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.565008  1052 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:10.721031 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:10.647761  1205 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:10.684492  1311 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 12:19:10.733532 28291 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:10.736649 28287 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:10.759167 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:19:10.762794 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 113041us
I20260709 12:19:10.625727  1307 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:10.765484 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.656257  1203 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:19:10.734058  1201 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:10.736085  1299 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:19:10.722110   904 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:10.789170 28434 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:10.794231 28285 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:10.799161 28288 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:10.764550  1319 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:10.764171  1112 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:10.847286 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 83043.5us
I20260709 12:19:10.848771 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.769398  1321 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.749217  1306 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:10.857276 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:10.798173 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:10.856079 28288 raft_consensus.cc:2410] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:19:10.856233 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:10.761437  1205 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:10.930358 28291 raft_consensus.cc:2376] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15. Re-sending same reply.
I20260709 12:19:10.934434 28292 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
W20260709 12:19:10.905556 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.943809 28360 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:10.853389  1112 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:10.999186  1316 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
W20260709 12:19:10.915808 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 118858us
I20260709 12:19:11.005079 28991 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:19:11.008428 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 211398us
I20260709 12:19:11.065317 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.009884 28285 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:10.854475  1330 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:11.012710 28284 raft_consensus.cc:2368] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15.
I20260709 12:19:10.842718  1336 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.207s
I20260709 12:19:11.066439 28291 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:11.067329 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.068873  1319 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:11.069142 28433 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:10.801524   948 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.028509  1315 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:11.098724 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.101027 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.116314 28285 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:10.932005  1321 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:11.119491 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.121004 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.123701 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.123726 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.129057 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:11.159911 28436 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:11.124536 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 268364us
I20260709 12:19:11.161466 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:11.118891  1080 consensus_meta.cc:287] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.002s
I20260709 12:19:11.164769 28288 raft_consensus.cc:2393] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:11.159863 28291 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.125967 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:11.165329 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 306154us
W20260709 12:19:11.130558  1338 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 12:19:11.096685  1315 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:11.150520  1335 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:19:11.150774  1322 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.501s	user 0.000s	sys 0.001s
I20260709 12:19:11.179368 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:11.184337 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.141610  1344 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.152232  1154 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:11.166059  1316 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.132138  1327 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:11.234022 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45553.3us
I20260709 12:19:11.238576 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.333258 28288 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:11.334426 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.335848 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.336728 28288 raft_consensus.cc:2368] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:11.197499 28436 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:19:11.176934  1112 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:11.266623 28291 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:11.337661 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.339224 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:11.339550 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.340445 28291 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 20: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:11.343889 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.346297 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.186012  1315 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:11.248513  1337 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:19:11.343425  1112 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:19:11.348307 28330 leader_election.cc:336] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.752s (SENT)
I20260709 12:19:11.353421 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.130810  1317 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.217s
I20260709 12:19:11.363361 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.364027 28289 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:11.356830 28332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.000s	sys 0.012s
I20260709 12:19:11.096493  1319 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.178236  1080 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:11.354658 28361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.003s	sys 0.008s
W20260709 12:19:11.369045 28361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.004s	sys 0.008s
W20260709 12:19:11.357391  1285 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:19:11.362267 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:11.367957 28330 leader_election.cc:336] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.115s (SENT)
I20260709 12:19:11.364605 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.372586 28291 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
W20260709 12:19:11.365725 28283 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.006s	sys 0.006s
W20260709 12:19:11.398476 28283 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.007s	sys 0.006s
I20260709 12:19:11.400573 28283 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:11.405958 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
W20260709 12:19:11.365648 28332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.000s	sys 0.012s
I20260709 12:19:11.406729 28291 raft_consensus.cc:606] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:11.409798 28291 raft_consensus.cc:614] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:11.414320 28359 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:11.359891  1307 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:11.453565 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.366813  1154 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:11.369485  1216 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.344527  1327 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.367328  1112 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:11.470919 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.486058 28291 raft_consensus.cc:2376] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18. Re-sending same reply.
W20260709 12:19:11.487164 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.450405  1330 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.495464 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:11.468977  1205 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:11.500310 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 132372us
I20260709 12:19:11.465039 28289 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:11.470032  1154 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.454768  1306 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:11.466379 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.513484 28288 raft_consensus.cc:2376] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18. Re-sending same reply.
W20260709 12:19:11.514047 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.340891  1316 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.516165  1316 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:11.522971 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2544) took 1003 ms. Trace:
W20260709 12:19:11.451995   904 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.008s	sys 0.004s
I20260709 12:19:11.523523 28284 rpcz_store.cc:276] 0709 12:19:10.519196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.519527 (+   331us) service_pool.cc:224] Handling call
0709 12:19:11.522701 (+1003174us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b5400163354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:11.522888 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1129360640}
W20260709 12:19:11.422045  1223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.000s	sys 0.012s
I20260709 12:19:11.510178 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.332387  1344 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:11.525231 28291 raft_consensus.cc:2376] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15. Re-sending same reply.
W20260709 12:19:11.523772   904 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.008s	sys 0.004s
I20260709 12:19:11.527344 28285 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:11.528690   904 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:11.527408  1241 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:11.530544  1241 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.536422   904 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:11.428159 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.551087 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.539196 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.553112 28289 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
W20260709 12:19:11.547324 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 385955us
I20260709 12:19:11.559603 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:11.546633 28437 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.007s	sys 0.008s
W20260709 12:19:11.562983 28437 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.008s	sys 0.008s
W20260709 12:19:11.549238 28438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.010s	sys 0.005s
W20260709 12:19:11.563881 28438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.010s	sys 0.005s
I20260709 12:19:11.548111  1333 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.583608 28289 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:11.572163  1333 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.600999  1340 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:11.601521  1319 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:11.601783  1340 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.602397  1333 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:11.603118 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.605093  1265 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:11.605271  1354 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.605705  1265 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:11.605811  1354 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.608069  1265 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:11.608527  1347 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:19:11.609265  1341 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.453s	user 0.001s	sys 0.000s
I20260709 12:19:11.487804  1216 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.614451  1216 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:11.614907 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2559) took 1074 ms. Trace:
I20260709 12:19:11.615417 28361 rpcz_store.cc:276] 0709 12:19:10.540407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.542449 (+  2042us) service_pool.cc:224] Handling call
0709 12:19:10.542809 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:10.574531 (+ 31722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.574579 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.575143 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 12:19:11.369653 (+794510us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:11.369699 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.369797 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.614622 (+244825us) raft_consensus.cc:1685] finished
0709 12:19:11.614649 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.614807 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31211,"raft.queue_time_us":34848,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":794075,"threads_started":1,"wal-append.queue_time_us":1037406}
I20260709 12:19:11.620503 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:11.622910  1080 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:11.630905 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 312966us
W20260709 12:19:11.624643 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.588049  1233 consensus_meta.cc:287] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.461s	user 0.013s	sys 0.006s
I20260709 12:19:11.623234  1321 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:11.632259 28330 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.753s (SENT)
I20260709 12:19:11.647974 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:11.666625 28435 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:19:11.647637 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:11.667645 28440 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:11.666996 28289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:11.655704 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:11.668715 28289 raft_consensus.cc:606] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:11.504141  1330 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:11.659435  1327 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:11.643882   935 consensus_meta.cc:287] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 1.193s	user 0.012s	sys 0.001s
I20260709 12:19:11.658470 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:11.677434 28436 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
W20260709 12:19:11.671133 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.572024  1112 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.672086  1361 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:11.687258 28436 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:19:11.687767 28436 pending_rounds.cc:85] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 7
I20260709 12:19:11.688023 28436 pending_rounds.cc:107] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:19:11.688252 28436 raft_consensus.cc:2889] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:11.687309   935 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:11.692013 28436 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:11.681249 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49038.2us
I20260709 12:19:11.678819 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:11.693011 28330 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.816s (SENT)
I20260709 12:19:11.693585 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:11.644485  1340 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:11.694835 28330 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.851s (SENT)
I20260709 12:19:11.700660 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:11.701216 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2899) took 1010 ms. Trace:
I20260709 12:19:11.701581 28441 rpcz_store.cc:276] 0709 12:19:10.691125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.691241 (+   116us) service_pool.cc:224] Handling call
0709 12:19:10.691494 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.691700 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:11.063361 (+371661us) log.cc:844] Serialized 14 byte log entry
0709 12:19:11.700832 (+637471us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:11.700889 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:11.700940 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.701013 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.701148 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":451801,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"thread_start_us":1008406,"threads_started":3}
W20260709 12:19:11.719166 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.738471 28330 leader_election.cc:336] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.909s (SENT)
W20260709 12:19:11.739441 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.688359  1323 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.742054 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:11.745477 28435 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 11.
I20260709 12:19:11.747735 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:11.711877  1321 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:11.755645 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 196086us
I20260709 12:19:11.733816  1212 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.715893  1344 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:11.739248   968 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:19:11.762800 28439 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.007s	sys 0.011s
W20260709 12:19:11.764911 28439 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.007s	sys 0.011s
W20260709 12:19:11.753397  1221 consensus_meta.cc:287] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.529s	user 0.016s	sys 0.000s
I20260709 12:19:11.751652  1336 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.751464  1307 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.754360  1333 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:11.762403  1296 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.573s	user 0.015s	sys 0.010s
I20260709 12:19:11.774842 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:11.783216 28363 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.775293 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:11.776556 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:11.781316 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2900) took 1080 ms. Trace:
I20260709 12:19:11.787618 28437 rpcz_store.cc:276] 0709 12:19:10.691470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.691587 (+   117us) service_pool.cc:224] Handling call
0709 12:19:10.721456 (+ 29869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.721702 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:11.563480 (+841778us) log.cc:844] Serialized 14 byte log entry
0709 12:19:11.764178 (+200698us) log.cc:844] Serialized 14 byte log entry
0709 12:19:11.771394 (+  7216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:11.771462 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:11.771517 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.771592 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.771717 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18837,"prepare.queue_time_us":423621,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":71,"thread_start_us":1000023,"threads_started":2}
I20260709 12:19:11.789657 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:11.791389 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:11.846565 28439 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:11.750447  1323 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:11.850114 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 67545.6us
I20260709 12:19:11.775050  1221 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:11.772614  1242 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:11.798198 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.900050 28291 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.792202 28288 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:11.767308  1319 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:11.790918  1361 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:19:11.872481 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 103483us
I20260709 12:19:11.807792 28359 raft_consensus.cc:2410] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:19:11.774056  1052 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.772871  1312 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:11.765444  1321 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.883606 28257 leader_election.cc:400] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17.
I20260709 12:19:11.915946 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:11.928041 28257 leader_election.cc:403] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:11.793568  1307 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:11.787783  1233 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:11.847667  1342 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:11.902150 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:11.903606 28363 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:11.847323  1154 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:11.906272 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:11.906952 28287 raft_consensus.cc:2410] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 18 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:19:11.907856 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 169422us
I20260709 12:19:11.863500   935 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:11.915571 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:11.961623 28984 raft_consensus.cc:2376] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15. Re-sending same reply.
I20260709 12:19:11.923362 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:11.923800 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:19:11.961404 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 198677us
W20260709 12:19:11.955853 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 82222.5us
I20260709 12:19:11.962885 28439 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.922837 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:11.956563 28330 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.984s (SENT)
I20260709 12:19:11.957276 28365 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:11.958148 28288 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:11.899098  1242 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:11.966037  1242 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:11.956646 28982 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46960 (request call id 590) took 2273 ms (client timeout 1919 ms). Trace:
W20260709 12:19:11.970155 28982 rpcz_store.cc:269] 0709 12:19:09.655893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.656216 (+   323us) service_pool.cc:224] Handling call
0709 12:19:11.929037 (+2272821us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b54004ec354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:11.929189 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1898275200}
I20260709 12:19:11.769238  1212 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.902985  1370 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:11.987337 28437 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:11.982154 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:11.988929 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:11.982937 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:11.983294 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:11.847894  1333 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.918118  1205 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:11.972221  1327 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:12.012494 28291 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:11.795569  1336 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:11.979041  1350 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.017386 28439 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:11.988845  1291 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.933200  1354 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:11.960355   792 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:19:11.953913  1276 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:19:12.036137  1379 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.036852  1379 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.038231   908 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:12.038828  1349 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 12:19:12.039850  1372 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
W20260709 12:19:12.040735  1351 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.271s	user 0.000s	sys 0.000s
I20260709 12:19:12.041684   928 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:12.042529   928 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:12.044214 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.045567 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.047909 28367 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:19:11.910288  1319 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.048795  1379 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.049273 28364 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:12.049574  1323 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.050132  1319 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.050907 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.051919  1195 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:19:12.051833 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 95304.8us
I20260709 12:19:12.052356  1195 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:11.994318  1221 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17.
W20260709 12:19:12.052850 28330 leader_election.cc:336] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.029s (SENT)
I20260709 12:19:12.052234 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.053472 28330 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:12.038087  1381 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:19:12.054646  1203 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.031834  1350 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.055562 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.056309 28363 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:12.052606 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:12.057442  1350 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.048341 28367 pending_rounds.cc:85] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 11
I20260709 12:19:12.048409  1212 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.057803 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.058036 28367 pending_rounds.cc:107] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:19:12.058714 28367 raft_consensus.cc:2889] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:12.026242   968 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:12.062469 28289 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:12.063143 28289 raft_consensus.cc:606] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:12.063491 28289 raft_consensus.cc:614] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:12.060731 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:12.063614 28330 leader_election.cc:336] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.129s (SENT)
I20260709 12:19:12.064179 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.064983 28285 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:12.008607  1327 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.066569 28285 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:19:12.078223 28285 pending_rounds.cc:85] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 8
I20260709 12:19:12.078505 28285 pending_rounds.cc:107] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:19:12.013163 28288 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:19:12.031980  1370 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.083007 28361 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.063726  1333 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.086748 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2530) took 1721 ms. Trace:
I20260709 12:19:12.087149 28290 rpcz_store.cc:276] 0709 12:19:10.365679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.365864 (+   185us) service_pool.cc:224] Handling call
0709 12:19:12.086556 (+1720692us) spinlock_profiling.cc:243] Waited 82 ms on lock 0x7b540009ecd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:12.086681 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188664576}
W20260709 12:19:12.075795 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 86966.2us
I20260709 12:19:12.007117  1309 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:11.933605  1052 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.028103 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:12.078758 28285 raft_consensus.cc:2889] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:12.091223 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.091557 28285 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:12.090945 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:12.092693 28285 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:12.092805 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:12.093539 28291 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:12.053280  1195 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.068099 28289 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:11.974097  1342 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.104849 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:12.157183 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52426.8us
I20260709 12:19:12.106389 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:12.159186  1276 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:12.159379 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 53086.3us
I20260709 12:19:12.160693  1276 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:12.155623 28283 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:12.168702 28289 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:12.083770  1327 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:12.173974 28291 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:12.082406  1321 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.083056  1330 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.184579 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:12.183065 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 119478us
I20260709 12:19:12.156018  1350 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:12.193270 28330 leader_election.cc:336] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.165s (SENT)
I20260709 12:19:12.155165  1353 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:12.198051 28330 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.213s (SENT)
I20260709 12:19:12.199203 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.197957 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:12.170629  1215 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:12.202463 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.035633  1291 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:12.203342 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 231601us
I20260709 12:19:12.204463 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:12.204759 28291 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:12.205209 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:12.167922  1393 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:12.209117 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.211683 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:12.216964 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:12.227198 28330 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.220s (SENT)
I20260709 12:19:12.238302 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.238986 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.174376  1221 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:19:12.200336  1330 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.224678 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:12.226276  1370 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.226640   908 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:19:12.235075  1388 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:12.181452  1212 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.239913 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:12.156088  1309 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.211158   928 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:12.224318  1350 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:12.253301  1333 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:12.202971  1321 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 12:19:12.262504 28330 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.243s (SENT)
I20260709 12:19:12.206540  1276 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:19:12.275956 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 863) took 1852 ms. Trace:
I20260709 12:19:12.276332  1271 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.276971  1271 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:12.265343 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 52313.5us
I20260709 12:19:12.278546 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:12.252180  1305 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:12.249569   792 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:12.270048 28985 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:12.082779  1361 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.287698   904 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:19:12.290778  1383 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:19:12.271445  1241 consensus_meta.cc:287] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.740s	user 0.011s	sys 0.003s
I20260709 12:19:12.292441   952 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.293105   952 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.293979 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 925) took 1143 ms. Trace:
W20260709 12:19:12.274243 28330 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.679s (SENT)
I20260709 12:19:12.294979 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.263461  1354 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:12.276345 28286 rpcz_store.cc:276] 0709 12:19:10.423313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.436355 (+ 13042us) service_pool.cc:224] Handling call
0709 12:19:10.475152 (+ 38797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.475412 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:10.542654 (+ 67242us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.085266 (+1542612us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b180002bde0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a522 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:19:12.275389 (+190123us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b180002bde0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287
0709 12:19:12.275568 (+   179us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:12.275614 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:12.275664 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.275745 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.275861 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21555,"prepare.queue_time_us":150957,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":101406,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":4201,"spinlock_wait_cycles":193536000,"thread_start_us":149430,"threads_started":2}
I20260709 12:19:12.285597  1396 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:12.286437   792 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:12.274578 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:12.303953 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.298959  1379 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.270808  1330 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:12.305055 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.294539 28433 rpcz_store.cc:276] 0709 12:19:11.150525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.150719 (+   194us) service_pool.cc:224] Handling call
0709 12:19:11.151137 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:11.493300 (+342163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:11.493332 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:11.506623 (+ 13291us) log.cc:844] Serialized 26 byte log entry
0709 12:19:11.734944 (+228321us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:11.734987 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.735083 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:12.293711 (+558628us) raft_consensus.cc:1685] finished
0709 12:19:12.293763 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.293911 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":470086,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"thread_start_us":466342,"threads_started":2,"wal-append.queue_time_us":636254}
I20260709 12:19:12.308662 28360 raft_consensus.cc:2410] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:19:12.271719  1212 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.315840 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:12.316097 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:12.321920 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:12.283411  1353 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.327041 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2569) took 1238 ms. Trace:
I20260709 12:19:12.328312 28283 rpcz_store.cc:276] 0709 12:19:11.088668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.401456 (+312788us) service_pool.cc:224] Handling call
0709 12:19:12.087454 (+685998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.162733 (+ 75279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.326815 (+164082us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b540009ecd0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:12.326969 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576960768}
W20260709 12:19:12.329578 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48242.9us
I20260709 12:19:12.329967 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:12.332880 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.299749  1241 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.338084 28362 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 10.
I20260709 12:19:12.298990  1271 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.351954 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.351766   792 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.359395  1330 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:12.309630  1221 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:12.361191  1221 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.361591  1221 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.362360  1342 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.362994  1342 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:12.363471  1342 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:12.364399  1342 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.364774 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:12.364933  1052 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.366343 28361 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
W20260709 12:19:12.367664  1398 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:19:12.368386 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.368983 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:12.368719  1330 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.370849  1291 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.367516 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:12.333454  1080 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:12.332688   565 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:12.377915 28330 leader_election.cc:336] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.535s (SENT)
I20260709 12:19:12.378240 28364 raft_consensus.cc:2410] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:12.338018  1216 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:12.307179   968 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:19:12.382738 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:12.383865 28330 leader_election.cc:336] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.190s (SENT)
I20260709 12:19:12.385036 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:12.385233  1328 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:12.386993 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.390683  1353 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.391237  1353 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.391894 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:12.393303  1353 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.393952  1333 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:12.396169 28285 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:12.397261 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:12.399823 28330 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.235s (SENT)
I20260709 12:19:12.400190 28290 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:12.400731 28290 raft_consensus.cc:606] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:12.401098 28290 raft_consensus.cc:614] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:12.394134 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:12.333632  1275 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:19:12.407209 28283 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:12.407799 28283 raft_consensus.cc:606] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:12.408140 28283 raft_consensus.cc:614] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:12.365633  1242 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.409410 28440 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:12.410292 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:19:12.410763  1242 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.327476  1379 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.377591  1080 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.430169  1080 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:12.446825 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 782) took 1440 ms. Trace:
I20260709 12:19:12.447299 28989 rpcz_store.cc:276] 0709 12:19:11.006217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.006439 (+   222us) service_pool.cc:224] Handling call
0709 12:19:11.013416 (+  6977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:11.507250 (+493834us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:11.576655 (+ 69405us) log.cc:844] Serialized 14 byte log entry
0709 12:19:11.908448 (+331793us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:11.908505 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:11.908557 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.316579 (+408022us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b18004ae420. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb7682a6 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419
0709 12:19:12.446548 (+129969us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b5400762d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:12.446597 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.446734 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5610,"prepare.queue_time_us":71264,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":101,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":16753,"spinlock_wait_cycles":1141215360,"thread_start_us":329566,"threads_started":1,"wal-append.queue_time_us":656317}
W20260709 12:19:12.453691  1329 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 1.457s	user 0.001s	sys 0.004s
I20260709 12:19:12.454259  1216 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:12.477149 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2904) took 1059 ms. Trace:
I20260709 12:19:12.417591  1215 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:12.414189  1328 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.477602 28432 rpcz_store.cc:276] 0709 12:19:11.417625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.417763 (+   138us) service_pool.cc:224] Handling call
0709 12:19:11.456108 (+ 38345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:11.699501 (+243393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:11.699538 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:11.700182 (+   644us) log.cc:844] Serialized 52 byte log entry
0709 12:19:11.883701 (+183519us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:11.926029 (+ 42328us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.440725 (+514696us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b180029d760. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:19:12.440930 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.441008 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:12.441053 (+    45us) raft_consensus.cc:1685] finished
0709 12:19:12.476888 (+ 35835us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.477070 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63898,"prepare.queue_time_us":42086,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":12005,"raft.queue_time_us":290917,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":12282,"spinlock_wait_cycles":9367424,"thread_start_us":372575,"threads_started":2,"wal-append.queue_time_us":291621}
I20260709 12:19:12.383797   565 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:12.489099 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 66673.5us
I20260709 12:19:12.490728 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.496961  1218 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:19:12.500402  1309 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:12.501399  1315 consensus_meta.cc:287] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 1.140s	user 0.008s	sys 0.003s
I20260709 12:19:12.514663  1315 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.515755 28441 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:12.511546   565 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.489084  1212 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.505476  1327 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:12.518227 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 535338us
I20260709 12:19:12.518739  1319 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.519117 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.520637  1314 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:12.521215  1314 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:12.513799  1221 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.520891  1327 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.519224  1315 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.522640  1364 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.524315  1221 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.519279  1319 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.529608  1370 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:12.530256  1370 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.532222  1370 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.522136  1314 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.533883  1319 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.535307  1379 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.482714  1328 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:12.518450  1366 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.520506 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.536684  1366 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.487849  1363 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:12.495633  1276 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:12.525944  1404 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:19:12.545251 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:12.540920  1319 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:12.475324  1216 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.551337 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:12.554762 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:12.555384 28290 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:12.556596  1370 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:12.552444 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.557837 28367 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:12.562628 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:12.563303 28367 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:12.548514  1080 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.544411  1377 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.572054  1201 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.572304 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.572729  1201 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.574920  1201 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.572131 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:12.576427  1391 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.285s	user 0.000s	sys 0.003s
I20260709 12:19:12.576952  1201 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.577840 28404 leader_election.cc:400] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:12.552248  1363 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.578361 28404 leader_election.cc:403] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:19:12.580446 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.582979 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:12.583410  1366 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.577759 28982 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:12.586814 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:12.588683 28290 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:12.590129 28364 raft_consensus.cc:2376] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16. Re-sending same reply.
I20260709 12:19:12.557099  1370 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:12.587858  1382 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:12.594172 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:12.604280 28367 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:12.595912  1370 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.611769  1364 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:12.605831  1366 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:12.593727  1201 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:12.589685  1205 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.560185  1319 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.612391 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.613504  1195 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.607431   792 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:12.622272 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.611387 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:12.623803 28290 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:12.612851  1216 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.607558  1379 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:19:12.629864 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:12.632704 28433 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:12.634274 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.638019 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.628328  1364 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:12.639422 28290 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:12.639546 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:12.626550  1366 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.626915   792 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:12.627915  1382 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:12.643831 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:12.655030 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:12.659401 28435 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:12.655108 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:12.660987 28440 raft_consensus.cc:2410] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:19:12.650332  1382 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.649487  1216 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:19:12.667919  1382 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.668545  1382 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.669060  1417 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:19:12.669679  1414 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:19:12.642539  1212 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:12.633554  1205 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:19:12.670202 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:12.672109  1205 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:12.672503  1205 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:12.669039  1382 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.674629  1319 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.674871  1382 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.675782  1363 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.676306 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:12.677150  1363 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:12.677520  1363 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:12.677928  1363 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:12.680034 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:12.680821  1319 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.681344  1319 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.683491  1319 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.684998  1416 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:12.685817  1327 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:12.687717  1368 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:12.688298  1368 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.689193 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.691567  1416 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:12.694002 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:12.700086 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:12.700651 28437 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:12.703629 28433 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:19:12.705618  1327 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:12.708084  1327 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.711409 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:12.712085 28434 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:12.713358 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:12.713965 28434 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:12.705401  1364 raft_consensus.cc:2764] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 12:19:12.717490  1379 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:19:12.722203  1379 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:12.722608  1379 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:12.723565  1379 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.724939  1379 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:12.725868 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.708742  1336 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:12.694279  1382 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:19:12.746579 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:12.673444  1205 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.716919  1416 raft_consensus.cc:2764] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:19:12.696357  1363 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.759111 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:12.763593  1363 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.764441 28433 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:12.764773 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 70832.1us
I20260709 12:19:12.765566 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:12.767343 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:12.768651   935 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:12.769073   935 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.769388   935 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:12.771286 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:12.775409 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:12.776674 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:12.777415   908 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:12.777832   908 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.779237   908 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:12.778352  1264 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:12.782121  1264 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:12.783030  1264 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.778823  1323 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:12.784930  1323 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:12.785341  1323 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.770364  1216 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:12.790644 28332 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:12.791062  1205 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.790585  1241 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:12.800050  1416 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.805147 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2589) took 1300 ms. Trace:
I20260709 12:19:12.805573 28288 rpcz_store.cc:276] 0709 12:19:11.504807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.089130 (+584323us) service_pool.cc:224] Handling call
0709 12:19:12.089557 (+   427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:12.090097 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:12.090124 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:12.155799 (+ 65675us) log.cc:844] Serialized 52 byte log entry
0709 12:19:12.310288 (+154489us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:12.348807 (+ 38519us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.375691 (+ 26884us) log.cc:844] Serialized 14 byte log entry
0709 12:19:12.502806 (+127115us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b180021fd00. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a522 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:19:12.581831 (+ 79025us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b180021fd00. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:19:12.804589 (+222758us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b180021fd00. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:19:12.804798 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.804875 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:12.804920 (+    45us) raft_consensus.cc:1685] finished
0709 12:19:12.804939 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.805080 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37454,"prepare.queue_time_us":40159,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":1190,"raft.queue_time_us":443262,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":67746,"spinlock_wait_cycles":245047808,"thread_start_us":107182,"threads_started":1,"wal-append.queue_time_us":207035}
W20260709 12:19:12.809397  1374 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.809s	user 0.004s	sys 0.001s
I20260709 12:19:12.725916   792 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:12.811784 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 438473us
I20260709 12:19:12.794389  1215 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:12.814635  1215 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:12.816817  1387 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.003s	sys 0.012s
W20260709 12:19:12.833662  1387 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.003s	sys 0.012s
W20260709 12:19:12.836777 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 71284.1us
I20260709 12:19:12.818085  1215 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.821333  1416 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.842634 28364 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:12.846129 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:12.847041 28363 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.851604 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.857435 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.858464 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:12.859807 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:12.863248 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.864804 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:12.869338 28363 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:12.870714 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.870482 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:12.854812  1344 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.877566 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.842005  1382 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:19:12.881076 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:12.851884  1328 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.871343  1393 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:12.888595  1344 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.892812  1342 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:12.852057  1416 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.900935   952 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.912238  1307 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.912709  1366 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.882579  1330 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:19:12.941720 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.921526  1344 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:12.942952 28286 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:12.923556  1342 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:12.937405 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:12.878496  1370 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.960095 28285 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:12.885331  1195 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:19:12.956353 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:12.956668  1080 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.889885  1314 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:12.962486 28432 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:12.968248   908 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:12.969925  1416 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:12.930003  1397 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:12.972492  1323 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:12.970000  1172 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:19:12.958806  1201 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:12.982236 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.986253 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.987048 28364 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:12.990141 28439 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:12.990562 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:12.992049 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:12.992754  1403 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.698s	user 0.000s	sys 0.004s
I20260709 12:19:13.000623 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:12.992888 28359 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:13.005395 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:13.007529 28361 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:12.974572  1342 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:13.018677 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2551) took 2249 ms. Trace:
I20260709 12:19:13.019104 28287 rpcz_store.cc:276] 0709 12:19:10.764884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.765153 (+   269us) service_pool.cc:224] Handling call
0709 12:19:13.013931 (+2248778us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54002413d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:13.014088 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2609706880}
I20260709 12:19:13.019577 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:13.028954 28359 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:12.978865  1314 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:13.071273  1363 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:13.074715  1363 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:13.075548  1323 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:13.076962  1363 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:13.077893  1314 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:13.078068  1431 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:19:13.091840 28404 leader_election.cc:336] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.519s (SENT)
I20260709 12:19:13.107975 28440 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:13.109207 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.100068  1328 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:13.118366 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.120517 28437 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:13.121963 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.123132 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:13.124653 28404 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.843s (SENT)
I20260709 12:19:13.125172 28435 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:19:13.125121 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:13.124023  1172 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.122381   908 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:13.131982   792 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:13.133510   792 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:13.133946   792 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:13.131551  1314 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.152618 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:13.156529 28985 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:13.173287 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:13.175361 28363 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:13.181674   928 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:19:13.178972  1433 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:19:13.133036  1172 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:13.195094  1172 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:13.206730  1438 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:19:13.234288 28366 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:13.217897  1215 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:13.236426  1342 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:13.238078  1215 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:13.238719 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47753.5us
I20260709 12:19:13.234366   792 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:13.240353 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:13.250752  1442 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:13.253638 28984 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:13.254237  1172 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.256872 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:13.270514  1172 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:13.274269   792 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:13.275321  1172 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:13.276137  1172 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:13.276659  1172 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:13.243933   935 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:13.280330   935 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:13.285152 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 954714us
I20260709 12:19:13.287173 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:13.288713 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:13.259742   968 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:13.389581 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:13.396057 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:13.348709  1342 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:13.407405 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.348884  1328 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:13.416138 28289 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:13.389564   968 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:13.424535 28292 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 12:19:13.398341   928 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:13.431388 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.384864  1344 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:19:13.433158 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.434391 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.435175 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.443696 28285 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:13.437410 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.445165 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.440795 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.440690 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.447219 28288 raft_consensus.cc:2368] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18.
I20260709 12:19:13.449115 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.411661  1195 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.451395 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.452829 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.453943 28289 raft_consensus.cc:2410] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:13.459417 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.460356 28285 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:13.402410  1352 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:19:13.468930 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.472599  1368 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:13.488113 28285 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:13.480463 28287 raft_consensus.cc:2368] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:13.489921 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:13.521204 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 264395us
I20260709 12:19:13.466943  1215 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.523805 28290 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:13.526473 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.531579 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:13.523881  1324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.231s	user 0.032s	sys 0.057s
I20260709 12:19:13.538483 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.539265 28433 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:19:13.552889 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:19:13.554524  1324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.033s	sys 0.058s
W20260709 12:19:13.554725  1357 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.196s	user 0.005s	sys 0.004s
I20260709 12:19:13.556835 28291 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:13.561812 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.564409 28285 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:13.565796 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.532099   968 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:13.567171   968 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.568181 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.571792 28437 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:13.552052 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:13.579275 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.580870  1218 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:13.581329  1218 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.581691  1218 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:13.582258 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.534396   935 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:19:13.527570  1215 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:13.595463 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:13.608500 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:13.611047  1368 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:13.622767 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:13.625792 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2912) took 1848 ms. Trace:
I20260709 12:19:13.626209 28441 rpcz_store.cc:276] 0709 12:19:11.777326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.777448 (+   122us) service_pool.cc:224] Handling call
0709 12:19:11.777784 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.516151 (+738367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.625512 (+1109361us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b540026b1d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:13.625707 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1696726784}
W20260709 12:19:13.621599 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.632068 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.632879 28439 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:13.634037 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.638753  1215 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:13.641186 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:13.642738 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:13.625090  1397 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.636169  1363 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:13.646905  1435 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.563s	user 0.003s	sys 0.000s
I20260709 12:19:13.654596  1363 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:13.651026  1397 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:13.689863  1363 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.705549  1218 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:13.708197  1218 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:13.700719 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:13.697364  1172 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:13.712379  1276 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:13.723448 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:13.724167 28439 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:19:13.728106  1172 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:13.730211  1397 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:13.738869 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 887) took 1184 ms. Trace:
I20260709 12:19:13.739414 28286 rpcz_store.cc:276] 0709 12:19:12.554349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.556033 (+  1684us) service_pool.cc:224] Handling call
0709 12:19:12.565684 (+  9651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.948353 (+382669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.738559 (+790206us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54000affd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:13.738790 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":867197184}
I20260709 12:19:13.753067 28439 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:13.770634 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.772069 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.772732 28439 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:13.773613 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.754748 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.762050  1370 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:13.782196 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:13.783540 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:13.799891 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:13.800621 28290 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:13.802592 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:13.802982 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 45335.6us
I20260709 12:19:13.805104 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:13.836551 28434 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:13.805826 28440 raft_consensus.cc:2410] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:13.806197 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.837549 28291 raft_consensus.cc:2376] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14. Re-sending same reply.
I20260709 12:19:13.808350 28290 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:19:13.838692 28290 pending_rounds.cc:85] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 9
I20260709 12:19:13.838961 28290 pending_rounds.cc:107] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:19:13.839196 28290 raft_consensus.cc:2889] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:13.809780 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:13.840446 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.842373 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:19:13.798375  1448 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.149s	user 0.003s	sys 0.000s
I20260709 12:19:13.805058  1397 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.825237 28982 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:13.804737  1370 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:19:13.843711 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2585) took 1002 ms. Trace:
W20260709 12:19:13.860543  1451 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:19:13.861974   928 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:13.862519  1221 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.850696  1215 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:19:13.878331 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2391) took 1013 ms. Trace:
W20260709 12:19:13.879509 28330 leader_election.cc:336] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.888s (SENT)
I20260709 12:19:13.884610 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.893074 28287 raft_consensus.cc:2376] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17. Re-sending same reply.
I20260709 12:19:13.879987 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:13.871796 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:13.892686 28363 rpcz_store.cc:276] 0709 12:19:12.864543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.871692 (+  7149us) service_pool.cc:224] Handling call
0709 12:19:13.878078 (+1006386us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54003babd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:13.878257 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446241024}
I20260709 12:19:13.895978 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:13.898067 28330 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.931s (SENT)
I20260709 12:19:13.898264 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.898587 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:13.885465 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.881686 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.881055 28361 rpcz_store.cc:276] 0709 12:19:12.841048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.843511 (+  2463us) service_pool.cc:224] Handling call
0709 12:19:12.843839 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:13.008034 (+164195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.843445 (+835411us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54001d7c54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:13.843630 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375937792}
I20260709 12:19:13.888914 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.899526 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.890102 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.891309 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.894826 28441 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:13.882855  1354 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:13.901384 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.901796 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.889770 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:13.900434 28283 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 11: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:19:13.903064 28330 leader_election.cc:336] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.947s (SENT)
I20260709 12:19:13.902377 28286 raft_consensus.cc:2376] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16. Re-sending same reply.
I20260709 12:19:13.904264 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.905136 28291 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 20: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:13.905786 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.906563 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.907011 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.907786 28287 raft_consensus.cc:2368] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17.
I20260709 12:19:13.907796 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.908268 28285 raft_consensus.cc:2368] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18.
I20260709 12:19:13.909619 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.910334 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.904958  1324 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:13.911513 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2920) took 1266 ms. Trace:
I20260709 12:19:13.914348 28432 rpcz_store.cc:276] 0709 12:19:12.645149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.645286 (+   137us) service_pool.cc:224] Handling call
0709 12:19:12.668637 (+ 23351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.969215 (+300578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.911278 (+942063us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400416754. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:13.911450 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675328384}
I20260709 12:19:13.904529 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:13.912122 28440 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:13.913054 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.905787 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:13.915905 28290 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:13.915580 28436 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 16
W20260709 12:19:13.904234 28330 leader_election.cc:336] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.012s (SENT)
I20260709 12:19:13.917035 28283 raft_consensus.cc:2368] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18.
I20260709 12:19:13.918315 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:13.909617   935 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:13.883296  1397 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:13.922784  1397 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:13.924193 28438 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:13.889191   928 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:13.933314 28289 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:13.935418 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:13.935693 28330 leader_election.cc:336] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.015s (SENT)
I20260709 12:19:13.936587 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:13.938313 28330 leader_election.cc:336] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.029s (SENT)
I20260709 12:19:13.938776 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:13.939459  1215 raft_consensus.cc:476] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Not starting pre-election -- already a leader
I20260709 12:19:13.869681  1363 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:13.943629   935 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.948170 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:13.959271 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:13.949712  1306 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:13.959434 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:14.146014 28290 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:13.960140 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:14.147771 28283 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:13.952014   952 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:13.958626 28291 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:13.913355  1354 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:13.914058  1324 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:13.895260  1450 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 12:19:14.154386 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.234416 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:14.235145 28285 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:14.236563 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 394225us
I20260709 12:19:14.248457 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2596) took 1004 ms. Trace:
I20260709 12:19:14.154721 28441 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:19:14.189276 28434 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:13.895058  1221 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.273097 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:14.192808 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 221153us
I20260709 12:19:14.133216   928 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.229583 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:14.232580 28439 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:14.249058 28366 rpcz_store.cc:276] 0709 12:19:13.243768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.243945 (+   177us) service_pool.cc:224] Handling call
0709 12:19:13.260976 (+ 17031us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:13.261312 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.261342 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:13.269309 (+  7967us) log.cc:844] Serialized 52 byte log entry
0709 12:19:13.640612 (+371303us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:13.640658 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.640734 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.248122 (+607388us) raft_consensus.cc:1685] finished
0709 12:19:14.248153 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.248374 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":17967,"thread_start_us":230389,"threads_started":1,"wal-append.queue_time_us":328779}
I20260709 12:19:14.303507 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:14.319732 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 372614us
I20260709 12:19:14.144276  1306 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:14.338310 28436 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:14.218683   952 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:14.241076  1352 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.240824  1342 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:14.345433 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 486651us
I20260709 12:19:14.242383  1464 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:14.390159 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:14.407706 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:14.394809  1364 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.393246 28285 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:14.358958 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:14.394258 28286 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:14.398042 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:14.381465 28290 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:14.400704 28436 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:19:14.402304  1276 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:14.444967 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:14.447629 28361 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:14.271533   935 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.148s
I20260709 12:19:14.452389 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:14.453372 28984 raft_consensus.cc:2368] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:19:14.357874   928 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 12:19:14.473449 28363 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:14.474241 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:14.475134 28441 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:14.475009 28435 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:14.479744 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:14.496729 28333 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.017s	sys 0.003s
W20260709 12:19:14.499917 28333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.017s	sys 0.003s
I20260709 12:19:14.523339 28436 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:14.524305 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:14.526028 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:14.527997 28435 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:14.457208  1276 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:14.404275  1352 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.433252  1364 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.441959  1324 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.575062  1324 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:14.419600  1464 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:14.534269 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:14.414251  1342 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.458048  1465 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 12:19:14.544543  1370 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:14.587244 28291 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:14.590993 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:14.441479  1354 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.579365  1397 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:14.524273 28286 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:19:14.465328  1306 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.586619  1466 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.657928  1342 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.602627  1370 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:14.660741  1471 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:14.662000  1447 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:14.662673  1447 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:14.529933   952 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:14.687991 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 367242us
W20260709 12:19:14.689467 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 419629us
I20260709 12:19:14.643415  1363 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:14.741845  1447 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.661131  1370 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.746531 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2583) took 3341 ms. Trace:
I20260709 12:19:14.747139  1364 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:14.742456  1221 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.748708   952 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:14.747110 28292 rpcz_store.cc:276] 0709 12:19:11.404682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.086125 (+681443us) service_pool.cc:224] Handling call
0709 12:19:12.811266 (+725141us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b5800000098. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:19:12.811499 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:13.425052 (+613553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.746225 (+1321173us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54000b18d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:14.746429 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1414225152}
I20260709 12:19:14.645994  1397 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:14.757918  1465 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:14.778730  1465 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:14.779722  1212 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:14.781561  1212 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:14.783272  1212 consensus_queue.cc:1059] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:14.781889  1465 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:14.785168  1465 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:14.785444  1221 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:14.785948 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:14.782193 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:14.792511 28435 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:14.666419  1466 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.760020  1476 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:14.803639  1352 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.786099  1221 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:14.853149 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 2938) took 1248 ms. Trace:
I20260709 12:19:14.855542 28439 rpcz_store.cc:276] 0709 12:19:13.604408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.773399 (+168991us) service_pool.cc:224] Handling call
0709 12:19:14.852943 (+1079544us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54003ca854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:14.853078 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1053797248}
W20260709 12:19:14.857335 28258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.010s	sys 0.036s
I20260709 12:19:14.807837  1484 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:19:14.858784 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 260092us
W20260709 12:19:14.859279 28284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.012s	sys 0.026s
I20260709 12:19:14.838721  1215 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:19:14.861624 28284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.012s	sys 0.026s
I20260709 12:19:14.861325 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:14.858875 28258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.010s	sys 0.036s
I20260709 12:19:14.856986  1475 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:14.883076 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:14.874877 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.859280  1352 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:19:14.919950  1352 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.921408  1352 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.872694   935 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:14.924961   935 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.925392   968 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:14.925985   968 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:14.926777  1484 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.927294   935 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.927863   935 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.928006   968 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:14.928332   935 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.929781 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:14.930486 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:14.931494   935 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.927330  1484 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.932260   935 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:14.932940   935 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:14.934334  1484 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.935273  1484 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:14.935676 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:14.935247   935 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:14.938284   935 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:14.938831   935 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:14.807194  1476 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:14.875880  1478 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:14.952813 28954 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.013s	sys 0.005s
W20260709 12:19:14.965065 28954 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.013s	sys 0.005s
I20260709 12:19:14.863734  1215 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:14.968433   935 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.976002 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:14.980844 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:14.981705 28290 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
W20260709 12:19:14.982419 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 289506us
I20260709 12:19:14.984491 28290 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:14.915373  1475 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:14.992197 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:14.998256 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:14.999022 28436 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:15.000260 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:15.006614 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 316360us
I20260709 12:19:15.013091 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.014142 28441 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:14.997267  1328 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.002287   928 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.084157 28291 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:15.095553 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:19:15.100782 28404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.017s	sys 0.019s
W20260709 12:19:15.102783 28404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.017s	sys 0.019s
I20260709 12:19:15.021482  1397 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:15.067727  1481 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.229s	user 0.000s	sys 0.000s
I20260709 12:19:15.078684  1363 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:14.973716  1478 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.020862  1464 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.068593   968 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.065001  1352 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.069120  1483 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.127453 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2513) took 1517 ms. Trace:
I20260709 12:19:15.108592  1471 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:19:15.065557  1475 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.068830  1487 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:15.134397 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.074676  1496 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:15.074436  1364 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:15.144116 28434 rpcz_store.cc:276] 0709 12:19:13.609787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.263505 (+653718us) service_pool.cc:224] Handling call
0709 12:19:14.263975 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.264429 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.264463 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:14.298733 (+ 34270us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.798477 (+499744us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:15.106533 (+308056us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.127098 (+ 20565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.127187 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.127233 (+    46us) raft_consensus.cc:1685] finished
0709 12:19:15.127245 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.127386 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20209,"raft.queue_time_us":22584,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":46805,"thread_start_us":656256,"threads_started":2,"wal-append.queue_time_us":593578}
I20260709 12:19:15.139173 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:15.167855 28330 leader_election.cc:400] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18.
I20260709 12:19:15.105618 28404 leader_election.cc:400] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18.
I20260709 12:19:15.171341 28404 leader_election.cc:403] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:15.069321  1485 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.169785 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.179888  1221 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.146122  1363 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.182615  1363 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.184134  1476 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.143694  1466 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:15.184737  1449 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:15.111030   928 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:15.198982 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 93545.8us
I20260709 12:19:15.200071 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:15.173998 28955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.011s	sys 0.025s
I20260709 12:19:15.054488  1215 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.117118  1328 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.139323   935 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:15.176295 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:15.201503 28955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.012s	sys 0.025s
I20260709 12:19:15.145557  1478 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.203117 28330 leader_election.cc:400] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17.
I20260709 12:19:15.145251  1475 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.217806 28330 leader_election.cc:403] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:19:15.172735  1471 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:15.214257 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:15.202153  1276 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.622s	user 0.012s	sys 0.001s
I20260709 12:19:15.218678 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2618) took 1276 ms. Trace:
I20260709 12:19:15.186265  1291 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:15.143280   968 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.207943  1466 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:19:15.217914  1450 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.001s	sys 0.034s
I20260709 12:19:15.218065  1363 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.221758 28284 rpcz_store.cc:276] 0709 12:19:13.942203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.942319 (+   116us) service_pool.cc:224] Handling call
0709 12:19:13.942684 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:13.956167 (+ 13483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.956202 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:14.142711 (+186509us) log.cc:844] Serialized 52 byte log entry
0709 12:19:14.864363 (+721652us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:14.909120 (+ 44757us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.212412 (+303292us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.217498 (+  5086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.218367 (+   869us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.218425 (+    58us) raft_consensus.cc:1685] finished
0709 12:19:15.218438 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.218613 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115405,"prepare.queue_time_us":50256,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":104,"raft.queue_time_us":95083,"raft.run_cpu_time_us":536,"raft.run_wall_time_us":198679,"thread_start_us":845143,"threads_started":2,"wal-append.queue_time_us":924505}
I20260709 12:19:15.234071 28358 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:15.236791  1328 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:15.240847  1489 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:19:15.242476 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2510) took 1651 ms. Trace:
I20260709 12:19:15.226433  1276 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.242935 28433 rpcz_store.cc:276] 0709 12:19:13.591365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.865012 (+273647us) service_pool.cc:224] Handling call
0709 12:19:13.865400 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.214049 (+348649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.214094 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:14.354567 (+140473us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.809641 (+455074us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:15.109382 (+299741us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.127808 (+ 18426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.127892 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.242162 (+114270us) raft_consensus.cc:1685] finished
0709 12:19:15.242193 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.242391 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21209,"raft.queue_time_us":76302,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":146,"thread_start_us":761567,"threads_started":3,"wal-append.queue_time_us":599466}
I20260709 12:19:15.210497  1324 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:19:15.247134 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:15.210084  1449 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:15.220378  1501 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:15.221009  1500 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:15.220765  1478 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.177029  1364 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.229686  1291 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:15.252563 28956 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:19:15.258375 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44223.9us
W20260709 12:19:15.261009 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 395114us
I20260709 12:19:15.251588  1276 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:15.231086  1450 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.002s	sys 0.035s
W20260709 12:19:15.258817 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 472961us
I20260709 12:19:15.229938  1471 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17.
I20260709 12:19:15.258431  1363 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.144335  1483 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.279263  1483 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.139608  1495 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.144s
I20260709 12:19:15.141605  1493 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.145s
I20260709 12:19:15.167557  1487 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.273533 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:15.202282  1485 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.172905  1496 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.287072 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.288712 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:15.265264  1172 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:15.293287 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:15.309120 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.310901 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:15.310957 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.313621 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.314702 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.308981 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:15.315433 28360 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:15.316324 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:15.317199 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:15.306172 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.259375  1478 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.319249 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:15.322518 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.322644 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:15.314785 28365 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:15.290102  1476 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:15.287808  1328 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:19:15.257194  1501 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.313057  1172 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.314045  1464 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:15.305187  1487 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.323796  1397 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:15.317603  1494 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.258039  1500 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.345574 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.360632 28360 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:15.317974  1350 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:15.377099 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:15.317849  1504 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:15.346127 28363 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:15.360108 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:15.362588 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46378.3us
I20260709 12:19:15.366420  1508 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.357833  1464 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.334383  1491 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.350244  1172 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:15.367730 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.320487  1370 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:15.342337  1490 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:15.364629   928 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:15.370735 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 147769us
I20260709 12:19:15.402621 28364 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:15.371236 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:15.313827  1449 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:15.374866  1499 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:19:15.331763  1291 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.355456  1397 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:15.349898 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.389927  1485 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:14.436497 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:15.380105 28365 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:19:15.444135  1501 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.452391   928 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:15.452996 28362 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:15.400949  1491 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.443429  1464 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.455945  1491 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.449848 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:15.395136 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:19:15.415201 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44047.1us
W20260709 12:19:15.460034 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 137577us
I20260709 12:19:15.334030  1354 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.403508 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:15.462539 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 59092us
I20260709 12:19:15.463347 28330 leader_election.cc:400] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18.
I20260709 12:19:15.398072 28360 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:15.378845  1306 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:15.458442 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2518) took 1651 ms. Trace:
W20260709 12:19:15.458907 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63875.2us
W20260709 12:19:15.465844 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.380263  1504 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:15.402835  1502 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:19:15.405208  1478 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.379613  1368 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:15.441929  1364 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.497561 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.447247  1215 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.402671   968 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.484766 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.485442 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:15.466938 28432 rpcz_store.cc:276] 0709 12:19:13.807244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.423813 (+616569us) service_pool.cc:224] Handling call
0709 12:19:14.424137 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:14.873553 (+449416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:14.873592 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:14.953659 (+ 80067us) log.cc:844] Serialized 26 byte log entry
0709 12:19:15.290885 (+337226us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:15.329674 (+ 38789us) log.cc:844] Serialized 14 byte log entry
0709 12:19:15.402313 (+ 72639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.402496 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.458195 (+ 55699us) raft_consensus.cc:1685] finished
0709 12:19:15.458216 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.458378 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1877,"prepare.queue_time_us":42355,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":158451,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":40975,"thread_start_us":792881,"threads_started":3,"wal-append.queue_time_us":396458}
I20260709 12:19:15.488437 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:15.437984  1470 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.794s	user 0.010s	sys 0.000s
W20260709 12:19:15.490492 28185 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 117) took 21486 ms (client timeout 15000 ms). Trace:
I20260709 12:19:15.502195 28434 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:15.469220  1491 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:15.495388 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.407521  1496 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.498252 28436 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:15.467023  1397 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.464284 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.493466  1482 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.409566 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.466821  1501 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.438133  1350 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.492631 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:15.543569 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 51015us
I20260709 12:19:15.490129  1464 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:19:15.505092  1509 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.493896  1181 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.497463  1465 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
W20260709 12:19:15.502595 28185 rpcz_store.cc:269] 0709 12:18:53.971227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.973268 (+  2041us) service_pool.cc:224] Handling call
0709 12:19:15.457980 (+21484712us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:08.629128 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:08.629937 (+   809us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:08.630000 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:19:08.630021 (+    21us) tablet.cc:662] Decoding operations
0709 12:19:08.711199 (+ 81178us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:08.711326 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:08.711353 (+    27us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:19:08.712674 (+  1321us) tablet.cc:701] Row locks acquired
0709 12:19:08.712730 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:19:08.712903 (+   173us) write_op.cc:270] Start()
0709 12:19:08.713062 (+   159us) write_op.cc:276] Timestamp: P: 1783599548712834 usec, L: 0
0709 12:19:08.713101 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:19:08.713965 (+   864us) log.cc:844] Serialized 6320 byte log entry
0709 12:19:09.274232 (+560267us) op_driver.cc:464] REPLICATION: finished
0709 12:19:09.492203 (+217971us) write_op.cc:301] APPLY: starting
0709 12:19:09.495007 (+  2804us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:09.610997 (+115990us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:09.611143 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:09.799469 (+188326us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:09.830392 (+ 30923us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:19:09.830471 (+    79us) write_op.cc:312] APPLY: finished
0709 12:19:09.830996 (+   525us) log.cc:844] Serialized 148 byte log entry
0709 12:19:09.831224 (+   228us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:09.831415 (+   191us) write_op.cc:454] Released schema lock
0709 12:19:09.831497 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:09.832081 (+   584us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:09.832120 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4787333,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":212480,"apply.run_cpu_time_us":24437,"apply.run_wall_time_us":339831,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70975,"num_ops":16,"prepare.queue_time_us":222573,"prepare.run_cpu_time_us":16444,"prepare.run_wall_time_us":344191,"raft.queue_time_us":111341,"raft.run_cpu_time_us":21572,"raft.run_wall_time_us":138622,"replication_time_us":560927,"thread_start_us":600179,"threads_started":4,"wal-append.queue_time_us":128770,"wal-append.run_cpu_time_us":22415,"wal-append.run_wall_time_us":2004403}]]}
I20260709 12:19:15.510963 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:15.516831  1491 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:15.514075 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.514798 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.515551  1342 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:15.447777  1500 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.491684   928 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.523216 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.411017   948 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.503172 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.544445 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:15.545166 28364 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:15.561512 28364 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:19:15.545677 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 81480.3us
I20260709 12:19:15.496353  1504 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:15.409461  1490 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.583753 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:15.511718 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.489240  1354 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.545783 28332 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:15.534143  1494 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:15.600508 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.563519  1521 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.562565 28364 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.564545   948 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.567595   792 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:15.574612 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.548296  1487 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:15.554878  1476 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:15.555903  1491 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.547660  1485 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:15.614212 28362 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:15.555213  1181 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.584476  1328 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:15.554574  1342 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:19:15.601585 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 55919.1us
I20260709 12:19:15.602393 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.542662  1370 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:19:15.604406 28364 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:15.605990 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45007.9us
I20260709 12:19:15.603464  1363 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.615346 28438 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 21: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:15.543962  1482 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.602977  1519 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:19:15.621629  1523 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:19:15.557253  1352 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:19:15.548934  1350 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.606793   792 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:15.605178  1500 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:15.631042 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:15.621076  1354 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.636567 28364 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 12:19:15.657141 28364 pending_rounds.cc:85] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 11
I20260709 12:19:15.657449 28364 pending_rounds.cc:107] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 12:19:15.605127  1465 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:15.647004 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 102650us
I20260709 12:19:15.642212  1475 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.664337  1493 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:15.670328  1493 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:19:15.670782  1512 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.291s	user 0.003s	sys 0.000s
I20260709 12:19:15.670782  1493 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:15.614244  1485 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.672359  1485 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:15.611167  1172 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.675868  1172 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.619184  1328 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.681488  1328 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:15.586916   928 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.604734  1368 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 12:19:15.689507  1526 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:19:15.659909  1476 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:15.657707 28364 raft_consensus.cc:2889] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:15.661350 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.669766 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:15.644899 28332 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:19:15.687928 28286 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2606) took 1789 ms (client timeout 1950 ms). Trace:
I20260709 12:19:15.659785  1500 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.655869 28404 leader_election.cc:400] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:15.693305 28365 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:15.693794  1350 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.695693   935 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:15.651553  1483 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:19:15.731148  1500 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:15.763813 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:15.767490 28292 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.697837  1478 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:15.699016 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.700795  1181 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:15.701444 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.776304 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.702337 28284 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:15.703007  1364 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.667315  1212 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:19:15.836722 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:15.703475  1368 consensus_queue.cc:1243] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:15.705134 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:15.845611 28289 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:15.667881  1342 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.686369  1465 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:15.714115 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:15.714303 28286 rpcz_store.cc:269] 0709 12:19:13.898190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.903993 (+  5803us) service_pool.cc:224] Handling call
0709 12:19:15.687729 (+1783736us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b540018f454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:15.687861 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1121985408}
I20260709 12:19:15.859617 28290 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
W20260709 12:19:15.668401 28330 connection.cc:256] RPC call timeout handler was delayed by -1.5645441889998892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
I20260709 12:19:15.725371 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:15.726169 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 75347.1us
W20260709 12:19:15.727846 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 82989.3us
I20260709 12:19:15.728335 28990 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:15.729017 28283 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:19:15.729434 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:15.729816 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.863369 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:15.863588 28438 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:15.730182 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:15.769393 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 101525us
I20260709 12:19:15.769454 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:15.780433 28436 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
W20260709 12:19:15.783061 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 50280.7us
I20260709 12:19:15.710503   928 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.731874  1172 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:19:15.887367 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:15.887763 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.888561 28441 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 13: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:15.890811 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:15.883876 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:15.726727  1306 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.784579  1530 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.900883  1530 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.901901 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.785270 28361 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:15.785017  1467 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:15.902328 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.724843  1352 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.904309  1530 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.904062  1352 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.904722 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.905165 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:15.766396  1472 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:19:15.803355  1482 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.712445  1476 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:15.930413  1476 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:15.931583  1476 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:15.817783  1493 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.814196  1500 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.822433 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:15.765937  1532 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:19:15.840420 28439 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 18: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:15.849548 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:15.945281 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:15.751123  1471 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:15.855311 28292 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:15.864967 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:15.770644  1478 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:15.870575 28985 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:15.871181 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.953858 28436 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:15.872566 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.861688 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:15.955366 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 90488.1us
W20260709 12:19:15.955744 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 94157.1us
I20260709 12:19:15.956786 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:15.956964 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:15.957922   928 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:15.958405   928 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:15.958065  1528 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:15.958879   928 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:15.959622  1350 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:15.960109  1350 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.960522  1350 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:15.781518  1503 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:15.962023  1503 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.962482  1503 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:15.873452 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:15.858333  1368 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:15.846576  1364 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.883737  1483 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:15.885128  1490 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.870079  1485 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:15.885937  1529 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.893618 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.986593 28434 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 9.
I20260709 12:19:15.889559 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.894364 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:15.987898 28290 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:19:15.989006 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:15.989787 28290 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:15.896603 28286 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:15.991959 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:15.899824 28289 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:19:15.869475  1496 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.907534 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:16.010264 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 102832us
I20260709 12:19:16.011138 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:15.925635 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:15.923673 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 54522us
I20260709 12:19:15.906387  1352 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:15.905664 28437 raft_consensus.cc:2410] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:19:15.926148 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.927320 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:15.935696 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:16.016427 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.949640 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:16.017089 28359 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:15.906296  1221 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.874641  1475 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:15.967530 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:15.968393 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.969272 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:15.967406 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:15.969954 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.022120 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.015223  1496 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.023366 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.023698  1328 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.024618  1496 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.025134  1475 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:16.025571  1475 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.025840  1276 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:16.026633  1276 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:16.027671  1276 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:16.027876 28437 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:16.028319 28437 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:16.029922  1181 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:16.030224  1473 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:19:16.031363 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 64032.4us
I20260709 12:19:16.031934  1221 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.032234 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:16.033277 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:15.970371 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.034263  1221 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.034849  1221 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:15.970453 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:15.985918  1530 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:15.975209 28404 connection.cc:256] RPC call timeout handler was delayed by -1.5174777230001837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
I20260709 12:19:15.923291  1370 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:16.024857   952 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:16.040436   952 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.040856   952 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:15.869588  1397 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:16.044001 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:16.045403 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:16.045939 28291 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:16.046670 28441 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:16.019734  1449 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:16.047988 28291 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:19:16.048130 28436 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:15.978691 28989 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:15.990935 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:16.012689 28330 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.019464  1529 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.051978  1472 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:16.037395  1370 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.020825 28367 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:16.053048  1529 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:15.751336   792 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:16.054261   792 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:16.056335   928 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:16.017932 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 90736.1us
W20260709 12:19:16.057960  1492 consensus_queue.cc:1175] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a3de34b6f4cf4606a61944db6e2739c8" responder_term: 11 status { last_received { term: 10 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:16.059088  1364 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:16.060797  1493 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.953088  1471 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.061998  1471 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.062427 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:16.062980  1500 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.063422 28987 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 19: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:16.065024 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:15.923929  1306 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.067391   928 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:16.043871 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:16.068410   948 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.020323 28361 raft_consensus.cc:2368] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15.
I20260709 12:19:16.050247  1363 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:16.069638 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.035866  1478 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.039491  1352 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:16.029114 28437 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.070221 28330 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.071147  1363 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:16.071656  1363 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.071935 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.073261  1483 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:16.039927 28404 connection.cc:256] RPC call timeout handler was delayed by -1.39323473200011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 12:19:16.074150 28404 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.4:39985 timed out after 1.697s (SENT) [suppressed 17 similar messages]
I20260709 12:19:16.039860 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:16.075242  1397 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:16.075937  1397 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:16.076184  1352 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.076618  1352 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.078253  1518 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:16.078732  1518 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:16.080046  1342 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.061412 28433 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:16.082432 28283 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 12.
I20260709 12:19:15.982358  1368 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.084025 28437 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.085242  1368 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:16.035524  1485 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.087806 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.088030 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.088603 28362 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:16.030254  1467 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:19:16.091885  1370 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:16.075865 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:16.096515  1195 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:16.097000  1195 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:16.097384  1195 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:16.097924  1490 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:16.098884  1490 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:16.099269  1490 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:16.099727 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.100524  1483 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:16.035547  1221 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:16.067554  1306 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.016227 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.089215 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.102360 28363 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:16.104315  1276 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:16.104516 28363 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:16.105347  1221 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:16.105822  1221 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.111709  1491 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:16.114341  1276 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:16.115566  1276 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.073570  1215 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:19:16.089545 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.135301 28362 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 19 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:19:16.100826  1519 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:16.130553 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:16.147362 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.147785 28292 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:16.148056  1471 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.148854 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:16.149322 28360 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:16.130744 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:16.150007 28290 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:16.100464  1490 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:15.998520  1466 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.748s	user 0.019s	sys 0.000s
W20260709 12:19:16.135591 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.136086  1328 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.140455 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.141471   952 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.135200 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:16.196501 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:16.197139 28291 raft_consensus.cc:2393] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 19: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:16.144011  1475 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.134312  1215 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:16.154433  1352 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.156519  1397 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:16.159055 28984 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:19:16.206699  1352 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:15.886756  1465 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.163655 28437 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:19:16.208508 28437 pending_rounds.cc:85] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 9
I20260709 12:19:16.208782 28437 pending_rounds.cc:107] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:19:16.209022 28437 raft_consensus.cc:2889] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:16.208988  1195 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.210403 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:16.211021 28437 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:19:16.211289  1195 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.211941 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.174994  1306 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:16.176184 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:16.212953  1475 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.183535 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.197335 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:16.203114  1215 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:16.213665 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.213929 28292 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:19:16.214937 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:16.214449  1215 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:16.209609 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:16.214442 28437 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 18: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:16.216701 28288 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.217870 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.219489 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.221062 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:16.221666 28437 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:16.224366 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:16.202841 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:16.199419 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:16.115171  1491 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:16.216089  1215 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:16.216038 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:16.231032 28285 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:19:16.233448 28285 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:16.237779  1370 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.239406   952 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.245959 28441 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:16.199395 28986 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:16.249996 28288 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:16.277892 28288 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:16.216373  1464 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
W20260709 12:19:16.263350 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 61729.2us
I20260709 12:19:16.218045  1478 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:16.300021 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 117394us
W20260709 12:19:16.279529 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.293932 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.279345 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 103396us
I20260709 12:19:16.310492 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.311936 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:16.312588 28367 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:16.303135 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:19:16.301781 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.313622 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:16.197464  1466 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18.
I20260709 12:19:16.206661  1471 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.319916 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:16.320380 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.324259 28360 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:16.331830 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:16.333678 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:16.325475 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:16.333451 28360 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:16.211409  1504 consensus_meta.cc:287] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.000s
I20260709 12:19:16.364024  1324 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:16.336838 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:16.339816 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.341691 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:16.335860 28432 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:16.331575  1181 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:16.337726   968 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:16.341104  1475 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.346004 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:16.299176  1370 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:16.350804  1493 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:16.355969 28364 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:16.339387   948 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:19:16.362764  1503 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:16.366137 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 53662.9us
I20260709 12:19:16.407058 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:16.408936 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:16.410451 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.368315 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.360859  1363 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.278039  1490 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.370720 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.372252 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:16.411273 28437 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.411739 28360 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:16.412771 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:16.413528 28360 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:16.273684  1521 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.305224  1352 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:16.300137  1368 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:16.443562 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.444258 28433 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:19:16.315999  1306 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:16.454979 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.455631 28441 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.376197 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:16.379848 28365 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:16.380733 28367 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
W20260709 12:19:16.381181 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 307039us
W20260709 12:19:16.457185 28404 connection.cc:256] RPC call timeout handler was delayed by -1.3032347320001918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
W20260709 12:19:16.457551 28404 connection.cc:256] RPC call timeout handler was delayed by -1.24023473200009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
W20260709 12:19:16.457861 28404 connection.cc:256] RPC call timeout handler was delayed by -1.2150347320002766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
W20260709 12:19:16.458138 28404 connection.cc:256] RPC call timeout handler was delayed by -1.2150347320002766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
W20260709 12:19:16.458410 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1898347320002358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 12:19:16.458688 28404 connection.cc:256] RPC call timeout handler was delayed by -1.1790777230000913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
I20260709 12:19:16.459645 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.279078  1215 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 12:19:16.388434 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:16.390105 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 53374.6us
I20260709 12:19:16.475682 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:16.477022 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 100908us
I20260709 12:19:16.419027 28359 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:19:16.423125 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.477831 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:16.387665  1324 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.441705 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:16.392016   792 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:16.424044  1491 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:16.451941 28439 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:16.376713  1538 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:19:16.474097 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.480506 28287 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:16.386790  1504 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.406703  1493 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:16.487314 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:16.488991 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:16.423662  1485 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:16.534336 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47102.1us
I20260709 12:19:16.492184 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:19:16.536037 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 43950.9us
I20260709 12:19:16.536895 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.539279 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:19:16.507464 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.399250  1181 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:16.454859  1306 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:16.546571 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 48259.2us
I20260709 12:19:16.479950  1324 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.485317  1195 raft_consensus.cc:2764] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:16.623471 28440 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:16.409492  1350 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:16.576637 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.485085  1328 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:16.485899  1466 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:16.613670 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.622555 28441 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:16.498360  1491 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:16.652297 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 318690us
I20260709 12:19:16.653055 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.654557 28441 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:16.657121  1548 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:19:16.497452  1484 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:16.658367  1484 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:16.658875  1484 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.625768  1306 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.440932  1521 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.663822  1521 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:16.664505  1521 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:16.533519  1493 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.667466  1504 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:16.534092  1485 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.670011  1485 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.672552 28361 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:16.497303  1529 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:16.499737   948 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:16.673461  1491 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:16.723992 28441 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:16.732069 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.800596 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:16.531020 28437 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:16.549481  1494 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.550014  1364 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:16.543210  1509 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:16.909556 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:16.474218  1368 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.549777   565 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:16.626933  1342 consensus_meta.cc:287] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.001s
W20260709 12:19:16.649497 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 107415us
I20260709 12:19:16.709599 28292 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:16.674213 28332 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:16.642203  1508 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:16.711653 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.653168 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:16.938756 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:16.941846 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 288763us
I20260709 12:19:16.942649 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:16.943817 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:16.484377  1221 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:16.850876 28434 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:16.896473  1547 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 12:19:16.914368  1370 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:16.895761  1181 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.917241  1363 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:16.968189 28437 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:16.811558 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:16.594388 28283 raft_consensus.cc:2955] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:16.653434  1465 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.158s
W20260709 12:19:16.933012 28363 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.172s	sys 0.270s
I20260709 12:19:16.921466  1493 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:16.954308 28364 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.013s	sys 0.085s
I20260709 12:19:16.956606 28292 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:16.974273 28364 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.013s	sys 0.085s
I20260709 12:19:16.958634 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:16.954097  1212 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:16.953989   935 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:16.969884 28363 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.173s	sys 0.270s
W20260709 12:19:16.940402 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 286641us
I20260709 12:19:16.964687  1478 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:16.934422  1490 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260709 12:19:16.981683  1484 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:16.957652  1509 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:16.995718  1509 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:16.996543  1509 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:16.996860  1551 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:19:16.953850  1215 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:16.997411  1221 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:17.008275  1449 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.008805  1449 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.010926  1449 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:17.008054 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.015475  1221 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.016089  1221 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.018193  1221 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.020051  1509 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:16.991740 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.027101 28438 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:17.028266 28332 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:16.973263 28288 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:16.998587 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2636) took 1238 ms. Trace:
I20260709 12:19:17.030041 28289 rpcz_store.cc:276] 0709 12:19:15.759800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.994178 (+234378us) service_pool.cc:224] Handling call
0709 12:19:16.021623 (+ 27445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.022063 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.022093 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.219412 (+197319us) spinlock_profiling.cc:243] Waited 64.8 ms on lock 0x7b2c00032f88. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:19:16.219801 (+   389us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.388727 (+168926us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:16.411001 (+ 22274us) log.cc:844] Serialized 14 byte log entry
0709 12:19:16.411223 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.411300 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:16.998260 (+586960us) raft_consensus.cc:1685] finished
0709 12:19:16.998387 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:16.998517 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":58157,"raft.queue_time_us":15228,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":19648,"spinlock_wait_cycles":169924352,"thread_start_us":168555,"threads_started":1,"wal-append.queue_time_us":314541}
W20260709 12:19:16.999287  1554 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.272s	user 0.000s	sys 0.000s
I20260709 12:19:16.999704  1484 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:16.958194  1550 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:16.969398 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 157943us
I20260709 12:19:16.958110  1508 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:16.972820  1342 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:17.001792 28284 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:19:17.039391 28284 pending_rounds.cc:85] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 11
I20260709 12:19:17.039690 28284 pending_rounds.cc:107] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:19:16.993198  1212 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:16.964586  1482 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:19:17.005911  1350 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.007344  1324 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.023037  1493 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.031399   935 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:17.050554   935 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:17.051466   935 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.001020  1368 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:17.001155  1558 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:19:17.054952 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:17.055708 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:17.056414 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:17.056897 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2966) took 1170 ms. Trace:
I20260709 12:19:17.057286 28435 rpcz_store.cc:276] 0709 12:19:15.886610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.276116 (+389506us) service_pool.cc:224] Handling call
0709 12:19:16.276570 (+   454us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:16.300289 (+ 23719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.300323 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:16.325075 (+ 24752us) log.cc:844] Serialized 52 byte log entry
0709 12:19:16.635656 (+310581us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:16.635726 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.635851 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.056682 (+420831us) raft_consensus.cc:1685] finished
0709 12:19:17.056701 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.056838 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23213,"raft.queue_time_us":23912,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":18578,"thread_start_us":271793,"threads_started":1,"wal-append.queue_time_us":348386}
I20260709 12:19:17.059358 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.060446 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.061111 28435 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:19:17.061571 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.063184 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.064644 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.040059 28284 raft_consensus.cc:2889] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:17.045608  1482 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:17.046334  1508 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:17.046803  1342 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:17.043363  1472 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:19:17.040939  1449 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:19:17.034333  1550 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.049188  1324 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.046413  1212 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.017477  1306 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.040010 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2540) took 1275 ms. Trace:
I20260709 12:19:17.095095 28433 rpcz_store.cc:276] 0709 12:19:15.764160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.243067 (+478907us) service_pool.cc:224] Handling call
0709 12:19:16.243566 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.444713 (+201147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.026493 (+581780us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b540026c5d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:17.039944 (+ 13451us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460932352}
W20260709 12:19:17.065981  1475 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.663s	user 0.008s	sys 0.002s
I20260709 12:19:17.069221 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:17.073566  1491 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.077263 28435 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:17.079191 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:17.107142 28359 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:19:17.107466 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:19:17.081013 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 120119us
W20260709 12:19:17.092330 28285 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.006s	sys 0.013s
W20260709 12:19:17.108721 28285 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.006s	sys 0.014s
W20260709 12:19:17.100272 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.102077 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2536) took 1386 ms. Trace:
I20260709 12:19:17.076363  1482 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:17.113935 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:17.114782 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.044461  1503 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.077155  1494 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:17.120515 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:17.122768 28330 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:17.125664 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:17.112318 28436 rpcz_store.cc:276] 0709 12:19:15.715568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.227829 (+512261us) service_pool.cc:224] Handling call
0709 12:19:16.228121 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.228491 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.228520 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.481519 (+252999us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b2c000ef378. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:19:16.488127 (+  6608us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.967249 (+479122us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:16.975788 (+  8539us) log.cc:844] Serialized 14 byte log entry
0709 12:19:16.976013 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.976087 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.077545 (+101458us) raft_consensus.cc:1685] finished
0709 12:19:17.101849 (+ 24304us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.102005 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7699,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":11535,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":1481,"spinlock_wait_cycles":48647040,"thread_start_us":334121,"threads_started":1,"wal-append.queue_time_us":524156}
W20260709 12:19:17.128892 28404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.001s	sys 0.012s
W20260709 12:19:17.129323 28404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.001s	sys 0.012s
W20260709 12:19:17.129954 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 590874us
I20260709 12:19:17.130770 28404 leader_election.cc:400] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:19:17.131119 28404 leader_election.cc:403] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:17.094556  1493 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:17.134359 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.135228 28432 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:17.119640  1475 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:17.138558 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:19:17.142120 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.147405 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:17.152567 28283 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:17.154920 28283 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:17.157768 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:17.158409 28285 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 11.
I20260709 12:19:17.161233  1519 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:17.133203  1561 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:17.178185 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:17.169453  1364 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:17.185072  1550 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.166329  1370 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:17.135875  1493 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:17.200443 28404 leader_election.cc:400] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15.
I20260709 12:19:17.206730 28404 leader_election.cc:403] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:19:17.202083 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 68358.8us
I20260709 12:19:17.153221   935 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:17.141150  1368 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
W20260709 12:19:17.199781  1556 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:17.201135  1218 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:17.190626  1561 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.166766  1568 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:17.206254  1529 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:17.212947 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:17.209563  1553 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:17.237860 28432 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:17.189826  1466 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:19:17.210203  1342 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.246134  1503 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.199183  1482 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.255916  1503 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.256350  1482 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.250569  1508 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.258248  1508 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.260427  1508 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:17.233372  1571 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15.
W20260709 12:19:17.266759  1485 consensus_meta.cc:287] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.000s
I20260709 12:19:17.267297  1485 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.245148 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:17.231884  1561 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.251914 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:17.225561  1218 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.211109  1518 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.262614  1503 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.231355  1568 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:17.275126 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:19:17.276679 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 156258us
W20260709 12:19:17.286957 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 143617us
I20260709 12:19:17.289041 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:17.275671  1571 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:17.269654  1485 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.286552  1218 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:17.302781 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.286700   952 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:17.310901   952 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.311381   952 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.311517  1576 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:17.311537   565 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 12:19:17.304168  1442 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:19:17.290611  1564 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:17.316129 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.316963 28437 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:17.285369  1508 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:17.307173  1491 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.307701  1306 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.309908  1566 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:17.310389  1518 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:17.314266   565 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:17.318395  1583 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:17.300122  1449 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:19:17.332963  1583 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.337141  1583 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.246814  1493 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.339017  1493 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:17.290984  1568 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:17.304405  1509 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:17.321345  1508 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.336354  1529 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:17.347594  1575 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:19:17.350404  1508 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.352667  1561 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.354851  1561 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:17.357051  1540 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.861s	user 0.009s	sys 0.002s
I20260709 12:19:17.353195  1583 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.364208  1583 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:17.424897 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2630) took 1705 ms. Trace:
I20260709 12:19:17.425904 28286 rpcz_store.cc:276] 0709 12:19:15.719115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.992905 (+273790us) service_pool.cc:224] Handling call
0709 12:19:15.993270 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.009993 (+ 16723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.010024 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.019399 (+  9375us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.195292 (+175893us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:16.222534 (+ 27242us) log.cc:844] Serialized 14 byte log entry
0709 12:19:16.329001 (+106467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.329080 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.356412 (+1027332us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:17.423825 (+ 67413us) raft_consensus.cc:1685] finished
0709 12:19:17.424105 (+   280us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.424553 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149440,"prepare.queue_time_us":27449,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":142914,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":143682816,"thread_start_us":173823,"threads_started":1,"wal-append.queue_time_us":381534}
W20260709 12:19:17.430748 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 131731us
W20260709 12:19:17.431277 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 125269us
I20260709 12:19:17.433550 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:17.435724 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:17.434307 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 130521us
I20260709 12:19:17.319802 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.475574  1466 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:17.478788 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.479919 28437 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:17.480854 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.481494 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 2627) took 1492 ms. Trace:
I20260709 12:19:17.480854  1466 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:17.483254  1466 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:17.484476 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:17.482069 28435 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.487589  1172 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:17.488185  1172 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:17.452438 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:17.454448 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 143562us
I20260709 12:19:17.485152  1466 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.332186   565 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:17.470351 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:17.458930 28438 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:17.474370 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:17.482162 28361 rpcz_store.cc:276] 0709 12:19:15.981292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.069346 (+ 88054us) service_pool.cc:224] Handling call
0709 12:19:17.473094 (+1403748us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54003ba1d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:17.473404 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385084160}
I20260709 12:19:17.483248 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2973) took 1040 ms. Trace:
I20260709 12:19:17.446597 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:17.514459 28285 raft_consensus.cc:2376] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21. Re-sending same reply.
I20260709 12:19:17.518826  1492 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:17.519553  1492 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:17.519972   968 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:17.513865  1472 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:17.368613  1568 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.550616   968 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.509867 28441 rpcz_store.cc:276] 0709 12:19:16.442101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.746810 (+304709us) service_pool.cc:224] Handling call
0709 12:19:16.747126 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:16.804248 (+ 57122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.804286 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:16.804332 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:16.804363 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.482572 (+678209us) spinlock_profiling.cc:243] Waited 56.9 ms on lock 0x7b5400184f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:17.482647 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.482961 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130840448}
I20260709 12:19:17.526187 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:17.553134 28291 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.553699 28286 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:19:17.555249  1544 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.908s	user 0.004s	sys 0.003s
I20260709 12:19:17.528693 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 2957) took 2041 ms. Trace:
I20260709 12:19:17.529492   565 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.530435 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:17.532204 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.534564  1561 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:17.541544 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:17.543083 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.528719 28986 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:17.525261 28287 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 2633) took 1798 ms (client timeout 1731 ms). Trace:
I20260709 12:19:17.555212 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:17.562266 28436 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.562722 28292 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.563482 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 924) took 1413 ms. Trace:
I20260709 12:19:17.563951 28290 rpcz_store.cc:276] 0709 12:19:16.149717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.150662 (+   945us) service_pool.cc:224] Handling call
0709 12:19:16.151003 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.151349 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.151379 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.151832 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.390311 (+238479us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:16.390351 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.390420 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.554173 (+1163753us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:17.556920 (+  2747us) raft_consensus.cc:1685] finished
0709 12:19:17.556943 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.563261 (+  6318us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2565,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":51255680,"thread_start_us":187428,"threads_started":1,"wal-append.queue_time_us":288662}
I20260709 12:19:17.565968  1561 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.550701  1568 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.521490   928 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:19:17.573422 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 99413.4us
I20260709 12:19:17.569485  1561 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:17.560704 28287 rpcz_store.cc:269] 0709 12:19:15.726964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.890531 (+163567us) service_pool.cc:224] Handling call
0709 12:19:17.524658 (+1634127us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54003ab454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:17.524995 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337197952}
I20260709 12:19:17.581001 28435 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:17.557339 28440 rpcz_store.cc:276] 0709 12:19:15.487528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.487679 (+   151us) service_pool.cc:224] Handling call
0709 12:19:15.495897 (+  8218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:16.633374 (+1137477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.528110 (+894736us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540026c0d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:17.528469 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2592644352}
I20260709 12:19:17.557421  1472 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:17.590024 28291 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:17.572132   968 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.592569 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:17.582321   928 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.588079  1172 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.521421  1492 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.595148 28292 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:17.545679  1218 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.599988 28286 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:17.609368  1464 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:17.610100  1582 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.626914 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2544) took 1656 ms. Trace:
I20260709 12:19:17.627568 28439 rpcz_store.cc:276] 0709 12:19:15.945467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.989711 (+ 44244us) service_pool.cc:224] Handling call
0709 12:19:17.587054 (+1597343us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400347954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:17.601898 (+ 14844us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":888266624}
I20260709 12:19:17.614836  1472 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.631386  1472 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:17.616901  1561 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:17.636924 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47733.6us
I20260709 12:19:17.639317   928 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.640527  1494 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:17.612864  1582 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.640736  1276 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:17.654420 28283 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:17.669425 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:17.672722  1582 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:17.675715  1561 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.698902 28291 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:17.682952  1501 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:17.705206  1501 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.705953  1561 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.708123  1363 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:17.708571  1522 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:17.710585  1522 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:17.711071  1522 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:17.669967  1212 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.662348  1276 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:17.713171   968 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:17.714813  1276 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.716353  1276 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.715430  1582 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:19:17.718037 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.717847  1582 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.720305  1582 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.721112  1442 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.721987  1442 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.707912  1501 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.724174 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 930) took 1085 ms. Trace:
I20260709 12:19:17.724741  1442 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.725271 28438 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:17.726728  1464 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:17.727280  1464 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:17.728076  1212 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.677174  1550 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:17.730754  1212 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.730558  1550 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.732765   968 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:17.733356   968 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:17.737041 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:17.737542   968 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.757191 28364 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:17.738478  1212 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:17.758443  1212 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.743515  1181 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:17.744827   952 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.746379  1172 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.765005   952 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.748929 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:17.749013 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:17.767215 28362 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:17.750746 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:17.757725   968 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.739053 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:17.770185 28359 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
W20260709 12:19:17.765578  1563 consensus_queue.cc:1175] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 15 status { last_received { term: 15 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 15 index: 12 } }
I20260709 12:19:17.744722  1493 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:17.771495  1493 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.777325 28289 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:17.779428 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.724633 28284 rpcz_store.cc:276] 0709 12:19:16.638975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.673851 (+ 34876us) service_pool.cc:224] Handling call
0709 12:19:16.674212 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:17.075971 (+401759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.723640 (+647669us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54003fa054. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:17.723977 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752162048}
I20260709 12:19:17.748543 28285 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:17.785081 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:17.787009  1492 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.788899 28363 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:17.802193 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:17.804431 28437 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:17.802819 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:17.824433 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.825968 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:17.733146  1550 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.829095   565 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:17.830330 28290 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:17.824640 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.831359 28438 raft_consensus.cc:2368] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13.
I20260709 12:19:17.814596  1561 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:19:17.831766 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 137474us
W20260709 12:19:17.832062 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 887227us
I20260709 12:19:17.835350  1588 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:17.712955  1522 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.835603  1496 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:17.837594  1522 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:17.838038  1496 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.838480  1496 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:17.842288 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.837704  1563 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:17.843250 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.847293 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:17.846357 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:17.842092  1561 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:17.826280 28435 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
W20260709 12:19:17.857443 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 57053.3us
I20260709 12:19:17.853530 28436 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
W20260709 12:19:17.855278 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:17.846426 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.846798 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.833554 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:17.861477 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:17.837900  1554 log.cc:927] Time spent T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.071s	user 0.004s	sys 0.000s
I20260709 12:19:17.825100  1493 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.855386  1563 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:17.871114 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:17.831947 28186 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 10 to 14, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 14 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:17.871658 28290 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:17.874269 28362 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:19:17.873194 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:17.871119 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 139321us
I20260709 12:19:17.870834 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:17.882688 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:17.878273 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:17.883591 28363 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:17.882045 28359 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:19:17.880707 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:17.889401 28986 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 18: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:17.889515 28292 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:17.891129 28432 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:17.886125 28366 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:17.894286 28332 leader_election.cc:400] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13.
I20260709 12:19:17.894639 28332 leader_election.cc:403] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:19:17.881938  1442 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:19:17.894835 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:17.895947  1218 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:17.946218 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 58773.7us
I20260709 12:19:17.948107  1477 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.948774  1493 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:17.950068 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:18.010571 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:18.011842 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:18.013052 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:17.952162  1561 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.013911  1477 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:17.955303 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 121921us
I20260709 12:19:18.015341 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:18.016371  1477 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.017109   935 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.955392 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:17.956051  1592 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:19:17.956557 28362 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 12:19:17.959528 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 77876us
I20260709 12:19:17.940538  1571 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:17.893872  1466 raft_consensus.cc:2764] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:19:17.893419 28440 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:18.026110 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:18.026791  1363 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:17.961436 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:17.952194 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:17.962769 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:17.964035  1212 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:17.964743  1508 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:17.968291 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.028539 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:18.019461 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:17.974601 28363 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:17.975327 28283 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 13: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:17.980401 28186 catalog_manager.cc:5697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:17.983712 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:17.985687 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:18.030400 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:17.987550 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:17.974409  1563 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:17.990897 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:17.994331 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:17.894052  1509 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:18.001169  1501 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.001631 28432 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:18.003059 28364 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:18.008795  1529 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:17.953193 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:18.021524  1571 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.029524 28288 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:17.986748 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.041441  1563 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:18.043267 28285 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:18.043349 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.043861 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:18.048519 28291 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:18.046869 28435 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:19:18.027562  1363 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.068063  1363 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.072885 28186 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 13 to 16, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 16 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:18.075737 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.076610 28292 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:18.079167  1496 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.081130  1212 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:18.050254 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:19:18.055787 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49631.1us
I20260709 12:19:18.060907  1583 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:18.061435 28358 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:18.062491  1364 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:18.063357 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.055035  1501 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:18.057251 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:18.046442  1466 raft_consensus.cc:2764] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:19:18.033586  1561 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:19:18.066254 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 46378.9us
I20260709 12:19:18.088771 28289 raft_consensus.cc:2368] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:18.074827 28186 catalog_manager.cc:5697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:18.069484 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:18.017867   935 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.082401  1491 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:18.098768  1583 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.101011  1464 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.081846  1508 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:18.081679  1496 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.103152  1529 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.081823  1212 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:19:18.106504 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49373.9us
I20260709 12:19:18.107137  1494 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:18.107651 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 61565.1us
I20260709 12:19:18.066181 28284 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:18.108708  1577 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:18.113035  1501 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:18.141908 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:18.142475  1492 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.143132  1492 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:18.054622  1571 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:18.134865 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 65474.9us
I20260709 12:19:18.087235  1509 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:18.147476 28367 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.147030 28292 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:19:18.147768 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:18.151319 28288 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 9. Current term is 10. Ops: []
W20260709 12:19:18.153520 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.157464 28433 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:18.142758  1477 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:18.135638  1493 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13.
I20260709 12:19:18.160339 28437 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:18.139945  1491 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:18.167882  1491 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.170151 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.145133  1577 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.145017  1494 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:18.177876 28358 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:19:18.180974 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.183597 28433 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:18.148893  1561 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:18.181685  1580 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:18.185600  1363 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:18.185631  1466 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:18.186555  1522 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:18.187054  1522 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:18.187093  1363 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:18.187597  1363 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:18.188194  1522 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.188577  1363 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.189528  1522 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.189898  1363 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.174144   928 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:18.158938   968 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:18.193017   968 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:18.194099   968 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.189555 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:18.197201  1212 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.199419  1368 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:18.199990 28365 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:18.162146  1477 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.200798  1477 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.201918 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.202549  1522 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:18.203262  1522 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:18.204319  1522 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.202724 28358 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:18.206471  1368 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:18.206920  1368 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:18.207953  1368 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.209410  1368 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.183792 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.210402 28362 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:18.168641   935 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.184715 28186 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 9 to 11, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 11 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:18.179553  1494 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.177623  1577 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:18.219169 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44598.9us
I20260709 12:19:18.223140   935 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:18.224182   935 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:18.224538   935 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:18.226331   935 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.227737   935 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:18.232116  1561 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.219797  1492 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.234294  1529 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.238183  1582 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:18.280019  1582 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:18.215553  1464 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.279582 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:18.283596 28186 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
W20260709 12:19:18.291211 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.291989 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.295944 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 278162us
I20260709 12:19:18.297434 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:18.298454   928 raft_consensus.cc:2764] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:19:18.300822 28364 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:18.302939  1352 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:18.303107 28257 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:18.304137  1306 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:18.304688  1306 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:18.305727  1306 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.306668  1352 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:18.306388  1477 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:18.307852  1352 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.310595 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:18.326498 28441 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.311906  1364 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:18.312446 28439 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:18.285480 28186 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:18.313616 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:18.314105 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 195620us
I20260709 12:19:18.315330  1583 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.329149 28437 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:18.322906 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:18.334306 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.330786 28361 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
W20260709 12:19:18.298470 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 212979us
I20260709 12:19:18.343851 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:18.344668 28438 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.347760 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:18.354863 28186 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 7 to 9, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 9 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:18.342269  1501 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:18.361642  1501 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:18.362095  1501 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:18.366281 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.356814 28186 catalog_manager.cc:5697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 8 to 11, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 11 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:18.371822 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.374622  1509 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.331489  1477 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.379146  1577 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.388343 28367 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:18.399073 28441 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:18.406708 28291 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:18.407156 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:18.406530 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:18.415438 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:18.416860  1291 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:18.417483  1291 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:18.418638  1291 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:18.425597  1577 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.427063  1508 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:18.401751 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:18.333272  1582 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:18.429035 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.384394 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:19:18.470984 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 86825us
I20260709 12:19:18.472510 28186 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:18.471848 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:18.478832 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:18.479693 28186 catalog_manager.cc:5697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 4 to 12, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 12 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:18.463384  1508 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:18.485322 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:18.486274 28285 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:18.469877  1580 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:18.486414 28186 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 9 to 12, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 12 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:18.486449 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 157112us
I20260709 12:19:18.487236 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.469559  1364 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:18.492658 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.492257 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:18.497383 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.497505 28186 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 7 to 11, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 11 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:18.503413 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.475569   928 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:19:18.515030 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.514063 28438 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:18.484403  1508 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:18.517316 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:18.488483  1477 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:18.505014 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:18.522310  1364 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.522861 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:18.522732 28186 catalog_manager.cc:5697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 9 to 13, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 13 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:18.523373  1494 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.510066  1580 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:18.525179 28186 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 6 to 2, term changed from 14 to 13, leader changed from <none> to f526369c0d0743b192be6e205540a059 (127.26.123.4), f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from VOTER to NON_VOTER, now has a pending config: VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1), VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2), VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3), VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 13 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:18.500105  1306 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:18.526437  1571 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.552438  1571 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.552606 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.526820   792 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:18.553719   792 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:18.554845   792 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:18.527981 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.557804 28358 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:18.528255 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.559237 28365 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.534550  1590 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:19:18.560778 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.530870 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.562203 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.540158  1368 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:18.562834 28358 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.535401  1306 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.563776 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.532092 28186 catalog_manager.cc:5697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 7 to 12, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 12 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:18.547044 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:18.565412  1306 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.566108  1493 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:18.527422  1550 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:18.567093  1550 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:18.561841  1590 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:18.525710  1494 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.568204  1550 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.536367 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.560351 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:18.541131  1580 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.525020  1364 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.553963  1508 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.573995  1494 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:18.574717  1494 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:18.575237  1494 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:18.560390 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.576582 28366 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:18.568222  1590 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:18.578533  1590 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.545060  1477 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.579025  1501 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.573639 28404 leader_election.cc:400] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:18.580026 28404 leader_election.cc:403] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:18.580078 28186 catalog_manager.cc:5697] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 6 to 10, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 10 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:18.581919  1364 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:18.579704  1477 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:18.587672 28361 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:18.590380 28186 catalog_manager.cc:5697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:18.594310 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:18.595158 28438 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.598919  1571 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:18.600749  1477 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:18.601583  1571 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:18.602356  1590 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.602741  1501 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.602480 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:18.603443 28367 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 22.
I20260709 12:19:18.604892 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:18.608110 28365 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:18.608670  1577 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.609941 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.621321 28361 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:18.610177 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.623153 28366 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:18.625716 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:18.626623 28984 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:18.626981  1571 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:18.630053 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.631347 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.632555 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.628230 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.634286 28360 raft_consensus.cc:2393] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:18.637275 28438 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:18.640514 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.647317  1477 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.654332 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.658941 28367 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:18.661541 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:18.662387 28438 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:18.662274 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:18.663971 28984 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 19: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:18.664878 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:18.665552 28986 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:19:18.661007 28332 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:18.669101 28358 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:18.673677 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:18.671826 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:18.610829  1508 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.629570  1571 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:18.678462 28364 raft_consensus.cc:2410] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:18.678133 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.688617 28360 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:18.663372  1364 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:18.624533  1577 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:18.697857 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.694552 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 68924.3us
I20260709 12:19:18.697463 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:18.671931  1368 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.697592 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:18.706810 28366 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:18.673170  1501 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:18.704947 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:18.706811 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.707580 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:18.708551 28361 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:18.709342  1590 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:18.689679  1571 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.709957  1590 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.712174  1590 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.712764 28285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:18.713593 28285 raft_consensus.cc:606] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:18.713954 28285 raft_consensus.cc:614] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:18.703018  1494 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.721220  1364 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:18.731743  1583 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:18.732442  1583 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:18.732959  1577 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.733407  1583 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.724252 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:18.740059 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:18.741066  1466 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:18.741631  1466 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 22 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:18.742962  1466 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.726241  1501 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:18.748790  1501 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.749428  1494 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.749498 28989 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:18.763870 28986 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:18.779080 28438 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
W20260709 12:19:18.797215 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 73917.1us
I20260709 12:19:18.802207 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:18.804369  1522 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:18.793435 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:18.819186 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:18.822337 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.819689 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.827569 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.828315 28364 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:18.828543 28366 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:18.829867  1473 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:18.830330  1473 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.830734  1473 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:18.831353  1550 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:18.829594 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:18.836704 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.722064  1368 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.839772 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:19:18.844290 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 142422us
I20260709 12:19:18.846455 28435 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:18.852926 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:18.853499 28986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:19:18.854174 28986 raft_consensus.cc:606] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:18.854519 28986 raft_consensus.cc:614] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:18.855885 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.831965  1550 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.833495  1442 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:18.835839   935 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.866377 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.868758 28435 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:18.843856  1590 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:18.876945 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:18.877691 28365 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:18.838725  1494 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:19:18.876746  1212 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:18.890319 28364 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:18.873937  1590 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.882045  1364 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.915436 28363 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:18.924762  1580 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.925503  1580 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.926338 28435 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:18.927732  1580 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.928296  1580 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:18.928903  1580 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:18.931542 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:18.931965 28285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:18.932706 28285 raft_consensus.cc:606] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:18.933082 28285 raft_consensus.cc:614] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:18.941049 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.942466 28285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:18.943151 28285 raft_consensus.cc:606] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:18.943547 28285 raft_consensus.cc:614] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:18.877713  1368 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:18.967000 28365 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:19:18.871474   935 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:18.883926  1212 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:18.895694  1508 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:18.931107  1590 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:18.968369  1582 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:18.977854  1212 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:19.012529 28285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:19.013628 28285 raft_consensus.cc:606] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:19.014305 28285 raft_consensus.cc:614] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:19.050884 28434 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:19.002341  1594 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.001260  1571 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.067173 28285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:19.067931 28285 raft_consensus.cc:606] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:19.034008  1508 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:19.067600  1612 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.075s	user 0.000s	sys 0.002s
I20260709 12:19:19.069442  1368 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:19.084000  1615 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.089s	user 0.002s	sys 0.000s
I20260709 12:19:19.067787  1522 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:19.086851 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.086620 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:19.093708 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.093806 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:19.103856 28283 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:19.086499 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:19.105362 28291 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:19.073433  1583 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:19.094297 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:19.114184 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.116734 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:19:19.084535  1508 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.117584 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:19.118619  1607 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:19:19.118919  1508 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:19.119453  1608 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.189s	user 0.002s	sys 0.002s
I20260709 12:19:19.120728 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:19.121284 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 268437us
I20260709 12:19:19.124082 28292 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:19.126832 28290 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.115980 28366 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:19.069403  1594 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.129913 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:19.136646 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.125375 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.132473  1501 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.136579 28285 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:19.137343 28291 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.138459 28284 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.139112 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:19.083333  1622 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:19:19.160645 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.106912  1368 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.111575  1522 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.175509 28283 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:19.122156  1583 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.159422  1529 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:19.138571  1630 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:19:19.184237 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.187397 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:19.159493  1501 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.188191  1594 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:19.187867  1529 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.191232 28283 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.201368 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.202327 28284 raft_consensus.cc:2393] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.202913 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:19.203595 28435 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:19.204854 28956 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:19.207666 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:19.211548 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.213322 28290 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:19.216068  1627 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:19.226400  1627 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.227002  1627 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.227383  1618 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:19.228089  1623 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.228713  1618 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:19.229212  1618 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:19.230118  1627 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:19.226954 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:19.216518  1571 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:19.240530 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.242250  1496 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:19.250051 28287 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 22: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.250923  1501 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:19.251719  1583 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:19.254254  1550 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:19.255533 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:19.255879 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:19.256610 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.257289 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.217893  1580 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:19.228708  1623 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.264662 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:19.280321 28435 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:19:19.282021 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:19.259130 28287 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.266218 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:19.284598  1496 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:19.285327 28255 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:19.286322  1496 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.287010 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:19.288690  1496 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.290489 28439 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:19.291868 28432 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.289011 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:19.313087 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:19.315200 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:19.376683 28436 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:19.321094 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:19.321542 28291 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:19.323853 28283 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:19.259675  1636 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:19.325237 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:19.326635  1522 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:19.328222  1594 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:19.264365  1212 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.331797  1618 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.337071 28432 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:19.258278  1473 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.262009  1623 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:19.337935  1550 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:19.338264 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.396625 28288 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 10.
I20260709 12:19:19.260623  1368 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:19.339010  1580 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:19.341230  1622 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:19.387198  1636 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.397948  1622 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.341487  1629 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:19.398919  1212 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:19.399354  1629 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:19.399758  1629 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:19.400233  1622 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:19.262439  1529 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:19.364475  1571 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.400555  1629 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.365243  1583 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.257695  1590 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.386848  1594 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.402179 28283 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:19.386710 28290 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:19.403198  1618 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.403962  1590 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:19.401319  1571 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:19.376739 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 61663.8us
I20260709 12:19:19.393802 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:19.391253  1627 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:19.406709  1571 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:19.389065  1496 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:19.401681  1583 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.407923  1496 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:19.408895  1496 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.409436  1583 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:19.398350  1636 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.409674 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:19.404736 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.410681 28283 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:19.413712 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:19.415308 28437 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:19.413859 28435 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:19.416698 28435 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:19.377513 28434 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:19.417841 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:19.419032 28436 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:19.402496  1594 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.421579 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:19.419080 28286 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2456) took 1557 ms (client timeout 2031 ms). Trace:
W20260709 12:19:19.325750 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 43813.9us
W20260709 12:19:19.425683 28286 rpcz_store.cc:269] 0709 12:19:17.861559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.944976 (+ 83417us) service_pool.cc:224] Handling call
0709 12:19:19.418878 (+1473902us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b54003d9fd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:19.419013 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1932402432}
I20260709 12:19:19.394553  1473 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:19.429701 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:19.435921 28364 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:19.437107 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:19.417793 28437 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:19.438509 28288 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:19:19.448896 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:19.449910 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 160959us
I20260709 12:19:19.451110 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.451318 28436 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:19.451845 28289 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:19.454056 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.459610 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:19.460310 28359 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:19:19.436689  1616 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.328s	user 0.002s	sys 0.000s
I20260709 12:19:19.463462 28285 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.463853 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:19.466148 28989 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.459810 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:19.467245 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:19.454668  1442 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.005s	sys 0.009s
I20260709 12:19:19.459437  1508 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:19:19.445359  1629 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:19.473925 28284 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.473692 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:19.474360 28436 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:19.467108 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:19.478634 28292 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:19.487918 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.489552 28359 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:19.481869 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:19.479579 28440 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:19.436937  1492 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:19:19.503726 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:19.479004  1442 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.005s	sys 0.009s
I20260709 12:19:19.510494 28288 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:19.509855 28434 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:19:19.510599 28436 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:19:19.512826 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:19.511293 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:19.518133 28366 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:19.517786 28436 pending_rounds.cc:85] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 7
W20260709 12:19:19.519753 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54150.8us
I20260709 12:19:19.476089  1583 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:19.525826 28365 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:19:19.517449 28434 pending_rounds.cc:85] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 11
I20260709 12:19:19.528223 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:19.536825 28359 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:19.529933 28332 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:19.472044  1352 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:19:19.518860 28436 pending_rounds.cc:107] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:19:19.548362 28436 raft_consensus.cc:2889] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:19.509922  1364 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:19.531280 28434 pending_rounds.cc:107] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:19:19.552631 28434 raft_consensus.cc:2889] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:19.471935  1509 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:19:19.557281 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:19.558017 28436 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:19.514161  1442 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:19.555409 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:19.520629  1466 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:19.537916 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:19.526384  1571 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:19.539281  1625 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.545s	user 0.003s	sys 0.005s
I20260709 12:19:19.565518 28359 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:19:19.566293 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:19.551203  1582 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:19.615023 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 153591us
I20260709 12:19:19.564872  1627 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:19.562573  1368 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.567619  1571 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:19.559376  1580 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:19:19.540244  1583 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:19.655107 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 93481.5us
I20260709 12:19:19.655939 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:19.641042 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:19.566872  1364 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:19.671456 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:19.619886  1582 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.678186  1582 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:19.562394  1635 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 12:19:19.606235  1212 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
W20260709 12:19:19.700138 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 162317us
I20260709 12:19:19.698772 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:19.708130 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.654287  1571 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.672660  1508 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:19.653326  1509 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:19.714995 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:19.652478  1627 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:19.732964 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:19.750404  1650 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.125s
I20260709 12:19:19.751230  1627 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.760228  1476 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.221s
W20260709 12:19:19.761479  1648 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
W20260709 12:19:19.778141 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.735715  1509 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.622505  1643 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 12:19:19.700563  1529 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:19.723960  1508 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:19.794137  1655 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 12:19:19.797413  1647 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:19:19.799702  1641 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:19:19.800331  1649 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:19.801872  1364 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.787948  1368 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:19.748852  1631 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:19.803925  1582 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.808699  1654 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:19.810544  1583 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:19.837961  1509 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:19.740294  1442 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.904927  1582 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:19.844215 28366 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
W20260709 12:19:19.863029 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 299818us
I20260709 12:19:19.730647 28436 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
W20260709 12:19:19.902192 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 203544us
I20260709 12:19:19.916445 28359 raft_consensus.cc:2368] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18.
I20260709 12:19:19.898128  1654 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:19.918403 28288 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.919587 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.920308 28290 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:19.923133 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:19.902868 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2586) took 1090 ms. Trace:
I20260709 12:19:19.926278 28363 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:19.927187 28365 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:19.928082  1649 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:19.929121  1649 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.928845 28438 rpcz_store.cc:276] 0709 12:19:18.812853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.817624 (+  4771us) service_pool.cc:224] Handling call
0709 12:19:18.818122 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:18.870011 (+ 51889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:18.870049 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:18.881070 (+ 11021us) log.cc:844] Serialized 26 byte log entry
0709 12:19:19.157474 (+276404us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:19.157538 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.801354 (+643816us) spinlock_profiling.cc:243] Waited 43.3 ms on lock 0x7b54002e27d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:19.801408 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:19.801463 (+    55us) raft_consensus.cc:1685] finished
0709 12:19:19.801476 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:19.902780 (+101304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3897,"raft.queue_time_us":743,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":3358,"spinlock_wait_cycles":99678592,"thread_start_us":168568,"threads_started":1,"wal-append.queue_time_us":273120}
I20260709 12:19:19.885236  1368 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:19.947916 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:19.948557 28363 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 22: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:19.950296 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:19.950914 28432 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:19.914093  1631 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:19.873000  1594 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.956153 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:19.945978  1618 consensus_meta.cc:287] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.000s
I20260709 12:19:19.929353  1654 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.963184  1654 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.963701 28367 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:19.964754 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:19.970482 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 54228.3us
I20260709 12:19:19.971851 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:19.974639 28291 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:19.978930 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:19.989611  1368 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.990991  1368 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.991505 28363 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:19.992478 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:19.993314  1623 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:19.993868  1623 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:19.994237  1464 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:19.994693  1464 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:19.995069  1464 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.995414  1623 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.996409  1631 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:19.999377 28286 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:19.898525  1636 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.993924  1508 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.003767 28986 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:20.003619 28983 raft_consensus.cc:2368] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:19.954520  1582 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:19.992620 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:19.920224  1652 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.360s
I20260709 12:19:20.001569 28290 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:20.000175 28291 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:20.018532 28366 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:19:20.032351 28432 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
W20260709 12:19:20.037886  1642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.000s	sys 0.029s
W20260709 12:19:20.038470  1642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.000s	sys 0.029s
I20260709 12:19:20.047747 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:20.048457 28285 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:20.057372 28285 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:20.057938 28438 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:20.024199 28255 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.062446 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:20.062806  1496 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:20.063371  1651 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:20.064410 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.065315  1496 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:20.066537  1651 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:20.067607  1651 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:19.959422  1655 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 12:19:20.086935 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 70006.2us
I20260709 12:19:20.083839 28332 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:20.003305  1636 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:20.094719 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:20.095392 28439 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:20.022977  1583 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:20.021831  1618 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:20.083124  1550 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:20.086639  1529 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:20.108558 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.100337  1508 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:20.083477  1582 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.101941  1583 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:20.135378 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46595.6us
W20260709 12:19:20.134084 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44322us
W20260709 12:19:20.138813  1619 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.063s	user 0.007s	sys 0.013s
I20260709 12:19:20.148918 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:20.169060 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.151844  1629 consensus_meta.cc:287] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.664s	user 0.007s	sys 0.004s
I20260709 12:19:20.123849  1550 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.172204  1466 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:20.176529  1632 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:20.176059  1640 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:20.175390  1661 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 12:19:20.177234  1364 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:20.117717  1618 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.180204 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:20.153975  1583 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:20.126410  1529 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.175310  1674 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:20.178874  1352 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:20.190307 28285 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:20.193202 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.195778 28288 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:20.208137 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:20.197149 28286 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:19:20.210602  1476 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.215140  1476 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.212572  1522 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:20.216585  1522 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.216915  1522 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:20.218420  1665 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:19:20.219555   935 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:20.226701  1571 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.227857 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:20.228637 28989 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:20.229137  1571 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:20.229357  1529 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:20.232066  1618 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:20.225004  1509 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.234968 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:20.236244 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:20.234148 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:20.235582 28289 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:20.239781 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:20.240608 28286 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 22: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:20.246533 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:20.214489 28285 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:20.252153 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:20.253441 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:20.258428 28288 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:20.252600 28290 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:20.258482 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:20.259846 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:20.260322 28285 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:20.226595  1646 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:20.191022  1494 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:19:20.235553  1466 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:20.274341 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:19:20.274897 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.275146 28989 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:20.271823 28284 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:20.277794  1689 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.278131  1582 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:20.280866  1689 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:20.282882 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 174436us
I20260709 12:19:20.283799 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.284377  1466 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:20.284850  1466 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.285761  1655 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:20.249924  1594 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:20.287551 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:20.295169  1655 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:20.297570 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.269330  1529 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:20.302053  1529 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.305188  1646 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:20.308005  1672 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:19:20.308427 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:20.309360  1655 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:20.310562 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.311115 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.304629  1529 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:20.313999  1594 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.314666  1594 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.333488 28285 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:19:20.269508  1630 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:20.351148 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:20.185727  1632 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.353071 28441 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:20.328341  1655 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:20.359700 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.364302 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.189318  1640 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.367931 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:20.285882  1666 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.149s
I20260709 12:19:20.372679 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:20.357360  1630 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:20.352444  1522 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.373252  1618 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:20.383948 28439 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:20.376487  1629 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.387864  1522 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:20.381103  1464 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.390681 28435 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:19:20.376549  1352 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:20.370533  1691 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.373756  1674 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:20.397122 28991 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:19:20.422295 28432 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:20.419440 28989 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
W20260709 12:19:20.424342 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47051.8us
W20260709 12:19:20.423820 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 274276us
I20260709 12:19:20.436338 28441 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:20.450237 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:20.426227  1646 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:20.456370 28440 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:20.396049  1629 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.462100 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:20.464155 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.468295  1464 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:20.469086  1696 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:19:20.431442  1689 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:20.472913 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:20.471630  1629 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:20.478996 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 270895us
I20260709 12:19:20.485919 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:20.488241 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 67077.6us
I20260709 12:19:20.483853 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:20.495512 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:20.495477 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:20.508018 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:20.447383  1618 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:19:20.494496 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:20.529112 28366 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:20.494890 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:20.478163  1582 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.478124  1352 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.494343  1652 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:19:20.530429  1612 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:20.528285  1522 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:20.442881  1630 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.523900  1655 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:19:20.494696  1688 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:19:20.528556  1654 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:19:20.534418  1494 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:19:20.534278  1368 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:19:20.546932  1661 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:19:20.541412  1466 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.546144  1582 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.623888  1466 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:20.492246  1674 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:20.630975 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 102424us
I20260709 12:19:20.631902 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:20.638878 28366 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:20.603875  1629 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:19:20.620927  1594 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:20.646880 28330 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:20.646090  1713 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:20.617813  1583 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.648120 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.676638 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.678871 28439 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:20.678953 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:20.650065  1694 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 12:19:20.650121  1442 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.681558  1702 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:20.677285  1713 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:19:20.693217 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 162526us
W20260709 12:19:20.681764  1508 consensus_meta.cc:287] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.546s	user 0.014s	sys 0.019s
I20260709 12:19:20.696552 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:20.705794 28289 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:20.707412 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:20.690747  1629 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:20.714805  1698 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:19:20.710667  1494 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:20.722620  1705 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:19:20.712436  1571 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:20.729038  1582 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:20.720307  1686 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.731833  1583 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:20.704777  1715 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:19:20.703696  1442 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.740947  1442 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:20.731833  1571 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:20.745863  1571 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:20.710177  1702 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:19:20.756613 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 77730.8us
I20260709 12:19:20.758765 28330 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.760092 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:20.760414 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:20.779057 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:20.779060 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:20.780956 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 230324us
I20260709 12:19:20.767745  1582 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.790591 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.749189  1702 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:20.791157 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 243718us
I20260709 12:19:20.793946 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.806573 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.807503 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:20.799124  1494 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:20.819238 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.802656  1466 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.833287  1710 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:20.834256  1725 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:20.812446  1713 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:20.814275  1582 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:20.865293 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.853598  1522 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:20.881104 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:20.898777 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:20.899523 28290 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:20.881839 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:20.874552  1715 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:20.951210 28440 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:20.951959 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 70347.7us
I20260709 12:19:20.929445 28441 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:20.945474  1522 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:20.949396 28986 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 620) took 1558 ms (client timeout 1717 ms). Trace:
W20260709 12:19:20.957525 28986 rpcz_store.cc:269] 0709 12:19:19.391320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.393459 (+  2139us) service_pool.cc:224] Handling call
0709 12:19:20.949121 (+1555662us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b540064f354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:20.949276 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1373310848}
I20260709 12:19:20.954262 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:20.959381  1727 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:19:20.955840  1716 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:20.956736  1701 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:19:21.091621 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 137431us
I20260709 12:19:21.089848  1522 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:21.085534 28404 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.4:39985 timed out after 1.704s (SENT) [suppressed 2 similar messages]
I20260709 12:19:21.141422 28290 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:21.143230 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:21.147832 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:21.148046 28330 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:21.100978  1710 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:21.152132 28290 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:21.140463  1713 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:21.147604  1715 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:21.157914 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:21.159448 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:21.160051  1716 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:21.160337  1674 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:21.168673 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:21.148907  1701 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.172052  1674 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:21.176082  1496 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:21.176514 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.176759  1496 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.177332 28986 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:21.172783  1702 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:21.177639 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.173537  1590 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:21.178761  1590 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:21.179128  1590 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:21.179962  1590 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:21.182922 28359 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 2671) took 1509 ms (client timeout 1693 ms). Trace:
I20260709 12:19:21.183355  1702 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:21.183418 28359 rpcz_store.cc:269] 0709 12:19:19.673692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.714670 (+ 40978us) service_pool.cc:224] Handling call
0709 12:19:21.182695 (+1468025us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b5400127854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:21.182846 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396265728}
I20260709 12:19:21.183878 28330 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:21.183974  1702 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.184788  1674 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:21.186461  1702 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.187621 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.188557 28287 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 10: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:21.189198 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 2489) took 1190 ms. Trace:
I20260709 12:19:21.183118 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:21.189991  1688 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:21.190524 28290 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:21.190737  1688 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:21.193285  1688 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.177597  1691 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:21.198532  1729 log.cc:927] Time spent T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:19:21.189538 28287 rpcz_store.cc:276] 0709 12:19:19.998744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.001077 (+  2333us) service_pool.cc:224] Handling call
0709 12:19:21.187461 (+1186384us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b5800000098. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762eee 000000000036eb39 00007f85c37d61e0 00007f85c389f0c4 00007f85c389f131 00007f85c384d0a0 00007f85c384c984 00007f85bcdc76d4 00007f85bcdc7662
0709 12:19:21.189118 (+  1657us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4838912}
I20260709 12:19:21.157464  1724 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:21.208191  1496 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.209710  1550 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.155397  1713 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.210875  1713 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:21.211875 28358 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:21.230191 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:21.232659  1561 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:21.233115  1561 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:21.233566 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:21.234053  1561 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.235330  1476 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.235249  1561 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.242813  1640 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.234836 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:21.249529  1702 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.249629  1727 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:21.250000 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.250487  1727 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.249387 28365 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:21.252230 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.252400 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:21.253453 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.254629 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.254918  1661 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.253232  1727 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.255414  1661 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.256099  1368 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:21.256640  1368 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.255218 28288 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:21.255685  1702 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.259907  1368 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:21.260473 28288 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 12:19:21.261883  1735 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:19:21.264248  1352 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:21.258484 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.271412 28290 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:21.147976  1582 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:19:21.275245 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.276819 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.294452 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.295209 28362 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:21.270810  1661 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:21.272873  1352 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:21.280054  1508 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:21.314488 28255 leader_election.cc:400] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18.
I20260709 12:19:21.317713 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 3053) took 1233 ms. Trace:
I20260709 12:19:21.318217 28432 rpcz_store.cc:276] 0709 12:19:20.084642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.084774 (+   132us) service_pool.cc:224] Handling call
0709 12:19:20.085107 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.423430 (+338323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.317430 (+894000us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54003ca854. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:21.317642 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774898944}
W20260709 12:19:21.319950 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43777.5us
I20260709 12:19:21.313629 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:21.329890 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 244380us
I20260709 12:19:21.321174 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:21.336246  1727 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:21.336902 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.337728 28361 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
W20260709 12:19:21.331707  1732 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:19:21.343950 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:21.323908  1629 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:19:21.315081  1713 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.322731  1508 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.349087 28367 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:19:21.353209 28367 pending_rounds.cc:85] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 8
I20260709 12:19:21.353533 28367 pending_rounds.cc:107] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:19:21.353961 28367 raft_consensus.cc:2889] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:21.349316  1582 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:21.360641 28983 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46934 (request call id 622) took 1890 ms (client timeout 1704 ms). Trace:
I20260709 12:19:21.362799 28359 raft_consensus.cc:2368] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
W20260709 12:19:21.363273 28291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 968) took 1954 ms (client timeout 1848 ms). Trace:
W20260709 12:19:21.365880 28291 rpcz_store.cc:269] 0709 12:19:19.409240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.409392 (+   152us) service_pool.cc:224] Handling call
0709 12:19:21.363039 (+1953647us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b5400059bd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:21.363179 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297688960}
W20260709 12:19:21.365460 28983 rpcz_store.cc:269] 0709 12:19:19.470078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.470200 (+   122us) service_pool.cc:224] Handling call
0709 12:19:21.358447 (+1888247us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54001caf54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:21.360534 (+  2087us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1130623360}
I20260709 12:19:21.332231  1352 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.383555 28291 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:19:21.384152 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:21.398113 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51868.8us
I20260709 12:19:21.400854  1704 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:19:21.406418  1704 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:21.406848  1704 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:21.408123  1704 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.409466  1704 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:21.410238  1352 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:21.414275 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:21.361888  1702 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.365379  1582 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:21.426254 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.427239 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.427487 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:21.375324  1522 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:21.378144  1661 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:21.432540 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:21.447465 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 97785.4us
I20260709 12:19:21.430323  1582 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:21.430866  1702 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.459582 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2607) took 1278 ms. Trace:
I20260709 12:19:21.437198  1674 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:21.463711 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2602) took 1490 ms. Trace:
I20260709 12:19:21.437668  1658 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:21.469568 28435 rpcz_store.cc:276] 0709 12:19:20.180885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.219963 (+ 39078us) service_pool.cc:224] Handling call
0709 12:19:20.220347 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:20.395070 (+174723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.459268 (+1064198us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54003cb254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:21.459494 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391076224}
I20260709 12:19:21.431140  1688 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:19:21.473694 28434 rpcz_store.cc:276] 0709 12:19:19.973250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.045776 (+ 72526us) service_pool.cc:224] Handling call
0709 12:19:20.046139 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:20.098956 (+ 52817us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.098988 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:20.099484 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 12:19:20.458064 (+358580us) spinlock_profiling.cc:243] Waited 7.55 ms on lock 0x7b18002eb9a0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85be42f4bc 00007f85bb77d48b 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1
0709 12:19:20.458224 (+   160us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:20.458254 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.458319 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:21.461772 (+1003453us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b18002eb9a0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb79a3f3 00007f85bb79c7c1 00007f85bb79c779 00007f85bb79c709 00007f85bb79c6d1 00007f85bb79b9cd 0000000000388634 0000000000388479 00007f85bb77d83a
0709 12:19:21.463437 (+  1665us) raft_consensus.cc:1685] finished
0709 12:19:21.463468 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.463633 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52411,"raft.queue_time_us":62633,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":7382,"spinlock_wait_cycles":30837504,"thread_start_us":25934,"threads_started":1,"wal-append.queue_time_us":26348}
I20260709 12:19:21.438086  1661 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.452560  1496 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:19:21.459287  1632 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.454753  1464 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 12:19:21.498813 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 64879.8us
I20260709 12:19:21.487617  1691 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.502565  1691 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:21.505160  1691 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.511981 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.512872 28983 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:19:21.514961 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:21.471359  1702 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.523010  1658 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:21.525331  1658 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:21.526465  1688 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:21.527920 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.528177  1661 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:21.528419  1713 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.529018  1508 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.530365 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.531668  1727 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:21.539103 28288 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:21.543758  1702 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.549096  1466 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:21.548842  1688 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.555325  1466 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.556106  1688 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:21.569299 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2599) took 1596 ms. Trace:
I20260709 12:19:21.569939 28438 rpcz_store.cc:276] 0709 12:19:19.972425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.974936 (+  2511us) service_pool.cc:224] Handling call
0709 12:19:19.975247 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.058452 (+ 83205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.568939 (+1510487us) spinlock_profiling.cc:243] Waited 82.4 ms on lock 0x7b540026b1d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:21.569207 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189577472}
I20260709 12:19:21.558775  1629 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:21.550485  1658 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:21.574136 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 91742.9us
I20260709 12:19:21.574991 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:21.575834  1661 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:21.576224  1661 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.576560  1661 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:21.576876 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:21.573302 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 58472.5us
W20260709 12:19:21.579721 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.585114 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:21.582129  1464 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:21.588876  1508 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:21.589553  1508 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:21.591954  1508 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.582777  1629 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:21.594252 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.594929  1629 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.576639  1658 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.589407 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.598955  1466 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:21.602969  1509 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.589188  1582 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.605628  1701 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.609635 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.634033 28364 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:21.623230  1571 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:21.637910  1571 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.627072  1754 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:21.638974  1754 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:21.640286  1571 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:21.608572 28367 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:21.626698 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.615478 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:21.646252 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.650141 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:21.650846 28286 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:21.651871 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.653592 28361 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:21.655244 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.657204 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.666491 28287 raft_consensus.cc:2410] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:19:21.670622 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.673712 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.674719 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.671598 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.676607 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.677234 28438 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:21.681764 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.689148 28359 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 12:19:21.610278 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.692031 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:21.692783 28433 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:21.682523 28287 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:19:21.694191 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.695443 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.644939  1368 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.609684  1666 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:21.698002 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.698529 28286 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:19:21.610383 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:21.653815  1618 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:21.708555 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.711041 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.711341 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.712673 28291 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:21.712798 28367 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:21.623857  1713 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:19:21.718004 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.714128 28330 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:21.718415 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.669227  1522 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:21.672971  1655 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:21.724074 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 113754us
I20260709 12:19:21.728351 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.730252 28291 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:21.731328 28367 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:21.728367 28286 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:21.737824 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.739939 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:21.709887  1754 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.743990 28438 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:21.728367  1618 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:21.749140 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.750061 28367 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:21.750174 28289 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:21.751302  1618 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.751693 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:21.753269 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.754637 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.753489 28289 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:21.719720  1666 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:21.762250 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.741106  1704 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:21.735555  1758 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:21.712148  1368 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:21.773139 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 48176.3us
I20260709 12:19:21.784693 28987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 819) took 1540 ms. Trace:
I20260709 12:19:21.785183 28987 rpcz_store.cc:276] 0709 12:19:20.227011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.229881 (+  2870us) service_pool.cc:224] Handling call
0709 12:19:21.764795 (+1534914us) spinlock_profiling.cc:243] Waited 7.29 ms on lock 0x7b58001f4d98. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:19:21.766800 (+  2005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:21.767040 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.767065 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:21.767109 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:21.767140 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.767205 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.767320 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16771328}
I20260709 12:19:21.786504  1368 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.741278  1476 consensus_queue.cc:1059] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:19:21.788000  1654 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:21.788028 28367 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:21.794391 28367 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:21.803627 28359 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:21.796679 28286 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:21.797842 28954 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:21.749140  1702 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:21.806274 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:21.735695  1522 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.825042 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.778908  1704 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:21.830390 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:21.834438  1632 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:21.834932  1632 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.835289  1632 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:21.835945  1750 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:21.836633  1750 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:21.817382  1172 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:21.838325  1172 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.838805  1172 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:21.842368 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2611) took 1081 ms. Trace:
I20260709 12:19:21.842828 28440 rpcz_store.cc:276] 0709 12:19:20.760456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.760622 (+   166us) service_pool.cc:224] Handling call
0709 12:19:20.760967 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.955553 (+194586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.842101 (+886548us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54002e27d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:21.842302 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436942592}
I20260709 12:19:21.846494  1674 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:21.849237 28986 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:19:21.808152  1701 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.850262  1701 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:21.850931 28989 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:21.852660  1701 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.854292  1666 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.855283 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:21.831883  1702 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:21.858645  1702 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.833536  1656 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:19:21.859025  1550 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:21.863473 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:21.875489 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:21.876214 28289 raft_consensus.cc:2376] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18. Re-sending same reply.
I20260709 12:19:21.883055  1674 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:21.884477 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:21.863843  1476 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.886219 28440 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:21.888743 28435 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:21.864514  1704 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.892755  1476 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.894320  1688 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:21.895401  1476 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.896486  1688 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:21.896801 28292 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:21.826941  1580 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:19:21.898073  1661 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.901520 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.902410 28440 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:21.905766 28441 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:21.837565  1750 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.827178  1464 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:21.907559 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:21.867774  1550 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:21.896703  1701 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:21.905117  1657 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.934752 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.938367 28440 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:21.862864  1172 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:21.961942 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.925326  1582 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:21.928838  1701 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:21.987727 28287 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:21.988165 28287 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:21.989028 28287 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.936441  1473 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:21.992626 28287 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:21.933992  1750 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:21.924611  1646 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:19:21.935325  1661 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.002351 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.003285 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.005585 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.005796 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:22.006489 28362 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:22.009194 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:21.986900  1701 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:22.029282 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:22.031359 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:22.041368 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:22.042595 28435 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:22.051069 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:22.054883 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:21.941916  1657 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:21.971530  1709 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:19:21.996590  1473 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:22.034317  1631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.010s	user 0.014s	sys 0.017s
I20260709 12:19:22.030045  1661 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:22.074641 28284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2490) took 1313 ms. Trace:
W20260709 12:19:22.116539 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.187992 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 134406us
I20260709 12:19:22.043864   928 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.044304  1766 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:22.188951 28433 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:19:22.193245 28404 leader_election.cc:400] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
W20260709 12:19:22.069952  1631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.064s	user 0.014s	sys 0.018s
I20260709 12:19:22.029449  1636 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:19:22.068173  1368 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.218495 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:22.064198  1765 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.067827  1473 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:22.236045 28288 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:22.236521 28288 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:22.064198  1752 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.226356 28404 leader_election.cc:403] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:19:22.188582 28284 rpcz_store.cc:276] 0709 12:19:20.760823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.761031 (+   208us) service_pool.cc:224] Handling call
0709 12:19:20.764852 (+  3821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.765058 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:20.799796 (+ 34738us) log.cc:844] Serialized 14 byte log entry
0709 12:19:21.854519 (+1054723us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b18002cc440. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be42723a 00007f85be891a2a
0709 12:19:22.074169 (+219650us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:22.074238 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:22.074295 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.074380 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.074551 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17767,"prepare.queue_time_us":32660,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":2096,"raft.queue_time_us":382,"spinlock_wait_cycles":6708096,"thread_start_us":218956,"threads_started":1}
I20260709 12:19:22.238178 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:22.259483 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 359695us
I20260709 12:19:22.262008 28332 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:22.263563 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:22.261521 28288 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.272886 28288 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:22.271979 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 304282us
I20260709 12:19:22.046152  1550 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.281625 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:22.288569 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:22.192240  1661 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:22.288878 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 95934.7us
I20260709 12:19:22.057801  1464 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.217067   928 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:22.259662  1522 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.272447  1629 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:22.281534  1580 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:22.308511 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:22.305081 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:22.231696  1766 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.282852  1760 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 10. Current term is 11.
I20260709 12:19:22.323160 28288 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:19:22.330111 28288 pending_rounds.cc:85] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 9
I20260709 12:19:22.335067 28288 pending_rounds.cc:107] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:19:22.335569 28288 raft_consensus.cc:2889] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:22.250510  1765 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.295015  1577 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:19:22.314462   935 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:22.342667 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:22.343693 28435 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:22.314847  1770 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.350286 28284 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:19:22.353180 28284 pending_rounds.cc:85] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Aborting all ops after (but not including) 9
I20260709 12:19:22.353499 28284 pending_rounds.cc:107] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:19:22.353756 28284 raft_consensus.cc:2889] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:22.322945  1522 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:22.353617  1577 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:22.371186 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:22.378925  1777 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:19:22.354746  1550 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:22.381922  1752 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.382584  1752 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.383069  1666 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:22.383644  1666 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:22.386339  1767 log.cc:927] Time spent T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:19:22.387408  1666 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.391021  1172 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:22.392098 28330 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.005s	sys 0.014s
W20260709 12:19:22.392556 28330 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.005s	sys 0.014s
W20260709 12:19:22.392856 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 512223us
W20260709 12:19:22.399341  1762 log.cc:927] Time spent T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 12:19:22.405426 28287 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:22.359762  1522 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:22.399056  1761 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
I20260709 12:19:22.413218 28435 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:22.378759  1714 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:19:22.355373  1760 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:19:22.410077  1522 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:22.399930   935 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:19:22.445442 28435 raft_consensus.cc:1240] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:19:22.449908   928 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:22.452874  1473 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:22.458012  1473 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.458595   928 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.459277   928 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:22.461539   928 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:22.462456  1781 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 12:19:22.464128  1774 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:22.466276  1752 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:22.470199  1772 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:19:22.364455  1770 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:22.478128 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 169726us
I20260709 12:19:22.475057 28989 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:22.457885 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.479532 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.444422  1776 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
W20260709 12:19:22.484925 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.413664  1784 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.071s
I20260709 12:19:22.443086  1466 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:22.497133 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 176691us
I20260709 12:19:22.514362 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:22.515221 28441 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:22.528220 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:22.502792  1657 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.537266 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:22.491734  1701 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:22.544531 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:22.538167 28441 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:22.546542 28983 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:22.550345  1172 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:19:22.556552  1773 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.298s	user 0.005s	sys 0.000s
I20260709 12:19:22.527124  1765 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.503647  1466 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.560973 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:22.570062 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:22.570976 28432 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:22.529601 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 208472us
W20260709 12:19:22.561545 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.548990  1778 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:19:22.535545  1661 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:22.598362 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:22.599668 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:22.613461 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:22.621544 28432 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
W20260709 12:19:22.623246 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 144145us
I20260709 12:19:22.624747 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:22.615841  1666 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:22.611023  1789 log.cc:927] Time spent T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 12:19:22.660851 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 46279.1us
I20260709 12:19:22.662734 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:22.669376 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44720us
I20260709 12:19:22.672446 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2717) took 1024 ms. Trace:
I20260709 12:19:22.673023 28283 rpcz_store.cc:276] 0709 12:19:21.648079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.648293 (+   214us) service_pool.cc:224] Handling call
0709 12:19:21.664164 (+ 15871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.666406 (+  2242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.666442 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:21.689034 (+ 22592us) log.cc:844] Serialized 26 byte log entry
0709 12:19:21.920893 (+231859us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:22.270866 (+349973us) log.cc:844] Serialized 14 byte log entry
0709 12:19:22.271118 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.271198 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:22.554378 (+283180us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:22.656014 (+101636us) raft_consensus.cc:1685] finished
0709 12:19:22.672163 (+ 16149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.672364 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14301,"prepare.queue_time_us":57674,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":3961,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":43236,"spinlock_wait_cycles":4116224,"thread_start_us":491204,"threads_started":2,"wal-append.queue_time_us":359460}
I20260709 12:19:22.630333   928 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:22.610929  1765 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:19:22.697041  1807 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.653587  1666 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:22.715606 28983 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
W20260709 12:19:22.719606  1632 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.884s	user 0.011s	sys 0.004s
W20260709 12:19:22.718027 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 55353.2us
I20260709 12:19:22.720268  1632 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:22.722834 28404 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:22.676214  1804 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:22.725929 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:22.654698  1629 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:19:22.721963  1666 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:22.734910 28987 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:22.723134  1754 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:22.754700  1754 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:22.757352  1754 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:22.750509  1805 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:22.751581  1765 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:22.722198  1807 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:22.769349  1704 consensus_meta.cc:287] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.002s
I20260709 12:19:22.774753  1701 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:22.775465 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.776295  1466 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:22.779837  1771 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.468s	user 0.000s	sys 0.005s
I20260709 12:19:22.725494  1657 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:22.728106  1804 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:22.784080  1632 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.746742  1770 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:22.785079 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:22.774633  1618 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:22.785847 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51108.7us
W20260709 12:19:22.731608 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 182171us
I20260709 12:19:22.799865  1618 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.800678 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:22.796557  1804 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:22.802465 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:22.803857 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:22.810348 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:22.799224 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:22.802625  1618 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:22.804971  1476 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:22.805153   928 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.821296 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 3063) took 1181 ms. Trace:
I20260709 12:19:22.835049  1807 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.836784 28436 rpcz_store.cc:276] 0709 12:19:21.639918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.642790 (+  2872us) service_pool.cc:224] Handling call
0709 12:19:21.643089 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.644558 (+  1469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.644590 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:21.664224 (+ 19634us) log.cc:844] Serialized 26 byte log entry
0709 12:19:21.975239 (+311015us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:21.975291 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.975381 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:22.779147 (+803766us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:22.820889 (+ 41742us) raft_consensus.cc:1685] finished
0709 12:19:22.820919 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.821206 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1077,"raft.queue_time_us":67355,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":10339,"spinlock_wait_cycles":11240320,"thread_start_us":264086,"threads_started":1,"wal-append.queue_time_us":546575}
W20260709 12:19:22.841447  1750 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.819s	user 0.010s	sys 0.003s
I20260709 12:19:22.808866  1631 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.846979 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:22.847752 28364 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:22.845757 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 46639.7us
I20260709 12:19:22.792413  1704 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.852681 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:22.811822  1770 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.859913  1212 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:22.848518  1632 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:22.848222  1702 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:22.861593  1212 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.862574  1702 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.859143  1522 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.867065  1522 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.868312  1750 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:22.869226 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:22.869952 28290 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:22.869774  1750 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.862020  1814 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:19:22.874047  1704 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:22.875819  1766 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:22.876701 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:22.875726  1750 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:22.877508  1727 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:22.862252 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 2726) took 1204 ms. Trace:
I20260709 12:19:22.878082 28288 rpcz_store.cc:276] 0709 12:19:21.657682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.724027 (+ 66345us) service_pool.cc:224] Handling call
0709 12:19:21.741174 (+ 17147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.336148 (+594974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.861993 (+525845us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b54003dbdd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:22.862186 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137787136}
I20260709 12:19:22.878104  1727 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.881234  1727 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:22.882113 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:22.883642 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.884337 28361 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:22.877327 28289 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:22.887936 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 80418.3us
I20260709 12:19:22.888388 28364 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:22.890094 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:22.888652 28290 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:22.892818 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:22.888832 28956 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:22.897397 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:22.853813  1661 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:22.900399 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.903012 28364 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:22.907279 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:22.912696 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.897933  1631 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.914602 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.895619  1715 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.898860   928 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:22.898499  1172 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:22.919718 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.920244 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.921280 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.923698 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.923071 28362 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:22.924696 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.922379 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:22.926043 28983 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:22.927244 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.899786  1476 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:19:22.905305  1582 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:19:22.928647 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 271433us
W20260709 12:19:22.937980 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 248701us
I20260709 12:19:22.929150 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:22.940291 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:22.941932 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44556.2us
I20260709 12:19:22.942447 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.942977 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.943269 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:22.952425 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:22.953346 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:22.956894 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:22.958137 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:22.953354 28362 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:22.912461  1786 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:19:22.967015  1816 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:22.967590  1816 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:22.968521  1816 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:22.912658  1810 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:19:22.916908  1765 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:19:22.978595 28439 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:22.936470  1631 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:22.981331 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:22.999536 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.000886 28954 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:23.003309 28437 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:23.003019 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:22.927011  1715 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.016249 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:22.982833  1816 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:23.028614 28436 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:22.936916  1582 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.003127  1631 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.041327 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.042105 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:23.033726  1804 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.035084  1582 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.045186 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.002767  1476 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:23.048844  1476 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.055018 28360 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:23.044998 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:23.061386 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:23.011422  1813 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:23.068482 28365 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:19:23.014506   928 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:23.052188  1476 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.047082  1631 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:23.040987  1464 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:23.074713  1582 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:23.088779 28289 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 13.
I20260709 12:19:23.077387  1655 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:23.071650  1466 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:23.109102  1582 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.101836  1655 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.115456  1804 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.094913  1713 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:19:23.071419  1766 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:23.118096 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 107492us
W20260709 12:19:23.124734  1783 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.782s	user 0.006s	sys 0.000s
I20260709 12:19:23.125002  1715 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.132268  1784 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.165637 28359 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2708) took 1358 ms. Trace:
I20260709 12:19:23.162638 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.165884  1784 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.166102 28359 rpcz_store.cc:276] 0709 12:19:21.807520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.831487 (+ 23967us) service_pool.cc:224] Handling call
0709 12:19:21.831965 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.832491 (+   526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.832525 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:21.866806 (+ 34281us) log.cc:844] Serialized 26 byte log entry
0709 12:19:22.227021 (+360215us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:22.227072 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.227163 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:22.736604 (+509441us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b48000615d8. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c37b0aa1 00007f85c37a7b37 00007f85c37a79ca 00007f85c389f26f 00007f85c389ef37 00007f85c38b355c 00007f85c38b3508 00007f85c38b3488 00007f85c38b3443 00007f85c38b26ec 00007f85c27dca96
0709 12:19:23.123852 (+387248us) spinlock_profiling.cc:243] Waited 8.6 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:23.165361 (+ 41509us) raft_consensus.cc:1685] finished
0709 12:19:23.165393 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.165538 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114,"raft.queue_time_us":95,"raft.run_cpu_time_us":3653,"raft.run_wall_time_us":898076,"spinlock_wait_cycles":75997312,"thread_start_us":327063,"threads_started":1,"wal-append.queue_time_us":416466}
I20260709 12:19:23.166534 28432 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:23.168767  1784 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.169718 28332 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:23.170607 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:23.170714  1784 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:23.171386  1784 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.171725  1784 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:23.080423  1666 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.095394  1813 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:19:23.181209 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 114108us
I20260709 12:19:23.120108  1466 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.080199   928 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.189989 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:23.201257 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:23.160130  1718 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:23.214154 28439 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:19:23.216694 28367 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:23.179201  1766 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.190716  1636 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:23.203655  1466 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:23.289798  1815 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:23.290536  1815 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.276268  1766 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:23.292804  1815 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.294343  1826 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:23.294988  1826 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.297272  1826 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:23.297998 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:23.299825  1825 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:19:23.307706 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.212713  1172 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:23.162822  1582 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.314061  1212 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.318255   928 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.206780  1818 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:23.326495 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 257226us
I20260709 12:19:23.336717 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:23.352234 28359 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:19:23.351800 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 73189.9us
I20260709 12:19:23.311017  1636 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:23.361997 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.363346 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.364398 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:23.365645 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.276276  1718 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.309494  1784 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.336165  1590 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:23.339725   928 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.382616  1831 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
W20260709 12:19:23.394941  1823 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:19:23.359922  1476 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:23.357913 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 376763us
I20260709 12:19:23.410989 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.413129  1811 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:23.413851  1811 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.415139  1666 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.416294  1811 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.419855 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.421247 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:23.422693 28983 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:23.423609 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.424046 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.429585 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:23.385968  1464 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:23.385754  1815 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:23.432915 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.433722 28361 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 12:19:23.432420 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.442607 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:23.436784 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:23.516921 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 80207.4us
I20260709 12:19:23.408118  1476 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:23.515142  1837 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:19:23.408774   928 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:23.578912 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 242852us
I20260709 12:19:23.442909  1815 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.533800  1476 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:23.599789 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:23.534343  1810 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:23.538877  1752 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:23.539848  1765 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:23.618429 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 74644us
I20260709 12:19:23.519595  1713 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:19:23.443043  1464 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.388471  1813 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.633541  1655 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.578346  1636 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.617231  1810 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:23.642377  1810 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:23.647038  1844 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:23.647714  1844 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.647967  1655 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:23.648684  1655 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.651181  1655 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.625469  1770 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:23.652815  1655 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:23.653412  1655 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.656057 28358 raft_consensus.cc:2410] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:23.656570  1686 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.657078  1686 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.656464  1655 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.597757  1846 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:23.664129  1846 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.666499  1846 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.667680  1784 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:23.669974  1769 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.690s	user 0.004s	sys 0.001s
I20260709 12:19:23.672185 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 2630) took 1184 ms. Trace:
I20260709 12:19:23.672582 28435 rpcz_store.cc:276] 0709 12:19:22.487793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.488568 (+   775us) service_pool.cc:224] Handling call
0709 12:19:22.488961 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.619296 (+130335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.619328 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:22.619372 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:22.619401 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.671903 (+1052502us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400086454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:23.671958 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.672101 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299231104}
W20260709 12:19:23.673874  1832 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:19:23.674479  1808 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.133s
I20260709 12:19:23.681835  1844 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.696710 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 832) took 1296 ms. Trace:
I20260709 12:19:23.697202 28989 rpcz_store.cc:276] 0709 12:19:22.399949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.400156 (+   207us) service_pool.cc:224] Handling call
0709 12:19:22.411549 (+ 11393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.478982 (+ 67433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.696455 (+1217473us) spinlock_profiling.cc:243] Waited 63.3 ms on lock 0x7b5400762d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:23.696642 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145506176}
I20260709 12:19:23.699380   928 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.700109  1782 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:23.700536  1782 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:23.627111  1752 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:23.708586  1839 log.cc:927] Time spent T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.505s	user 0.000s	sys 0.001s
I20260709 12:19:23.632091  1765 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.713027  1765 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:23.714203  1793 log.cc:927] Time spent T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.314s	user 0.003s	sys 0.013s
I20260709 12:19:23.716512  1464 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.717705  1582 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.718619  1801 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:23.719043  1801 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:23.719157  1718 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:23.723147  1829 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.524s	user 0.000s	sys 0.003s
I20260709 12:19:23.725301  1752 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:23.601648  1826 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.740983 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.743309 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:23.751745 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:23.751828 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:23.760573 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:23.669653  1631 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.003s
I20260709 12:19:23.761857  1815 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.745164  1691 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:23.780838  1476 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.781262  1629 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:23.755090   928 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.755525  1846 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.804611 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:23.805706 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:23.764017  1754 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:23.793107 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:23.814492 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:23.795095 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 468272us
I20260709 12:19:23.771113  1765 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:23.811327  1476 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.819681 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:23.828866 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.829825 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.866523 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:23.867009 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 62470.7us
I20260709 12:19:23.866561 28437 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 22: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:23.883628 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.888375 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:23.701396  1782 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.771188  1464 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:19:23.769950  1631 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.893425 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.896968 28366 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:19:23.780755  1828 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 12:19:23.813416  1808 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:23.814307  1754 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.901464 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.902349 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.903352 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:23.886008  1831 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:23.917856 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:23.925127 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:23.931381 28989 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:23.794725  1691 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.762676  1826 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:23.797514  1816 consensus_meta.cc:287] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.756s	user 0.012s	sys 0.000s
I20260709 12:19:23.886164  1805 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:23.976627  1805 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.977718  1827 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.978327  1827 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.978989  1805 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:23.887387  1848 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:23.815055   928 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.982208   928 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.934942  1808 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.985126  1808 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.816809  1846 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.888383  1834 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:19:23.987277  1846 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.927345  1657 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.988582  1657 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:23.991407  1657 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:23.931116  1754 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:23.923848  1849 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:23.992357  1849 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:23.994565  1849 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.999960 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:23.991829  1691 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:23.893117  1752 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.003743 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.004371 28359 raft_consensus.cc:1813] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:24.009759 28359 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 19: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:23.982337  1476 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:23.893124  1765 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.024677 28365 raft_consensus.cc:1240] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 12:19:23.980427  1827 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:24.009714  1766 consensus_meta.cc:287] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.718s	user 0.007s	sys 0.006s
I20260709 12:19:24.051441 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.052381 28291 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 18: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:24.053465 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.054203 28287 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:24.055742 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.056341 28287 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:24.057616 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.058462 28287 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:24.073582 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.071910 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.074681 28292 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:24.077944  1702 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:24.081028 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.036787  1807 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:24.010234  1752 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.087646 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.084290 28291 raft_consensus.cc:2410] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 12:19:24.082655 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 146146us
I20260709 12:19:24.093611 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.093822 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:24.088287 28292 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:24.072613  1631 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:24.086334  1848 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:24.060534  1766 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.106832 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.079663  1666 consensus_queue.cc:1059] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:19:24.125721  1766 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:24.126008  1765 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:24.127254  1810 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.128435 28440 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:24.129366  1826 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:24.129922  1466 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.798s	user 0.012s	sys 0.000s
I20260709 12:19:24.130344  1466 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.131608  1702 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:24.132160  1702 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.087550  1808 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:24.132427  1466 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:24.141994 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.103439  1854 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:24.158605 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:24.150278 28291 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:24.190907 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.190531 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:24.088132  1807 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.190235 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:24.191768 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:24.195289 28365 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:24.193159 28288 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:24.195324 28986 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:24.197661 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:24.129448  1810 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:24.196970  1825 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:19:24.201288  1702 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:24.203934  1718 raft_consensus.cc:3055] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:24.215082  1718 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:24.215781  1807 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:24.215984  1718 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.217100  1718 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:24.206820 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:24.161643  1854 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.222128  1854 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:24.112674  1800 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 12:19:24.229249  1686 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:24.208566  1752 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:24.209472 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.234520 28436 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:24.211699  1818 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:24.236423 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:24.237002 28367 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:19:24.233349 28365 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:24.244342 28986 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:24.244665 28288 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:24.198980 28987 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:19:24.251302 28983 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:24.251174 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.105473  1848 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.249176 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:24.258605 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.259804 28365 raft_consensus.cc:2410] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:19:24.260535 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.261247 28363 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
W20260709 12:19:24.237865  1855 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:19:24.263443 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:24.221490  1754 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:24.268318 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44256us
I20260709 12:19:24.238960  1818 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:24.270766  1827 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:24.273051 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.274106 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.274981 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.297056 28285 raft_consensus.cc:1813] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:24.297701 28285 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:19:24.269500  1810 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:24.278918 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.282794 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:24.283573 28990 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:24.286244 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:24.148444  1808 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.300693 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:24.313325 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.314168 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:24.270704  1862 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:19:24.319830 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:24.320465 28367 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:24.320660 28285 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:24.323118 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.324816 28435 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:24.331871 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:24.343096 28285 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:24.348690 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.349596 28436 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:19:24.350302 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.351454 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.351361 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.353773 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.355686 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 3082) took 1447 ms. Trace:
I20260709 12:19:24.356181 28441 rpcz_store.cc:276] 0709 12:19:22.907797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.908026 (+   229us) service_pool.cc:224] Handling call
0709 12:19:22.908325 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:22.908671 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.908695 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:22.915583 (+  6888us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.001421 (+1085838us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b18004a6aa0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:19:24.107094 (+105673us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:24.107151 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.107227 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.355330 (+248103us) raft_consensus.cc:1685] finished
0709 12:19:24.355359 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.355607 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21043,"raft.queue_time_us":7448,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":13473,"spinlock_wait_cycles":5101440,"thread_start_us":102980,"threads_started":1,"wal-append.queue_time_us":1350315}
I20260709 12:19:24.358605 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.359306 28441 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:24.296250  1827 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:24.356335 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:24.365514 28284 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:24.325953  1807 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:24.322439  1466 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:24.319901  1810 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:24.370658 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:24.371414 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.375708  1827 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.384545  1808 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:24.365053  1861 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:24.376886 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.423326 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.377028  1827 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:24.438758 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:24.369362  1686 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:24.440032 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 68369.1us
I20260709 12:19:24.443966 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:24.444722 28285 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:24.426187  1765 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:24.369081  1854 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.392370  1466 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:24.434356  1861 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:24.455365 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:24.377260  1801 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.455574 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:24.458683 28289 raft_consensus.cc:2368] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:24.448211 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.459877 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.440809  1631 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.482234 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:24.483160 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.483865 28290 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:24.473076 28367 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:19:24.440673  1522 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:24.487828 28285 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:24.486445 28286 raft_consensus.cc:2393] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 23: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:24.494172 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.499715 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:24.501029 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:24.502354 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.502439 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:24.487783 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.504231 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.471220  1861 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:24.523165  1801 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:24.534162 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.535861 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:24.536514 28290 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:24.530295 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.538482 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.539707 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.540535 28289 raft_consensus.cc:2368] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17.
I20260709 12:19:24.541507 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.553576 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:24.555754 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.557395 28289 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:24.480094  1718 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:24.480645  1765 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:24.448642  1686 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.499444  1831 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:24.565835 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:24.567147 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:24.542557  1851 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:19:24.519277  1666 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:24.506352  1760 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:24.510530  1766 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:24.546360  1853 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:24.567634  1866 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:19:24.610937 28987 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:19:24.624601  1854 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:24.592109  1466 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:24.628991 28358 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 2681) took 3380 ms (client timeout 1615 ms). Trace:
W20260709 12:19:24.629464 28358 rpcz_store.cc:269] 0709 12:19:21.248898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.250283 (+  1385us) service_pool.cc:224] Handling call
0709 12:19:24.628743 (+3378460us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b54003bc4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:24.628871 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5525853568}
I20260709 12:19:24.592332  1666 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:24.630864 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:24.572799  1718 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.495709  1631 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.635104 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 3088) took 1684 ms. Trace:
I20260709 12:19:24.497318  1522 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.592425  1754 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:24.625460  1717 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:24.637595  1717 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:24.635557 28438 rpcz_store.cc:276] 0709 12:19:22.951116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.952357 (+  1241us) service_pool.cc:224] Handling call
0709 12:19:24.634358 (+1682001us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b58001d4698. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762eee 000000000036eb39 00007f85c37d61e0 00007f85c389f0c4 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:19:24.634539 (+   181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.634742 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.634770 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:24.634813 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:24.634846 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.634903 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.635032 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5101824}
I20260709 12:19:24.597520 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.645242 28289 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:24.645504 28330 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:24.584149  1831 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:24.650780 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:24.652535 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:24.654247 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:24.678413 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:24.679191 28285 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:24.680195 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:24.521649  1810 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:24.645493  1865 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:19:24.682067 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:24.659719  1817 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.471s	user 0.006s	sys 0.019s
I20260709 12:19:24.688701 28290 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:24.655285  1803 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:24.630970  1666 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:24.678380  1727 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:24.678278  1854 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:24.678478  1864 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:24.655529  1878 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.002s
I20260709 12:19:24.680850  1850 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:24.656495  1831 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:24.727475 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:24.728750 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.685940  1826 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:24.688606  1631 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:24.740119 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 237731us
I20260709 12:19:24.741127 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:24.644654  1818 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.750721  1718 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:24.656286  1717 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.754609  1717 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:24.755369 28185 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 15 to 19, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 19 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:24.761234  1686 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:24.763988 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39592 (request call id 999) took 1898 ms. Trace:
I20260709 12:19:24.764544 28283 rpcz_store.cc:276] 0709 12:19:22.865239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.865363 (+   124us) service_pool.cc:224] Handling call
0709 12:19:22.865705 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:22.866087 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.866115 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:22.866656 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:19:23.017411 (+150755us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:23.017458 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.017534 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.631979 (+1614445us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b240017ff18. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6209c4 00007f85bb61eacb 00007f85bb7d9dc9 00007f85bb7d9a55 00007f85bb7d909d 00007f85be40b10e 00007f85be42288c 00007f85be421bb6 00007f85be4217d8 00007f85be42b221
0709 12:19:24.763567 (+131588us) raft_consensus.cc:1685] finished
0709 12:19:24.763596 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.763896 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2614,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":24047488,"thread_start_us":150137,"threads_started":1,"wal-append.queue_time_us":222483}
I20260709 12:19:24.768177 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:24.737864  1765 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.734452  1580 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:24.770243 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:24.741922  1476 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:24.736685  1582 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.744966  1854 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:24.788218 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47205.1us
I20260709 12:19:24.773877  1718 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:24.788997 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.790998 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.794006 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:24.799242 28361 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 16: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:24.801059 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:24.749787  1850 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:24.772128  1631 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.773499  1522 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:24.811137 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.814010 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.838008 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:24.840754 28292 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.013s	sys 0.016s
I20260709 12:19:24.805716  1853 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:24.841894 28292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.014s	sys 0.016s
I20260709 12:19:24.792821  1686 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:24.804248  1754 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:24.806318  1808 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:24.781562  1784 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:19:24.789500  1580 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.807576  1718 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.805645  1854 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.800683  1582 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.872051 28436 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:24.854854  1686 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.882409 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:24.847180  1853 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:24.860337 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:24.904456 28358 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:24.848507  1522 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:19:24.838536  1631 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.839697  1850 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.909533 28359 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:24.912103 28361 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:24.928244 28432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53280 (request call id 1060) took 1110 ms. Trace:
I20260709 12:19:24.928720 28432 rpcz_store.cc:276] 0709 12:19:23.817476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.836116 (+ 18640us) service_pool.cc:224] Handling call
0709 12:19:23.836471 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.994555 (+158084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.994593 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:23.994641 (+    48us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:23.994672 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.927948 (+933276us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400347454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:24.928020 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.928175 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363014400}
I20260709 12:19:24.905982  1853 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:24.922693 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 240703us
I20260709 12:19:24.908224  1808 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:24.928390  1818 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:24.958338  1801 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:24.959071  1801 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:24.924995  1784 consensus_queue.cc:1059] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:19:24.961437  1801 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:24.977005 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:24.974217  1861 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:24.980937 28439 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:24.978165 28257 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:25.004566 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:24.946828  1808 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.023231 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:24.980316  1861 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:25.040537 28283 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:25.041903 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 231830us
I20260709 12:19:24.984503  1750 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:25.022575  1848 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:25.042760 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:25.046006 28289 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:25.024240  1818 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.028890  1476 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.079200 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:25.083211 28284 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:25.045588  1750 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:25.090107 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:25.090315  1580 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.074769  1466 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.099742 28284 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:25.094591  1631 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:25.100389  1827 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.664s	user 0.013s	sys 0.000s
W20260709 12:19:25.094269  1894 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 12:19:25.136680 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:25.112035  1891 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:19:25.104574  1807 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.735s	user 0.008s	sys 0.004s
W20260709 12:19:25.115484  1766 consensus_meta.cc:287] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.005s
I20260709 12:19:25.093187  1750 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:25.084479  1899 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:25.107138  1782 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:19:25.121061  1892 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:19:25.117306  1686 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.099799  1808 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.110460  1854 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.111989  1718 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:25.126667  1466 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.115619  1582 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:25.090983  1890 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 12:19:25.148797 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 221718us
I20260709 12:19:25.149927  1807 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:25.153367  1766 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.144156  1850 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.127012  1666 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.187058 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:25.193945 28358 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:25.173558  1760 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:25.219991 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.220227 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:25.227730 28359 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:19:25.167546  1899 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.231326 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:25.240614 28359 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:19:25.235479  1895 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
W20260709 12:19:25.252398 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 162373us
I20260709 12:19:25.253338 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:25.235724  1850 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:25.230121  1646 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:19:25.238060  1845 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:19:25.258275 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:25.267414  1850 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.276304  1666 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:25.277191  1582 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:25.283147  1902 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:19:25.283466  1899 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.284567  1845 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:25.285014  1845 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:25.285981  1845 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.287325  1845 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:25.233968  1795 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:19:25.292670  1795 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:25.293061  1795 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:25.293949  1795 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.295487  1795 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:25.262172  1860 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 1.052s	user 0.009s	sys 0.000s
I20260709 12:19:25.259529 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:25.303073 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43636.5us
I20260709 12:19:25.304025 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:25.293524  1717 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.000s
I20260709 12:19:25.311769 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:25.309178 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:25.313378 28359 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:25.317011 28433 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:25.319584 28432 raft_consensus.cc:1240] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:25.324468 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:25.329267  1831 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.580s	user 0.005s	sys 0.005s
I20260709 12:19:25.300210  1765 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:25.312429  1886 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.522s	user 0.004s	sys 0.004s
I20260709 12:19:25.332535 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 2529) took 1321 ms. Trace:
I20260709 12:19:25.333662  1831 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.333398  1911 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.335245  1911 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.306835  1666 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.341288 28438 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:25.309509  1760 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.348542 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:25.352299 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.332952 28362 rpcz_store.cc:276] 0709 12:19:24.011280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.011418 (+   138us) service_pool.cc:224] Handling call
0709 12:19:24.011794 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.018246 (+  6452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.018280 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.018870 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.379751 (+360881us) spinlock_profiling.cc:243] Waited 5.58 ms on lock 0x7b18003eb660. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:19:24.550168 (+170417us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:24.550217 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.550294 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.286032 (+735738us) spinlock_profiling.cc:243] Waited 58.8 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:25.332266 (+ 46234us) raft_consensus.cc:1685] finished
0709 12:19:25.332292 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.332449 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6174,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":9002,"spinlock_wait_cycles":148171392,"thread_start_us":170139,"threads_started":1,"wal-append.queue_time_us":723244}
I20260709 12:19:25.323601  1850 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.361424 28434 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:25.364940 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 3105) took 1560 ms. Trace:
I20260709 12:19:25.365401 28435 rpcz_store.cc:276] 0709 12:19:23.804431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.805386 (+   955us) service_pool.cc:224] Handling call
0709 12:19:25.364716 (+1559330us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b54000d8754. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:25.364866 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1172504960}
I20260709 12:19:25.370221 28362 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:19:25.348793 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 114412us
I20260709 12:19:25.370718 28362 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:25.371668 28362 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.373221  1718 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.373802  1718 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.372866 28362 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:25.376339  1912 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:19:25.378783  1889 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.481s	user 0.003s	sys 0.006s
I20260709 12:19:25.380409  1752 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.380798 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:25.382259 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:25.382979 28366 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:25.362437  1907 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:19:25.385363  1907 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:25.385759  1907 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:25.386145 28362 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:19:25.386922  1724 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:19:25.386686  1907 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.388092  1907 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.390173 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:25.380429 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:25.396759 28363 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:19:25.374737  1853 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.401340 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:25.402735 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:25.405984 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:25.410241 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:25.412174 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:25.417644 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:25.418027 28359 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:25.415935 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:25.425169 28440 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:25.407744  1850 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.441897 28363 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
I20260709 12:19:25.442549  1867 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.444046 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:25.414860  1811 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:25.447397 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.453473 28991 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:19:25.453948 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.453356 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:25.473810 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 115370us
I20260709 12:19:25.469154 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:25.415061  1871 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.414701  1899 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:25.473155 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:25.485319  1827 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:25.485340 28255 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:25.448487 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:25.406085  1752 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.416746  1797 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:19:25.471482  1850 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.542557  1850 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.480787 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.481623  1853 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:25.482096  1750 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.482764 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:25.545218  1750 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:25.472298  1782 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:25.470515  1911 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.516204 28359 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:25.445694  1464 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:19:25.471338  1811 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:25.475946  1854 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:25.562446 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 101683us
I20260709 12:19:25.564239 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.566210 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.487757  1666 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:25.536303  1827 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:25.574283 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 89042.9us
I20260709 12:19:25.566910  1811 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:25.586565  1897 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.561s	user 0.005s	sys 0.000s
I20260709 12:19:25.588678  1831 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:25.590073 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 2533) took 1353 ms. Trace:
I20260709 12:19:25.590508 28291 rpcz_store.cc:276] 0709 12:19:24.236544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.242501 (+  5957us) service_pool.cc:224] Handling call
0709 12:19:24.242864 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.255440 (+ 12576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.255505 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.633617 (+378112us) spinlock_profiling.cc:243] Waited 7.9 ms on lock 0x7b2c00032f88. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd190372 00007f85be8c2a01 00007f85be88743a 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2
0709 12:19:24.637790 (+  4173us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.856394 (+218604us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:24.856485 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.856583 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.586189 (+729606us) spinlock_profiling.cc:243] Waited 9.19 ms on lock 0x7b240016d8e8. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6209c4 00007f85bb61eacb 00007f85bb7d9dc9 00007f85bb7d9a55 00007f85bb7d909d 00007f85be40b10e 00007f85be42288c 00007f85be421bb6 00007f85be4217d8 00007f85be42b221
0709 12:19:25.589846 (+  3657us) raft_consensus.cc:1685] finished
0709 12:19:25.589871 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.590008 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25942,"raft.queue_time_us":24267,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":33112,"spinlock_wait_cycles":39313024,"thread_start_us":218160,"threads_started":1,"wal-append.queue_time_us":334828}
I20260709 12:19:25.571164  1666 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.563903  1782 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.596020  1782 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:25.600644 28957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.599584 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:25.610410 28435 raft_consensus.cc:1240] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:25.612475 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:25.613124 28435 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:25.616894 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:25.620568 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:25.621593 28437 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:25.622488 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:25.622495 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:25.625660 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:25.626281  1919 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:25.626718 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.627386 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.630918 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.630995 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:25.626744 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.631809 28362 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:25.631497 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:25.603183  1717 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.623009 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:25.633212 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:25.603355  1827 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.637387 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.652448 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:25.653482 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:25.651124 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:25.655997 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:25.650079 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:25.658299 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.659106 28987 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:25.605551  1752 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:25.663662 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:25.664968 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:25.665302 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:25.672384 28437 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:25.673795 28284 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:25.675028 28330 leader_election.cc:400] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17.
I20260709 12:19:25.675493 28330 leader_election.cc:403] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:25.677794 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:25.648339 28366 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:25.682261 28291 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:19:25.682631 28436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2650) took 1335 ms (client timeout 1690 ms). Trace:
W20260709 12:19:25.683148 28436 rpcz_store.cc:269] 0709 12:19:24.346868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.353426 (+  6558us) service_pool.cc:224] Handling call
0709 12:19:25.682407 (+1328981us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b54002e45d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:25.682548 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1188668800}
I20260709 12:19:25.687950 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:25.686555 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:25.664058  1803 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:25.694828  1911 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:25.651087  1899 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:25.698491  1899 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:25.699469  1899 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.669111  1900 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:25.701946  1900 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.702346  1900 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:25.710384 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:25.712426 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:25.665488  1907 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:25.660915  1750 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
W20260709 12:19:25.725044 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 120618us
I20260709 12:19:25.685823  1784 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.736353 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:25.738024 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:25.687563  1212 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.726305  1686 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:19:25.744243  1686 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:25.744611  1686 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:25.677572  1655 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:19:25.746091 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:25.747037  1655 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:25.747427  1655 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:25.748359  1655 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.740316  1750 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.753383  1750 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.742269 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:25.660650  1717 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.758564  1717 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:25.745440  1686 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.759794  1686 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:25.761796  1907 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:25.762894  1907 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.684801  1905 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17.
I20260709 12:19:25.755448  1903 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.768899  1666 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.769905 28435 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
I20260709 12:19:25.770887  1657 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:25.754729  1845 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:25.649787  1919 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.775331 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:25.785090 28363 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 12:19:25.785714 28363 pending_rounds.cc:85] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 16
I20260709 12:19:25.785996 28363 pending_rounds.cc:107] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 12:19:25.747557 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:25.786253 28363 raft_consensus.cc:2889] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:25.766021  1655 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:25.748309  1916 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:25.793620 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:25.796550 28363 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 23.
W20260709 12:19:25.792086 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 50527.7us
I20260709 12:19:25.795272  1818 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:19:25.807235  1900 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:25.775315  1831 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.821410 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:25.824566 28985 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 19: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:25.824103 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:25.824605 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:25.779276  1816 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:25.830572 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:25.831333 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:25.809659  1900 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:25.835400  1750 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:25.792096  1797 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:19:25.830168  1752 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.838054 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:25.848846 28283 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:25.837776 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:25.849861 28359 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:25.826606  1850 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:25.843493 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:25.832026  1818 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.831921  1702 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:25.855770 28330 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:25.837476  1782 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:25.832973  1816 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.863664  1782 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:25.864198  1782 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.856392  1850 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:25.828002  1854 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:19:25.876266 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:25.875510 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 87353.8us
W20260709 12:19:25.939420 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 83887.2us
I20260709 12:19:25.940621 28330 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:26.051640 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:26.053179 28283 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 19: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:19:26.055954 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 115446us
I20260709 12:19:26.058660 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:26.069684 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:26.074672 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:26.071435 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:26.055228  1724 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:26.050279  1631 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:26.150358 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:26.167718  1702 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.168900  1919 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:26.190302 28289 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:26.081547  1752 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.082208  1905 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:26.149341 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:26.180552  1811 consensus_meta.cc:287] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.598s	user 0.004s	sys 0.007s
I20260709 12:19:26.181711  1724 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.181059  1900 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:26.233273 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 194697us
I20260709 12:19:26.222910 28290 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:19:26.236243 28283 raft_consensus.cc:2410] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 21 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 8.
I20260709 12:19:26.207247  1631 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.224781  1724 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:26.217646  1905 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:26.212448  1752 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:26.252748  1842 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:26.218366  1811 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.237901  1750 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:26.242420  1871 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:26.231201  1900 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.312695 28434 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:26.315047 28359 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:26.316898 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:26.268272  1842 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:26.311710  1924 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:19:26.328466 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 269899us
I20260709 12:19:26.329900 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:26.299208  1750 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.303637  1871 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.320432  1848 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:26.334270 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:26.338150  1922 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 12:19:26.385982 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51769.3us
I20260709 12:19:26.388600 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:26.389420 28290 raft_consensus.cc:2368] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:26.389729  1718 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:26.391232 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:26.393330 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:26.343046  1842 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:26.395849 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:26.397291 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:26.401510 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:26.356720  1702 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:26.407816 28288 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:26.332191  1924 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.324261  1905 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.416246 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:26.402767  1368 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:26.403054  1522 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:26.402930  1666 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:26.431310  1927 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:19:26.435310  1871 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.403155  1815 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:26.451807  1815 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:26.452316  1815 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:26.441978 28257 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.3:33721 timed out after 2.075s (SENT) [suppressed 8 similar messages]
I20260709 12:19:26.460143  1702 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:26.460841  1702 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.463747  1631 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.467869  1811 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:26.446571  1750 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.450495  1900 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.451297  1752 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:26.480026 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:26.481189 28363 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:19:26.481828 28284 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:26.506711 28291 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:26.484232 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.490298 28991 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:26.492190 28360 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:26.517549 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:26.520040 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:26.521051 28991 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:19:26.523023 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:26.538180 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:26.540609 28363 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:26.551524 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.553200 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.565825 28365 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
W20260709 12:19:26.573462 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 139266us
I20260709 12:19:26.577494 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:26.535574  1928 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.538452  1843 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:19:26.580739 28441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 2651) took 2216 ms (client timeout 2075 ms). Trace:
I20260709 12:19:26.581228  1848 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:26.583770  1724 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.565960 28291 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
W20260709 12:19:26.585680 28441 rpcz_store.cc:269] 0709 12:19:24.357251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.553282 (+196031us) service_pool.cc:224] Handling call
0709 12:19:26.567601 (+2014319us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b540026cfd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:26.573941 (+  6340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1580871552}
I20260709 12:19:26.534018  1811 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:19:26.606221 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.607268 28363 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
W20260709 12:19:26.609551 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 80394.6us
I20260709 12:19:26.631913  1900 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:26.632769  1900 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.635452  1900 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.636144  1900 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.636654  1900 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.642470 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:26.588450  1843 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.652390 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:26.653378 28285 raft_consensus.cc:2410] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:19:26.604878  1750 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:26.609133  1702 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.674943 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 3123) took 1526 ms. Trace:
I20260709 12:19:26.676859 28440 rpcz_store.cc:276] 0709 12:19:25.148275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.151399 (+  3124us) service_pool.cc:224] Handling call
0709 12:19:25.151664 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:25.432653 (+280989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.674655 (+1242002us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54000d8c54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:26.674863 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628378880}
I20260709 12:19:26.639569  1924 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.648607  1867 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.650686  1843 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:26.691360 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:26.695673 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:26.695137 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 848) took 1065 ms. Trace:
I20260709 12:19:26.594417  1928 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:26.701123 28985 rpcz_store.cc:276] 0709 12:19:25.629750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.681289 (+ 51539us) service_pool.cc:224] Handling call
0709 12:19:26.694766 (+1013477us) spinlock_profiling.cc:243] Waited 67.7 ms on lock 0x7b540046e954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:26.694918 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155596672}
I20260709 12:19:26.707121 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:26.701159 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:26.676940  1657 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:26.726657 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:26.730587 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:26.747977 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.651208  1632 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:19:26.678042  1522 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:19:26.712857  1831 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:26.753108 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.683244  1750 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:26.788882 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.750802 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.751398 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.709020  1900 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.680569  1903 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:19:26.790838 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:26.772204  1782 consensus_meta.cc:287] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.908s	user 0.015s	sys 0.000s
W20260709 12:19:26.792845 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 89450.9us
I20260709 12:19:26.754643 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:26.725229  1807 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:26.782785 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 72612.7us
I20260709 12:19:26.790784 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:26.799638 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 98559.3us
I20260709 12:19:26.773531  1702 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.803025 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:26.739919  1850 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.775744  1910 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.755857  1816 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.705806  1867 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:26.784024  1937 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:26.788070  1934 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:26.729094  1657 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:26.803673  1831 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:26.715231  1848 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:26.824455  1831 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.708724  1932 consensus_queue.cc:1059] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
W20260709 12:19:26.830659 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 123647us
I20260709 12:19:26.808941  1900 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.815639  1811 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:26.818552  1937 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:26.804613  1782 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.829203  1816 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:26.874964  1900 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:26.851255  1724 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:26.875483  1940 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:19:26.880488  1900 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:26.876606  1782 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:26.881881  1867 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:26.877417 28358 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:26.816601  1833 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:19:26.876600  1750 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:26.881584  1900 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.849893  1811 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:26.888516 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 89085.9us
I20260709 12:19:26.897140  1657 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:26.870985  1937 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:26.877750  1854 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:26.908274  1646 raft_consensus.cc:3055] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:19:26.909403 28366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 2558) took 1508 ms (client timeout 1564 ms). Trace:
I20260709 12:19:26.818248  1910 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:26.948984 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 114735us
I20260709 12:19:26.883517  1928 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:26.972401 28366 rpcz_store.cc:269] 0709 12:19:25.400896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.401038 (+   142us) service_pool.cc:224] Handling call
0709 12:19:26.909160 (+1508122us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400126e54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:19:26.909320 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514786176}
I20260709 12:19:26.932545  1811 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:26.978570 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:26.970306  1522 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.989718  1522 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:26.991879  1522 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:26.992565  1815 consensus_meta.cc:287] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.540s	user 0.013s	sys 0.000s
I20260709 12:19:26.974570  1854 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:26.995128  1854 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:26.995781  1657 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:26.996309  1657 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:26.998400  1657 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:26.976883  1910 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:26.999274  1655 consensus_meta.cc:287] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 1.203s	user 0.010s	sys 0.005s
I20260709 12:19:26.883414  1816 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.006363  1816 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:26.993055  1815 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.008169 28359 raft_consensus.cc:2410] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:19:27.010172  1815 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:27.014019  1842 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.004557  1811 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.016134  1811 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:26.972162  1646 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:27.019141 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:27.020627 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.038281 28287 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:27.039062 28257 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:27.040163 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 235337us
I20260709 12:19:27.046042 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:27.046818 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:27.048957 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:27.049219 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:27.040197  1816 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:27.040493  1655 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:27.050454 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:27.054993 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:27.058101 28437 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:27.055317 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.059609 28440 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:27.043608  1854 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:27.020202  1794 consensus_queue.cc:1175] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 17 status { last_received { term: 17 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 17 index: 10 } }
I20260709 12:19:27.083228 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.084831 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:27.056689  1816 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:27.086462  1816 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:27.086643 28435 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:27.102077 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.106058  1932 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:27.106833  1848 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:27.107307  1932 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:27.107754  1932 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:27.107442  1848 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.108719  1932 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.085254 28438 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
W20260709 12:19:27.117103  1939 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.281s	user 0.005s	sys 0.004s
I20260709 12:19:27.121285 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:27.050498  1818 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:27.056916  1815 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:27.127898  1815 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.130396  1803 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:27.131603  1854 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.132169  1854 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.132645  1212 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:27.131448  1803 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.135843  1464 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:27.138823  1464 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.139844  1903 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:27.140583  1903 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:27.150645 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 71690.7us
I20260709 12:19:27.155906 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.072078  1646 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.156347 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.156977 28360 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:27.162858 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.163409 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:27.163861 28366 raft_consensus.cc:2393] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 19: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:27.164197 28365 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:27.171861 28439 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 21: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:19:27.171996 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 65611.2us
I20260709 12:19:27.179345 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:27.178400 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.184159 28432 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:27.178732 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.183621 28989 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:27.177003 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:27.178056 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.189209 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.189924 28288 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:27.191272 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.191869 28288 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:27.177762 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:27.194555 28433 raft_consensus.cc:2393] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 18: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:27.189469 28436 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:27.195493  1848 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:27.198151  1646 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:27.201613 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.203294 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.205350 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.206806 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.208429 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.209270 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.247798  1784 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:27.248422  1784 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.249671 28441 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:27.249917 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.210798  1803 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:27.250684 28288 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:27.214398  1903 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:27.225957 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.227685 28437 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
W20260709 12:19:27.231976 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.232388 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.260790 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.233757 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:27.234287  1843 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.000s
I20260709 12:19:27.234882 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.235071  1953 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:27.269397 28989 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:19:27.236781  1464 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:27.239552  1937 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.240973  1815 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:27.241994  1686 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:27.242689  1854 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:27.246423 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 446318us
I20260709 12:19:27.252259 28365 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:19:27.172957  1932 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:27.253459 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.313050 28257 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:27.254782  1831 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:27.255710 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.255882  1784 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:27.257437 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.262247 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.236323 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:27.270370 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.323902 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.325047 28437 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:27.327230 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:19:27.323836 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 87617us
I20260709 12:19:27.328225 28984 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 19: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:27.271098 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:27.271522 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.329274 28366 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:24.245362 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:27.277597  1752 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.256059  1900 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.282938 28185 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.312831 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.313454 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.316112 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:27.316856 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.346127  1815 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:27.340042  1842 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:27.351778  1961 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:19:27.353281  1815 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:27.353070  1752 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:27.353941 28186 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 225) took 41206 ms (client timeout 15000 ms). Trace:
I20260709 12:19:27.354370  1815 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.356837 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:27.269668  1953 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:27.340448  1655 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:27.358590  1818 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:27.358460  1953 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.359934  1953 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.266094  1808 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:27.362169  1808 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:27.354362 28186 rpcz_store.cc:269] 0709 12:18:46.148184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.148582 (+   398us) service_pool.cc:224] Handling call
0709 12:18:46.567050 (+418468us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f1090 00007f85be88ff3a 00007f85c4645e9c 00007f85c474404e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc 00007f85c2c04392 00007f85bcbc248c 00007f85bcbc18c6
0709 12:19:27.353818 (+40786768us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:19.141452 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:19.232150 (+ 90698us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:19.232207 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:19:19.234375 (+  2168us) tablet.cc:662] Decoding operations
0709 12:19:19.241929 (+  7554us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:19.242029 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:19.242058 (+    29us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:19:19.243537 (+  1479us) tablet.cc:701] Row locks acquired
0709 12:19:19.243590 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:19:19.243783 (+   193us) write_op.cc:270] Start()
0709 12:19:19.243938 (+   155us) write_op.cc:276] Timestamp: P: 1783599559243713 usec, L: 0
0709 12:19:19.243977 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:19:19.244860 (+   883us) log.cc:844] Serialized 7379 byte log entry
0709 12:19:20.038966 (+794106us) op_driver.cc:464] REPLICATION: finished
0709 12:19:20.060888 (+ 21922us) write_op.cc:301] APPLY: starting
0709 12:19:20.060983 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:20.070416 (+  9433us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:20.070569 (+   153us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:20.080783 (+ 10214us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:20.081151 (+   368us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:19:20.081198 (+    47us) write_op.cc:312] APPLY: finished
0709 12:19:20.081566 (+   368us) log.cc:844] Serialized 164 byte log entry
0709 12:19:20.081801 (+   235us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:20.082000 (+   199us) write_op.cc:454] Released schema lock
0709 12:19:20.082074 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:20.118538 (+ 36464us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:20.125325 (+  6787us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":638537,"reported_tablets":52,"spinlock_wait_cycles":65196288,"child_traces":[["op",{"apply.queue_time_us":21639,"apply.run_cpu_time_us":21711,"apply.run_wall_time_us":30962,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79331,"num_ops":18,"prepare.queue_time_us":432892,"prepare.run_cpu_time_us":14298,"prepare.run_wall_time_us":170555,"raft.queue_time_us":564090,"raft.run_cpu_time_us":1344,"raft.run_wall_time_us":22891,"replication_time_us":794789,"spinlock_wait_cycles":1150848,"thread_start_us":764638,"threads_started":4,"wal-append.queue_time_us":35541,"wal-append.run_cpu_time_us":34872,"wal-append.run_wall_time_us":1805192}]]}
I20260709 12:19:27.366174 28434 raft_consensus.cc:2393] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:27.314055  1464 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:27.311617 28436 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:27.340303 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:27.375636 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:27.269330  1843 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.358353  1631 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:27.340652 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 69695.3us
I20260709 12:19:27.379478 28185 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.335755  1949 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:27.381286 28185 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.339267 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:27.313650  1686 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.328086 28330 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:27.387429 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48283.7us
I20260709 12:19:27.389422 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.389667 28185 catalog_manager.cc:5697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 6 to 18. New cstate: current_term: 18 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.390190 28986 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:19:27.326212  1831 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:27.387904 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 133963us
I20260709 12:19:27.394165 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:27.374570  1464 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.383674  1811 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:27.388926 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 60931us
I20260709 12:19:27.395211  1655 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:27.400683 28185 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 9 to 11, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 11 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:27.388779  1848 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.405247 28435 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:27.397452 28330 leader_election.cc:400] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:27.385082  1949 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:27.406469  1949 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:27.390343  1784 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:27.408867 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:27.409350 28330 leader_election.cc:403] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:19:27.404891 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:27.407804 28185 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 2 to 6, term changed from 13 to 18, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2), f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1), VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2), VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3), VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:27.409536 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.413839 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.417474 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:27.416677  1854 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:27.423957 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "189fdd6055df4826a42af9c225596e34"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:27.419920 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:27.424575 28359 raft_consensus.cc:606] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.425004 28359 raft_consensus.cc:614] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:27.402155  1464 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:27.425412 28257 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:27.415360  1766 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:27.418398  1853 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:19:27.427186  1937 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:27.433936  1831 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:27.402973  1811 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.420673  1854 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:27.437177  1811 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:27.419319  1954 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16.
I20260709 12:19:27.404750  1631 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:27.421470  1850 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:27.431710  1843 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:27.443394 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:27.444015  1900 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:27.427455  1784 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:27.438758  1686 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:27.437132  1854 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:27.448729 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.449803 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:27.429984 28185 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 6 to 12, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 12 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:27.444279 28436 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:27.453064 28185 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 16, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 16 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.426820  1950 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:27.457304 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:27.454742 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:27.458972 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:27.455431  1954 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:27.460363 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.460119 28185 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 16, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 16 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.462256 28985 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:19:27.464031 28985 pending_rounds.cc:85] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 10
I20260709 12:19:27.464890 28985 pending_rounds.cc:107] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:19:27.465175 28985 raft_consensus.cc:2889] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:27.440393  1631 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:27.450204  1686 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:19:27.470685 28257 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.484997  1931 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:27.486935 28185 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 21. New cstate: current_term: 21 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.458417  1948 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:27.484759 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:27.489451 28185 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.490305  1899 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:27.485201  1848 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:27.494614 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:27.510619 28363 raft_consensus.cc:606] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.498207  1752 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:27.487778 28257 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:27.461439  1853 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:27.461612  1850 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:27.461313  1900 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:27.440961  1816 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.513181 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:27.525966 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:27.485085  1631 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:27.522428 28185 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.515829  1831 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:27.528666 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.528929 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.531032 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.533126 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:27.534204 28283 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:27.536111 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:27.538225 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.542524 28288 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:27.526795  1750 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.543919 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:27.543759  1750 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.548593  1655 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.553054  1949 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.515681  1871 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:27.529942  1831 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:27.578182 28291 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:27.554193 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:27.579190 28363 raft_consensus.cc:606] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:27.579563 28363 raft_consensus.cc:614] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:27.530514  1811 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:27.557031 28284 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:27.558121  1816 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:27.558588 28185 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 9 to 13, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 13 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:27.593152 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:27.560889  1750 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:27.594110  1931 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:27.562680 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:27.595085 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:27.597115  1686 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:27.597342  1368 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:27.598166  1784 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:27.563411  1903 raft_consensus.cc:2764] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:19:27.599133  1924 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:27.571923  1854 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.560492 28285 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:27.600595  1752 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.601171  1752 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.575207 28283 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:27.583279  1831 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.587210 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.617252 28292 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:27.583684  1871 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.586874 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:27.583950  1811 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.591533 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:27.622394 28289 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 19: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:27.592661 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:27.623144 28288 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:27.599552  1924 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:27.516463  1850 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.594099 28185 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.625653 28185 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 13 to 17, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 17 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.603670 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:27.604846 28984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "777967d6330c4654813298b12226a560"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:19:27.609846 28290 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:27.611534  1854 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:27.615182 28435 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:19:27.628312 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:27.615348 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 105463us
I20260709 12:19:27.616184 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:27.629096 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:27.613842 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:27.632709 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:27.652145 28366 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:27.652624  1368 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:27.654127 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:27.653705 28185 catalog_manager.cc:5697] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 3 to 11, term changed from 11 to 18, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 18 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:27.627836 28984 raft_consensus.cc:606] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:27.659526 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.659677 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.653611 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.659475 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:27.661092 28363 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:19:27.667462 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:27.669566 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:27.625139  1953 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:27.674737 28185 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 7 to 9, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 9 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:27.621627  1871 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:27.625121  1811 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:27.648361  1850 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:27.648180  1656 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:27.733997 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 89238.4us
W20260709 12:19:27.734310 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 69177.2us
I20260709 12:19:27.742801 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:27.648079  1843 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.663146  1907 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.662382  1848 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:27.755014 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.662914  1923 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:27.648017  1931 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 12:19:27.675897  1854 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:27.771857 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.662009  1900 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 12:19:27.776597 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.670187  1655 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.103s
I20260709 12:19:27.690138  1717 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 12:19:27.759761  1848 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:27.745555  1368 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:27.790899 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:27.804198 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:27.805428 28364 raft_consensus.cc:2376] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18. Re-sending same reply.
I20260709 12:19:27.774094  1934 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:27.769398  1808 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:27.757663  1656 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.809345  1934 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.756830  1843 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.805523  1848 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:27.781195  1816 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:19:27.776722  1854 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.807541  1655 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:19:27.801870  1899 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:19:27.823143  1934 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:27.771018  1923 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.767176  1924 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:27.844518 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 181939us
W20260709 12:19:28.035027 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 365315us
I20260709 12:19:27.857036 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:19:28.052121 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 194993us
I20260709 12:19:28.054634 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:28.057482 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.057778 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:28.058284 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.053503 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:28.059723 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:28.061040 28364 raft_consensus.cc:2393] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:28.072885 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.070899 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:28.047524 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:27.920607  1368 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.834050  1655 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:28.195191  1655 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.947335  1656 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:27.966162  1752 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:28.197723  1655 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:28.109211  1686 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.109246  1808 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:27.993182  1811 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.232834  1811 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:28.113059  1815 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:28.233961  1815 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:28.235360  1815 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:28.236243 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 188910us
I20260709 12:19:28.238123 28440 raft_consensus.cc:2368] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:19:28.238929  1899 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:28.239849  1899 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:28.240492  1899 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.967075  1871 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.947188  1843 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:27.998932  1848 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.976380  1850 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.960041  1854 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:27.992056  1915 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:28.247124  1915 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:28.248653  1915 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.971357  1923 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:27.974453  1924 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:27.991627  1934 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:27.935106  1949 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:27.993585  1972 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.142s	user 0.002s	sys 0.000s
I20260709 12:19:27.926440  1977 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:28.256532  1977 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:28.258085 28287 raft_consensus.cc:2368] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19.
I20260709 12:19:27.951099  1978 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:28.259536  1978 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:27.934907  1980 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:28.261271  1980 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:27.987377  1981 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:19:27.930258  1984 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:28.263703  1984 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.266194  1984 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:28.162082  1831 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:28.268432  1831 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:28.173069  1999 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:28.167836  1948 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:28.269972  1999 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.270280  1948 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.184799  1750 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:28.271739  1750 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.185423 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4036b67205e41c597049a10a71e9467"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:28.273399 28363 raft_consensus.cc:606] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.273598  1977 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:28.175812  1952 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:28.276839  1952 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.192595  1931 raft_consensus.cc:993] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:28.279196  1931 raft_consensus.cc:1081] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:28.279588  1899 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.282842 28985 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:28.269145 28435 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:28.268702 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:28.150436 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 213449us
I20260709 12:19:28.295929 28363 raft_consensus.cc:614] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:28.310416 28434 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:28.326485  1899 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:19:28.333056 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 214690us
I20260709 12:19:28.337091 28362 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:28.345022  1978 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.346714  1727 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:28.347543  1727 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:28.350103  1995 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:28.351104  1995 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.335237  1915 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.353135  1948 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.325979  1686 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.356138  1686 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.362677 28290 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:28.363426 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:28.364410 28289 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:19:28.367044 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:28.367697  1899 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.341727  1871 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.346868  1848 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.371762 28436 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:19:28.372530 28436 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:28.372937 28436 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:28.380089 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 51400.2us
I20260709 12:19:28.370318  1899 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:28.389081  1631 consensus_meta.cc:287] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.857s	user 0.011s	sys 0.009s
I20260709 12:19:28.391824 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.392731 28364 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:19:28.393990 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.397850 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.398627 28433 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:19:28.362527 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:28.406282 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65881.8us
I20260709 12:19:28.409029 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:28.412163 28363 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:28.412994 28363 raft_consensus.cc:606] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.413456 28363 raft_consensus.cc:614] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:28.418691 28287 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 20: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:28.424945 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:19:28.426760 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.429981 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.432366 28437 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:28.435806  1750 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.436583  2003 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:28.437414  1903 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:28.450507  1903 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:28.443475 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:28.446483  1934 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:28.443423  1850 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.448819  1952 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.443228  1995 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.449841  1727 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:28.458783 28436 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.462740 28436 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:28.464452 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.466472 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:28.466801  1999 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.467229  1980 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.461606 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:28.405848  1631 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.470317  1903 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.478186 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:28.478945  1797 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:28.479734  1797 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:28.480935  1845 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:28.481475  1845 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.483218  1845 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.483707  1797 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.486500 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:28.481068  1631 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.491470 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:28.493260 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:28.494186 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:28.495019 28287 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:28.490991  1843 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:19:28.509915 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:28.498150 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:19:28.514765 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 448043us
I20260709 12:19:28.516892 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:28.521071  2003 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:28.523090  2003 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.523496  2003 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.524112 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.527894 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:19:28.530170 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 121253us
I20260709 12:19:28.545138 28359 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.552073 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:28.556111 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.556996 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:28.558357 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.531822 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:28.570744 28366 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.560485 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.570340 28286 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:28.559079 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.584313 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.586769 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.582950 28289 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:28.586313 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:28.589238 28364 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:28.585637 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.530743  1946 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:19:28.590382 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.592234 28358 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:19:28.599313 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.599721 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:28.627836 28255 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:28.628293 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.628558  1797 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:28.633153 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:28.649228 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 88908.9us
I20260709 12:19:28.655942 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.658905 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.654826 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:28.629995  1980 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:28.657696 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:28.632738  1797 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.638998  1686 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:28.665032  1912 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:28.632056  2003 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.684806  1924 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:28.685581  1924 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.689816 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.690796 28330 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.692373  1815 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:28.692830 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.695168  2002 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:28.695956  2002 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:28.698966  2002 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:28.693213  1924 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.701074  2003 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.663309  1980 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:28.702065 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.703024 28432 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 18: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:28.703398 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:28.692888  1815 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:28.668285  1686 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:28.706624  1815 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:28.708642 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.668458  1797 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:28.715628 28433 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.667714  1727 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:28.715988 28364 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:28.701990  1980 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:28.718321 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 56216.5us
I20260709 12:19:28.706836  1686 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:28.723511 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.738829 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.753945 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.754618 28359 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:19:28.749706 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:28.759924 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:19:28.778936 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 86281.7us
I20260709 12:19:28.776196  1907 raft_consensus.cc:2764] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:19:28.780609 28435 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.744004  1845 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:28.790594 28359 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:28.794056 28257 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.794832 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.795229 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:28.804679 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.806236 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:28.790814  1934 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.808987 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:28.809669 28358 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:28.810920 28365 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:28.778193  1727 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.813476  1368 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:28.823463 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:28.824312 28367 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:28.826787 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:28.824646 28367 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:28.835193 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:28.837112 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:28.819499  1368 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.838023 28991 raft_consensus.cc:2368] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
I20260709 12:19:28.827342  1655 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.839882 28990 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:19:28.840323 28990 pending_rounds.cc:85] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 13
I20260709 12:19:28.840585 28990 pending_rounds.cc:107] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:19:28.840869 28990 raft_consensus.cc:2889] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:28.841831 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:28.842972 28330 leader_election.cc:400] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19.
I20260709 12:19:28.847447 28330 leader_election.cc:403] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:19:28.838112  1368 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.864979 28365 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:19:28.808157  1934 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.869184 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:28.848065  1656 raft_consensus.cc:993] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:28.870654 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:28.814153  2003 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:19:28.865835 28367 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:28.883046 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 223989us
I20260709 12:19:28.884078 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.890702  1845 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.892849  1934 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:28.893919  1655 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:28.885574 28257 leader_election.cc:400] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:19:28.896777 28257 leader_election.cc:403] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:19:28.883232  1946 raft_consensus.cc:1081] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:28.897903 28367 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:28.908869 28366 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:19:28.893482  1934 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.916961 28435 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:19:28.909767 28404 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:28.924554 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.923125 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:28.929636 28434 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.915829  1808 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.938764 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:28.913930  1931 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19.
I20260709 12:19:28.943101 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.945571 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:28.951134 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.920763  1631 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:28.932287  1984 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:28.958079 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:28.900379  1368 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.923584  1934 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:28.949872  1995 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:28.950663  2003 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.960261  1999 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:28.986742 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 90740us
I20260709 12:19:28.973634  1952 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.998098 28436 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:28.985176  2003 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.001430  2003 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.007441  1920 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:19:29.018028  1727 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.019907  1368 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:29.020119  1945 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:29.020608  1945 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:29.023530 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:29.024753 28434 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:19:29.028834 28433 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:19:29.022119  1797 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.045367 28985 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 20: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:19:29.046702  2017 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:19:29.047678  1797 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:29.054097 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47232us
I20260709 12:19:29.055473 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:29.056962 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:29.058413  1987 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:29.058837  1987 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:29.059221  1987 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.061704  1903 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:29.038069 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:29.067137 28367 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:19:29.038426  1977 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:29.074828 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 96780.8us
I20260709 12:19:29.077782 28257 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:29.069643  1915 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:29.081156 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:29.081560  1850 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:29.091373 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:29.077167  1655 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:29.021677  1945 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.099781  1850 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:29.133857  1850 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.136904  2024 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:29.138300  2000 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:29.139030  2000 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:29.138161 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 46931us
I20260709 12:19:29.098976 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:29.100013 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:29.144820 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:29.094655  1915 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:19:29.145963 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47128.7us
I20260709 12:19:29.147920 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:29.082686  1686 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.151477 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:29.141135  1977 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:29.140801  1655 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.146363  1952 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:29.156488  1999 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:29.157243 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:29.157944 28365 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:29.151439  1924 raft_consensus.cc:2764] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:19:29.159018 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:29.159875 28362 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 12: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:29.161059 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:29.161799 28360 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:19:29.162580  1995 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:29.163192  1995 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:29.163419  1686 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.164144  1686 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:19:29.165961 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.154357 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:29.148458  2014 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.363s	user 0.004s	sys 0.002s
I20260709 12:19:29.144817  2024 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:29.164688  1686 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:29.156823  2020 log.cc:927] Time spent T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:19:29.171562 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:29.172737 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 77039.3us
I20260709 12:19:29.170149  1631 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:19:29.170285  1934 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.175836  1945 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.176755  1999 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:29.194180  1368 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:29.194826  1368 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.195211  1368 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.176038  1850 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.198055  1977 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.206393  1977 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.207069 28434 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:29.209633 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.217772 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.219734  1915 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.222604  2000 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.173484  2024 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.225173  2024 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:29.227306  1977 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:29.229893 28360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "777967d6330c4654813298b12226a560"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:56626
I20260709 12:19:29.230629 28360 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:29.231055 28360 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.233147 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:29.234004 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:29.234800 28362 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 20: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:29.236256 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.236907 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.237573 28290 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.243921  1655 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:29.188227  1995 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.189826  1952 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:29.247362 28361 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.249011 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:19:29.249859 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 95575.6us
W20260709 12:19:29.215473 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 64118.8us
I20260709 12:19:29.253933 28257 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:29.255035 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:29.215279  1999 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:29.260128 28404 leader_election.cc:400] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
I20260709 12:19:29.261209 28404 leader_election.cc:403] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:29.259501 28255 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:29.267328  1686 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.267625  1952 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:29.268558  1999 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.275002  1903 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
W20260709 12:19:29.276060  1980 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.535s	user 0.010s	sys 0.004s
I20260709 12:19:29.276544  1980 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.278841  1980 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.279592  1987 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.280725 28433 raft_consensus.cc:1240] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:29.284132  1815 consensus_meta.cc:287] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.002s
I20260709 12:19:29.287197 28985 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:29.262262 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:29.299132  1803 consensus_queue.cc:1059] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:29.300028  1843 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:29.300390  1803 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:29.300787  1803 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:29.302171  1803 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.302731  2023 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:29.275298 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:29.305773  1808 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:29.261510  1655 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:29.306209  1808 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:29.306679  1808 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.308288 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:29.304025  1815 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.296479 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.306334  1984 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:29.310415  2022 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:29.310982  2022 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:29.312083  2022 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.316362 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:29.290937  1987 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:29.322266 28361 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:29.329113 28290 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:19:29.330633 28436 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 14: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:29.330539 28289 raft_consensus.cc:1240] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:29.331990 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 73955us
I20260709 12:19:29.333999 28360 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.325979  1815 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.344810 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:29.346228 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:29.348469  1934 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:29.348958  1934 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.349264  1934 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.354005  2024 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:29.354606  2024 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.354959  2024 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:29.360472 28291 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:29.365382 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.367238 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:29.279775  2002 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:29.368603 28284 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:29.371517 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.372769 28287 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:29.371071 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:29.375716 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.379134 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:29.342356  1980 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:29.379935 28367 raft_consensus.cc:2368] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19.
I20260709 12:19:29.379719 28360 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:29.275255  1977 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.093s
I20260709 12:19:29.364413  2023 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:29.393676 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:29.382676 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:29.371905  2002 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:29.380838  1803 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.381862  1946 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:29.419335 28291 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:19:29.400298  1980 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:29.429219 28255 leader_election.cc:400] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19.
I20260709 12:19:29.440732 28255 leader_election.cc:403] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:19:29.428524 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 119245us
I20260709 12:19:29.448324 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:29.417287  1999 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.455240 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.456034 28286 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:19:29.467095 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.449795  1912 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.456314  1655 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.472323 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.417016  2023 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.476735 28283 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:29.464110  1999 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.485977  2002 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.487548  2002 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.489940 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.490669 28285 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.490882  2023 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:29.479765  1815 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:29.494051  1815 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.474155  1912 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.506542  2028 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:29.506369 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.508701  2028 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.507925 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:29.511471 28984 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:29.511790 28984 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:29.512707 28984 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.491218  1368 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.513785 28984 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.514537 28441 raft_consensus.cc:2368] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18.
I20260709 12:19:29.515674  1368 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:29.517115 28332 leader_election.cc:400] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18.
I20260709 12:19:29.517468 28332 leader_election.cc:403] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:19:29.518144  2023 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.519304 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:29.520222 28986 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 24: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:29.529127  1980 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:29.531474  2030 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:19:29.534617 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:29.536309  1999 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.536664 28286 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:29.554646 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.555104  2000 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:29.555585 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.555718  2000 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.558944  1912 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.560076  2000 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.562243 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.564577 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.565274 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.565937 28288 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.566244 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.571126  2028 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.574323  2003 consensus_queue.cc:1059] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:29.575642  1727 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:29.576081  1727 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:19:29.505923 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 124617us
I20260709 12:19:29.577111  1727 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.578899  1727 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.579833 28288 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:29.575546 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.580793 28286 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 24.
I20260709 12:19:29.585546  1815 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:29.586252 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.587002  1934 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.589267  1934 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.591338 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.591917 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.593063 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.594761 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.596451 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.599702 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:29.600555 28441 raft_consensus.cc:2368] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19.
I20260709 12:19:29.606676 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:29.607780 28366 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:29.610774 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:29.612936 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.613875 28288 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:19:29.617492 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.620749 28984 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:29.612498 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:29.634815 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:29.635519 28366 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:29.638840 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.640477 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.642987 28436 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:19:29.645018 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:19:29.650451 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 171014us
I20260709 12:19:29.652278 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:29.606240  2022 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:29.655591 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:29.652407 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:29.644600  1843 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.659020 28990 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:29.672034  2022 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:29.673236  2022 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.579880  1808 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.679659  1808 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:29.683380  2002 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:29.652336  2003 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:19:29.660087  1815 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:19:29.687870  1815 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:29.688306  1815 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:29.688903 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:29.689849  2034 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:29.690377  2034 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:29.673097  1843 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.691416  2034 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.698065 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:29.699158 28291 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:29.699321 28367 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:29.700932 28285 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:29.702970 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:29.715453 28366 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:29.654903  2037 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:29.673048  2023 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18.
I20260709 12:19:29.722226 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:29.724741 28360 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:29.731839 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:29.730829 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 69197.8us
I20260709 12:19:29.734594 28364 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:29.740155 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:29.741147 28363 raft_consensus.cc:2368] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
W20260709 12:19:29.752907 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 100591us
I20260709 12:19:29.754047 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:29.764338  2027 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:29.707790  2024 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.775583 28283 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:19:29.782823 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.689256  1815 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:29.794737 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.801272 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.803421 28286 raft_consensus.cc:2368] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
W20260709 12:19:29.806079 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 50539.4us
I20260709 12:19:29.808657 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:29.810789 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.811405 28287 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:29.812145 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 102544us
I20260709 12:19:29.810595 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.819120 28290 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 20: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:29.772927  1920 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:29.824203 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:29.775266  1907 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:29.820402 28292 raft_consensus.cc:1240] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:29.823766 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.826392 28289 raft_consensus.cc:2368] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:29.833277 28367 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:29.838034 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:29.838714 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:29.839380 28436 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.841791 28434 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:29.732592  2037 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.847616 28283 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:19:29.836011 28255 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:29.853379 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.854410 28285 raft_consensus.cc:2368] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:29.855015 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:29.856909 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:29.857090 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.857607 28288 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:19:29.858163 28283 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:29.858235 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 126439us
I20260709 12:19:29.835299  1920 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.859578 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.823588  1686 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:29.868854 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:29.862370 28991 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:19:29.844050  2056 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:29.861872  1920 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:29.873190  1907 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:29.882138  2048 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 12:19:30.027370  1843 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:30.034667  1845 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:30.018018  2058 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:29.854048  2003 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:19:30.010633  2024 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:29.942605  2050 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:30.024699  1803 consensus_meta.cc:287] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.004s
I20260709 12:19:30.004798  2027 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.094729  2027 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.095948  1808 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:30.096577  1808 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:29.939585  1631 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:30.090966  2058 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.098263  1631 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.098814  1808 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:30.101344  1631 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:29.871781  1924 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:29.994638 28283 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:29.869656  1686 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.054948 28436 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
W20260709 12:19:30.017328  1655 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.541s	user 0.013s	sys 0.000s
I20260709 12:19:29.938591  2023 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.876725  2056 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:29.999977  2037 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:29.987946 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 247865us
W20260709 12:19:29.995580 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 113441us
I20260709 12:19:30.059103 28291 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:29.975311 28287 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:19:30.070060  2050 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:30.072669  1815 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.082715 28360 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:19:30.083441 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:30.000908 28257 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.104496 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:30.107712 28361 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
W20260709 12:19:30.122860 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 267932us
I20260709 12:19:30.133477 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:30.128509 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:30.133867 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:30.117395  1655 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19.
W20260709 12:19:30.144783 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 143952us
I20260709 12:19:30.147876 28257 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.151806  1686 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:30.153539  2050 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.154388 28285 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:30.154816  1920 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.155774 28990 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:19:30.157261  1980 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:30.158200  2000 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:30.158643  2000 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:30.159005  2000 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:30.153273  1999 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:30.148360 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:30.162312 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:30.163911 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:30.149214 28257 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.163849 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:30.166388  1655 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:30.170082 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:30.170253  1999 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:30.170845 28433 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:19:30.179569  1797 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:30.183146 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:30.184517  1686 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:30.122030  2056 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:30.185130  1686 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:30.186131  1686 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.187837  1920 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:30.151806  2058 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:30.194890 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:30.164310 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:30.197204 28287 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:30.198963 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49843.9us
I20260709 12:19:30.199867 28257 leader_election.cc:400] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19.
I20260709 12:19:30.201843 28433 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:19:30.211236 28363 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:19:30.213104 28956 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:30.209443 28255 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:19:30.207417 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43659.1us
I20260709 12:19:30.215346 28330 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.202659 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.118530  2023 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.218488  1803 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:30.241918  1803 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:30.242383  1803 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:30.242798 28330 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.243754  1924 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:30.244259  1924 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:30.245168  1924 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.164225  2027 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.246764  1907 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:30.247311  1907 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:30.247603  2027 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:30.248199  1907 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.252146 28289 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:30.252938  2056 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.222155  1999 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:30.230669  1920 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:30.231382 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:30.238359 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:30.278083  1631 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:30.280084 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 69977.7us
W20260709 12:19:30.220203  2064 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:19:30.211843  2058 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:30.241477 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:30.242481  2040 consensus_queue.cc:1059] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
W20260709 12:19:30.319725 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 100136us
I20260709 12:19:30.326614 28359 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:30.330781 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:19:30.328553 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 87164.3us
I20260709 12:19:30.312451 28367 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
W20260709 12:19:30.325259 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 87000.6us
I20260709 12:19:30.318555 28287 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:19:30.336127 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.345300  1995 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:30.335021  2024 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:30.347162  2024 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.330750  1920 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:30.348322 28404 leader_election.cc:400] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:30.348742 28404 leader_election.cc:403] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:19:30.346515  2059 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:19:30.330647  2065 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:30.358465 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:30.358326  2065 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.360250  2048 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:30.362308  1920 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.331085  1845 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:30.370781 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:30.375244 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:30.335052  2034 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:30.376722  2034 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.377189  2034 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.377413 28436 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:30.379305  2024 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:30.380517 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:30.381177 28985 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:30.382066 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:30.382589 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:30.375253 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:30.384903 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:30.385715  2052 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:30.386336  2052 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:30.387540  2052 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.388487  1977 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:30.389045  1977 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:30.390631  1977 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.391719  2000 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.393234 28432 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:30.394145 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:30.395471  2065 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:30.396515  2059 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:30.398748  2003 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:30.397473  1803 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.430902  2056 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:30.431133  2023 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:30.462333  1803 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:30.432488  2034 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.438083 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.465296 28436 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:30.465649  2037 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:30.466187  2037 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:30.466581  2037 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:30.467890  2034 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:30.446143  2054 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:30.444796 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:30.447923 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:30.407989  2024 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:30.424260  1980 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.422428  1903 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:19:30.452770 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:30.470700 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:30.474622 28985 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 24: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:19:30.473304  1980 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.447158 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:30.472874  2054 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:19:30.485163 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 45412us
I20260709 12:19:30.487846 28363 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:30.491959 28367 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:30.484855  2024 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:19:30.529281  2024 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:30.534931 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:30.484859  2054 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:30.543175 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:30.536000  1995 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.576398 28289 raft_consensus.cc:1240] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:30.578835 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:30.581673 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:30.554183  2023 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:19:30.589457 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 162991us
I20260709 12:19:30.594993 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:30.599912 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:30.606027 28360 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:30.607213 28434 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:30.608111 28283 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 24.
I20260709 12:19:30.585779  2047 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:19:30.608731 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:30.615767 28289 raft_consensus.cc:2368] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:19:30.612828 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:30.615880 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:30.607769  1999 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:19:30.601943  1987 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:30.581539  1995 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.632062 28404 leader_election.cc:400] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
W20260709 12:19:30.636729 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59157.7us
I20260709 12:19:30.643501 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.651295 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:30.628542  1987 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:30.633334  1980 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:30.659946 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:19:30.660569 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:30.662680 28284 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:30.638793  1808 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:30.682551 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:30.683522 28288 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:30.684003 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:30.657483  2051 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:30.686234 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:30.669322  2000 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:30.696568 28404 leader_election.cc:400] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:30.703047 28404 leader_election.cc:400] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:30.614852  2054 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.659974  1655 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:19:30.708405 28289 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:19:30.676970  2023 consensus_queue.cc:1059] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 12:19:30.713588 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 53750.9us
I20260709 12:19:30.696676  1686 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:30.716089 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:30.732033 28288 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:30.737123  1995 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:30.715565  2084 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:30.758888  1686 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 24 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:30.760241  1686 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.738991  2084 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.749964  1907 raft_consensus.cc:3055] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:30.729676  1980 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:19:30.725972  2000 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:19:30.710194  1987 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:30.790580 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 74621.5us
I20260709 12:19:30.762501  2092 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:19:30.785228  2073 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:19:30.782847  1907 raft_consensus.cc:740] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:19:30.799104 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 84646.4us
I20260709 12:19:30.782691  1903 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:30.767472  1368 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:19:30.830397  1903 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.830924  1907 consensus_queue.cc:260] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.832408  1907 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:30.833009  1903 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:30.835240 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:30.835880  2081 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:19:30.836879 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.836673  2081 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:30.858275  2075 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
W20260709 12:19:30.860337  2091 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:19:30.862043  2082 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:19:30.870770  1845 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.878006  2027 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.878654  2027 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.909341 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:30.910241 28366 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:30.797870  2047 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:19:30.923659 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 66310us
I20260709 12:19:30.924736 28287 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:19:30.943876 28288 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:30.805342  1987 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.947883  2047 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:30.952126 28984 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:19:30.935468 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:30.803777  2037 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.871405  1845 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:30.898381  2040 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:30.923558  2092 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.996363 28985 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:30.975437  1999 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:31.005216 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 281557us
I20260709 12:19:30.983955  2040 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:30.988633  2065 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.008435 28404 leader_election.cc:400] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:19:31.013561 28404 leader_election.cc:403] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:31.014410 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 80388.9us
I20260709 12:19:31.024252  2083 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:31.024863  2083 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.025231  2083 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:31.030309  2027 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:31.031399  2096 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 12:19:31.033282 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.034134  2040 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.036341  2040 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:31.037582  2047 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:30.802189  2034 consensus_queue.cc:1175] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a3de34b6f4cf4606a61944db6e2739c8" responder_term: 19 status { last_received { term: 17 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:31.040212  2081 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.020679  1903 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:19:31.042606  2037 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.035082  1845 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:31.043985  2084 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.016390  2065 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.047155  2065 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:30.989969  2092 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.051016  2092 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.052644 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:31.022536  2079 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:19:31.072070 28359 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:31.075860 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.022162  1924 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.078433 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.080957 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:31.082010 28366 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:31.085896 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:31.086833 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.087641 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.090184 28362 raft_consensus.cc:2393] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:31.069823  1815 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:31.094264  1686 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.097716 28286 raft_consensus.cc:2410] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:19:31.021667  2024 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:31.103530  2100 log.cc:927] Time spent T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:19:31.104322  2086 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.139s	user 0.000s	sys 0.002s
I20260709 12:19:31.105722  1808 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:31.090035  2094 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:31.107972  2003 consensus_meta.cc:287] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.682s	user 0.006s	sys 0.006s
I20260709 12:19:31.115178 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.132313 28364 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:31.076010  2065 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:19:31.134734 28360 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:31.135807 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:31.136023  2103 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.136756  2084 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:31.136718  2103 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:31.137305 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:19:31.139700  2084 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:31.140340  2056 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:19:31.118726  1686 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.143080  1977 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:31.143990  2079 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:31.144378  2079 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:31.144819  2094 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.145267  2079 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:31.147343  2094 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:31.146777  2079 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:31.152457  2024 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:31.088376  1924 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.156168  1924 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.145968  2003 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:31.157588 28363 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:31.114643  1368 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:19:31.164821 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:31.165323 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:31.164836  2059 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:31.166174  1368 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:31.143733  1686 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.172658 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:31.178632  2059 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.179104  2059 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:31.179466  2059 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.180918  2034 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:31.187135 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:31.189857 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.194098  1631 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:31.200639  2022 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:31.194139 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.192332 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:31.203516 28284 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:31.204561  1655 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:31.205009  1655 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.205353  1655 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:31.208273 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.210702 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.213240 28441 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:31.214468 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.164115  2024 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:31.139672  2103 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:31.217895 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.201304  2022 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.227706  2054 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:31.230525 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.224797 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:31.246192 28363 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:31.247732 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:31.242805 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.258669 28434 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:31.251156 28438 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:31.254251 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:31.258966 28441 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:19:31.263870 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.266279 28286 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:31.267714 28991 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:31.268777 28284 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:31.271169 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.271747 28287 raft_consensus.cc:1813] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:31.272181 28287 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 20: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:31.273320 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.275921 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.277483 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:31.283313 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.225450  2024 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.256947  1980 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.291639 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.284183 28284 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:31.264698  2000 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:31.290238  2061 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:19:31.247386  1924 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:19:31.302718  2097 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:31.303117  2097 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:31.303949  2097 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.305156  2097 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.307255  2024 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:31.307418  2023 consensus_queue.cc:1175] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aea2391841b4888b4e195e5a1f4623b" responder_term: 15 status { last_received { term: 13 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:31.308862 28438 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:31.309326 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.309684 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.314607  2044 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:31.297942 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:31.300936  2022 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:31.324723 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.324920 28286 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:31.325172 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.325546 28362 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:19:31.325405 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:31.257601  2083 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.330952 28365 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:31.329671 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:31.331858 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.332197 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:31.335004 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.339351 28438 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:19:31.340773  1907 consensus_meta.cc:287] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.001s
I20260709 12:19:31.342350 28358 raft_consensus.cc:1813] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Handling vote request from an unknown peer 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:31.343000 28358 raft_consensus.cc:2410] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:19:31.338081 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.339380 28292 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:31.344265 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:31.348393 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:31.346956  1907 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:31.349843  2047 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:31.350306  2047 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.350674  2047 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:31.329356  2091 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:19:31.330008  2024 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:31.323844  1980 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:31.367224  1980 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:31.371628 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:31.372488 28984 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:31.388219 28288 raft_consensus.cc:2368] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:31.392117 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:19:31.394865 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63473us
I20260709 12:19:31.397984 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.399117 28404 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:31.395098 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.361095  1987 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:31.402163 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:31.332485  2059 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.410249 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:31.390829  2024 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:31.424613 28437 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
I20260709 12:19:31.405151  1987 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:31.431967  1987 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.404068  2038 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.205s
I20260709 12:19:31.434856 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:31.400095  1368 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:19:31.435672  2061 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:31.436792  2083 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.438268  2059 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:31.441038  2022 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:31.428314 28439 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:31.410697  1980 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:31.446961  1980 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.447727  2083 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:31.449239  1980 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.449608 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:31.450408  1980 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:31.453294 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:31.411847  2052 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:31.458721 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:31.416628  1903 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:31.463862 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:31.467571 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:31.468729 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:31.474195 28292 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:31.474993 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:31.446614  1656 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:31.475932 28440 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
I20260709 12:19:31.482781 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:31.485363 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:31.488011 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.460868  1980 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:31.465243  2000 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:31.464133  2083 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:31.497275 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 81844.1us
I20260709 12:19:31.499963 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:31.500488  2000 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:31.500128 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:31.489239  1987 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.504304  1987 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:31.515573 28365 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:31.501019  2000 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:31.479522  1656 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 22 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:31.489286  1803 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.436693  2024 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:31.484748  2092 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:31.495616  2028 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:31.531427 28436 pending_rounds.cc:85] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 8
I20260709 12:19:31.533233 28436 pending_rounds.cc:107] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 12:19:31.533527 28436 raft_consensus.cc:2889] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:31.524816  1803 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.547816  1903 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:31.531602  2028 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:31.552647 28404 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:31.549881  1656 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:31.561661  2028 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:19:31.577421  2119 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:31.557765  1803 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.558274  1655 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.586410 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:31.591861 28286 raft_consensus.cc:2393] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:19:31.619194  2118 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:19:31.622102 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:31.563622  2126 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:31.696702 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 57146.4us
W20260709 12:19:31.697687  2086 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:19:31.723627 28991 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:19:31.698607  2126 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:31.738212 28286 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:19:31.691812  2028 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:31.765898 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 76555.1us
I20260709 12:19:31.774348  2065 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:31.781906  2126 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:31.775498  2127 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:19:31.767746  1655 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.831585  2000 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:31.848953 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:31.850157 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:31.850952 28365 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 23: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:19:31.852932  2138 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:19:31.861258  2131 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:19:31.863667  2094 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:31.864385  2094 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.849694 28359 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:31.870604 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:31.871649 28365 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:31.880589 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:31.892288 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:31.893023 28991 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:31.868453  2028 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:31.906644  2052 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:31.912081 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:31.912858 28435 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 17
W20260709 12:19:31.917644 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 165443us
I20260709 12:19:31.834646  2000 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:31.924921  1803 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.926895  2094 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:31.908772  1987 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:31.928246  1987 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:31.930423  1987 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:31.933269 28433 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
I20260709 12:19:31.942973 28359 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:31.973707 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:31.976511 28991 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
W20260709 12:19:31.940963  2047 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:19:31.985524 28984 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:19:31.991444 28360 raft_consensus.cc:2368] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:19:32.000811 28367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39084
I20260709 12:19:31.978832 28365 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:32.093843 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:32.092156 28435 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:31.979156  2130 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.133s
I20260709 12:19:32.111867 28359 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 23: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:32.110216 28367 raft_consensus.cc:606] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Received request to transfer leadership
I20260709 12:19:32.050503 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:19:32.143720  2118 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:19:32.155336 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 104949us
I20260709 12:19:32.003764  2065 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:32.110517  1999 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:32.161091 28367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39084
I20260709 12:19:32.161641 28255 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:32.165776 28367 raft_consensus.cc:606] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Received request to transfer leadership
I20260709 12:19:32.142970  2085 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:32.140618  2047 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.158823  1631 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:32.165236  2065 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:32.178891 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:32.179409 28433 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.169088  1980 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:32.184295  2148 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:32.188735 28363 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:19:32.192935  2148 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.181211  1631 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:32.198899  1631 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:32.200740 28185 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 118) took 16437 ms (client timeout 15000 ms). Trace:
I20260709 12:19:32.172392  2000 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:32.203300  2000 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.178475  1803 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:32.169715  1999 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.214310  1980 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:32.215502  1980 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:32.216411  2147 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.367s	user 0.002s	sys 0.000s
I20260709 12:19:32.217726 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 2856) took 1024 ms. Trace:
I20260709 12:19:32.218153 28366 rpcz_store.cc:276] 0709 12:19:31.192831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.192986 (+   155us) service_pool.cc:224] Handling call
0709 12:19:31.193309 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:31.215831 (+ 22522us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:31.215863 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:31.219731 (+  3868us) log.cc:844] Serialized 26 byte log entry
0709 12:19:31.739646 (+519915us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b18003b75e0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:19:31.766466 (+ 26820us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:31.766515 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:31.766603 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:32.215803 (+449200us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:19:32.217459 (+  1656us) raft_consensus.cc:1685] finished
0709 12:19:32.217486 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:32.217651 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22094,"raft.queue_time_us":23543,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":1627,"spinlock_wait_cycles":18070656,"thread_start_us":26575,"threads_started":1,"wal-append.queue_time_us":620737}
W20260709 12:19:32.226605  2144 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:19:32.229988  1903 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:32.230973  1903 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:32.231321  1903 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:32.232264  1903 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.233278  2047 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:32.234516  2157 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:19:32.237855  2000 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:32.245190  1999 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:32.246649 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:32.248179 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:32.251718 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:32.268642  2132 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:32.269565  2148 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:32.271121 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:32.265357  1803 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:32.273718  1803 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.171511  2085 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.276690  2085 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:32.269361  2132 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:32.233556  1903 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:32.282187 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:32.282277 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:32.286681 28986 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:32.287283  1995 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:32.275985  1803 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:32.291566 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:32.308930  2156 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:19:32.311874  1995 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:32.327369  1803 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:32.313105  1995 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.331292  2003 raft_consensus.cc:993] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:32.339854  2132 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:32.342595 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:32.342900 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:32.346056 28404 leader_election.cc:400] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
W20260709 12:19:32.201262 28185 rpcz_store.cc:269] 0709 12:19:15.763337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:15.764341 (+  1004us) service_pool.cc:224] Handling call
0709 12:19:32.200477 (+16436136us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:28.375017 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:28.375793 (+   776us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:28.375846 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:19:28.375872 (+    26us) tablet.cc:662] Decoding operations
0709 12:19:28.377534 (+  1662us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:28.377636 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:28.377670 (+    34us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:19:28.402952 (+ 25282us) tablet.cc:701] Row locks acquired
0709 12:19:28.403003 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:19:28.403203 (+   200us) write_op.cc:270] Start()
0709 12:19:28.403391 (+   188us) write_op.cc:276] Timestamp: P: 1783599568403120 usec, L: 0
0709 12:19:28.403433 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:19:28.442676 (+ 39243us) log.cc:844] Serialized 7114 byte log entry
0709 12:19:29.041206 (+598530us) op_driver.cc:464] REPLICATION: finished
0709 12:19:29.186196 (+144990us) write_op.cc:301] APPLY: starting
0709 12:19:29.208534 (+ 22338us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:29.245634 (+ 37100us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88432a 00007f85be809f14 00007f85be809ec3 00007f85be809e38 00007f85be809df3 00007f85be8090dc 00007f85be832e16 00007f85be81acbb 00007f85be80539d 00007f85be808d75
0709 12:19:29.496098 (+250464us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:29.496148 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:30.012520 (+516372us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:30.012925 (+   405us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:19:30.012990 (+    65us) write_op.cc:312] APPLY: finished
0709 12:19:30.013484 (+   494us) log.cc:844] Serialized 164 byte log entry
0709 12:19:30.352454 (+338970us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:30.352681 (+   227us) write_op.cc:454] Released schema lock
0709 12:19:30.352767 (+    86us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:30.353287 (+   520us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:30.353316 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3604437,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":130874,"apply.run_cpu_time_us":82804,"apply.run_wall_time_us":1177017,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79331,"num_ops":18,"prepare.queue_time_us":308340,"prepare.run_cpu_time_us":64318,"prepare.run_wall_time_us":222400,"raft.queue_time_us":228178,"raft.run_cpu_time_us":26496,"raft.run_wall_time_us":237306,"replication_time_us":630422,"spinlock_wait_cycles":28204928,"thread_start_us":810363,"threads_started":5,"wal-append.queue_time_us":603122,"wal-append.run_cpu_time_us":37544,"wal-append.run_wall_time_us":2348641}]]}
I20260709 12:19:32.365386 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:32.368029 28364 raft_consensus.cc:2368] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
W20260709 12:19:32.430362 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 50827.8us
I20260709 12:19:32.437585 28433 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:32.444941 28332 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:32.449550 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:32.450811 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:32.452541 28438 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:32.453509 28440 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:19:32.457715 28174 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 6c8442db4dc44c619fd67179aa212063 already in progress [suppressed 2 similar messages]
I20260709 12:19:32.442224  2003 raft_consensus.cc:1081] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:32.451895  2132 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:32.482684 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:32.483016 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:32.486020 28404 leader_election.cc:400] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:32.488476 28404 leader_election.cc:403] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:32.476279  1995 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:32.490446  1995 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.492818  2126 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17.
I20260709 12:19:32.493022  1995 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:32.477101  2132 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:32.497195  2132 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.498574  1907 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:32.499004  1907 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:32.499858  1907 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:32.500872  1907 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:32.497853 28437 raft_consensus.cc:2410] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:19:32.459738  2108 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:19:32.515049 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:32.519761 28984 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:19:32.519986 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:32.533102 28438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:19:32.533711 28438 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:32.534053 28438 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:32.545401 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:32.549328  2023 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.550032  2023 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:32.552284  2023 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:32.550324 28367 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39084
I20260709 12:19:32.553443 28367 raft_consensus.cc:606] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Received request to transfer leadership
I20260709 12:19:32.547485 28284 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.587711 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:32.589963 28358 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:32.595113 28404 leader_election.cc:400] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:19:32.595542 28404 leader_election.cc:403] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:19:32.595613 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:32.618122  2126 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 15. Current term is 16.
I20260709 12:19:32.593768  2158 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:32.620694  2158 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:32.622901  2158 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:32.626430  2065 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.627640 28362 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:32.632203 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:32.633064 28363 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:19:32.642836 28285 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:32.645032 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:32.646206 28288 raft_consensus.cc:2393] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 20: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:32.654778 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:32.656976  2027 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:32.657567  2027 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:32.658613  2027 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:32.659940  1655 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:32.664623 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:32.668010 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 114fa146124747358c997ac1008e1b93 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:33064
I20260709 12:19:32.676542 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:32.703531 28284 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:32.702174 28438 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:32.720588 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:32.721521 28991 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:32.734257  2083 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:32.736434  2083 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:32.736853  2083 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:32.745031 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:32.748405 28358 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:32.751849 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:32.702247  2165 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.760640 28257 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:32.758201 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:19:32.768694 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.776696 28438 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:32.776899 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:32.780961 28362 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:32.782817 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:32.781334 28362 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:32.759357  2092 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:32.784265 28991 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:32.787241 28361 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:32.789929 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:32.765467  1995 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:32.765448  2126 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.770467  2153 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:32.772396  2024 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:32.806092 28991 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:19:32.737691  2083 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.802621 28362 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:32.788755  2092 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.767234  2165 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.819319 28291 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:32.820144 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:32.820749 28288 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:32.780402  2050 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:19:32.827941 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:32.828577 28285 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:19:32.829639 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:32.817444  2092 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:32.798377  2126 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.826383  1987 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:32.832445  2083 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:32.873989 28288 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
W20260709 12:19:32.878356 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 88492.5us
I20260709 12:19:32.880818 28190 catalog_manager.cc:5697] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 6 to 12, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 12 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:32.883718 28190 catalog_manager.cc:5697] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 7 to 12, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 12 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:32.887329 28360 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:19:32.902657  2175 log.cc:927] Time spent T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:19:32.915851 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:32.916074  1808 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:32.916782  1808 consensus_queue.cc:1243] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer a3de34b6f4cf4606a61944db6e2739c8 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:19:32.924117 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 128242us
I20260709 12:19:32.925388 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:32.939517 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:32.917495  1984 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.942929 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:32.948642 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:32.950387 28330 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:32.950103 28190 catalog_manager.cc:5697] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 7 to 12, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 12 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:32.947887  2022 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:32.958077 28190 catalog_manager.cc:5697] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:32.949059  2173 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:32.959980  2182 tablet_replica.cc:333] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:19:32.919184  2050 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:19:32.952209  2130 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:32.970399 28367 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:32.960847  2153 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:32.954880  2037 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:32.960039  2134 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:32.975500  2176 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 12:19:32.978459 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.964123  2173 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:32.983093  2165 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:32.984102  2181 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:19:32.983803  2173 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:32.985215  2173 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:32.980517 28190 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 8 to 14, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 14 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:32.953401  1984 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:32.961266  2158 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 12:19:32.974725  2037 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:19:33.016784 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:33.019629 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:32.994030  2165 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.994465  2126 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:33.050119 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 94795.1us
I20260709 12:19:33.001492  2022 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:33.050791 28438 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:33.017777  2037 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:33.065644 28190 catalog_manager.cc:5697] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 2 to 10, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 10 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:33.079679 28190 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:33.050426  2165 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.082763  2062 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.096997 28990 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:33.101456  2022 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:33.106410  1984 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:33.107228  2182 raft_consensus.cc:2243] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:33.111987  2165 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:33.118283  2065 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:33.135946 28190 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:33.098343  2062 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.143307 28190 catalog_manager.cc:5697] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 1 to 14. New cstate: current_term: 14 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:33.145787  2182 raft_consensus.cc:2272] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:33.145283 28190 catalog_manager.cc:5697] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:33.147933 28362 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.147553 28190 catalog_manager.cc:5697] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:33.149163  2182 ts_tablet_manager.cc:1916] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:33.149721 28190 catalog_manager.cc:5697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 3 to 9, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 9 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:33.153632  1999 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:33.159794 28439 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:19:33.160470 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:33.176652 28433 raft_consensus.cc:2410] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:19:33.161170 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:33.162583  2134 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.162997  2065 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.180907  1999 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.181066  2065 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.181546 28284 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:33.183202 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:33.183564 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:33.183885 28360 raft_consensus.cc:2410] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:19:33.185878  1999 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:33.186739 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:33.188572 28361 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:33.144475  2037 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:33.169685 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:33.173283 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.190405 28366 raft_consensus.cc:2410] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:19:33.173782 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:33.177461 28436 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:19:33.164431 28362 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:19:33.194285 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:33.198092  2062 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:33.182538  2134 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.200887  2092 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:33.201893 28366 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:33.202600  2167 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:33.188200  2027 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:19:33.208485 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:33.209244  1655 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.211041 28436 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:33.211575  1655 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:33.212905 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:33.213600 28433 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:33.214893 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:33.215453 28433 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 23.
I20260709 12:19:33.216328 28436 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:33.220161 28290 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:19:33.200515  2134 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:33.229897 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:33.229954 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:33.226969 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:33.233481 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:33.234570  1803 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:33.236793 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:33.235025  1803 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.241499  1803 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.244347  2062 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:33.235066  2083 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:19:33.239565  2024 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:33.251277  2024 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.251658  2024 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.237653 28286 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:33.261276 28362 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:19:33.264055 28984 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:19:33.276327 28362 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "014b394597e7430489bd093d8616dcad"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:33.276983 28362 raft_consensus.cc:606] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:33.254767  2134 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:33.283111  2134 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.239779  2065 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:33.283425 28441 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:19:33.283926 28441 pending_rounds.cc:85] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 9
I20260709 12:19:33.284157 28441 pending_rounds.cc:107] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:19:33.284407 28441 raft_consensus.cc:2889] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:33.257424  1999 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:33.286161 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:33.287767  1977 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:33.288177  1977 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.288533  1977 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.282498 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:33.279984 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:33.283552  2134 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:33.306357 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:33.308962 28984 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.305447  2092 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:33.314963 28991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1410767867a5470e81621556e44c6560"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:19:33.315634 28991 raft_consensus.cc:606] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:33.320405 28238 auto_rebalancer.cc:893] Removing replace marker failed: Replica 6634fa3fbd594726a5ec3c1686900284 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:19:33.320967 28238 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 6c8442db4dc44c619fd67179aa212063 already in progress
W20260709 12:19:33.321272 28238 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 6c8442db4dc44c619fd67179aa212063 already in progress
I20260709 12:19:33.334975  2024 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.336593  1803 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.347994 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:33.348762 28432 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:33.352139 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:33.352319 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:33.353335 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:33.357832 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:33.358182 28291 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:19:33.354668  2132 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:33.348301 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:33.358491 28289 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.371022 28288 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:33.278918  2050 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 12:19:33.365006 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:33.386623 28984 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:19:33.388777 28330 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:33.397761 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:33.406275 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:19:33.375891  2065 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:33.416560 28289 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:19:33.422833 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:33.423038  2065 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:33.423508 28364 raft_consensus.cc:606] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:33.423887 28364 raft_consensus.cc:614] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:33.424069  2065 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.406734  2023 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:33.431461  2023 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.431859  2023 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.406962  1999 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:19:33.391680  2167 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:33.435153  2167 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:33.435526  2167 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:33.444443  1977 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.445696 28439 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 21: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:33.446561  2037 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:33.395282  2058 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:19:33.451963  1903 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.395344  1907 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:33.453392  1907 raft_consensus.cc:697] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:33.454293  1907 consensus_queue.cc:237] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:33.470772 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:33.475337 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:33.476151 28286 raft_consensus.cc:2368] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 20. Current term is 21.
W20260709 12:19:33.476625  2201 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:19:33.512372 28954 leader_election.cc:400] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 20. Current term is 21.
I20260709 12:19:33.512792 28954 leader_election.cc:403] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 12:19:33.520849 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 109360us
I20260709 12:19:33.521891 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:33.537289 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:33.538646 28364 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:33.539673  1803 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:33.546469 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:33.547369 28285 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:19:33.528190  2161 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 20. Current term is 21.
I20260709 12:19:33.551777 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:33.553558 28438 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:33.585434  2207 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:19:33.573493  2024 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:33.591368  2134 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.598101 28438 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:33.599472 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:33.585474  1977 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:33.608250 28358 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 23. Current term is 24. Ops: []
I20260709 12:19:33.608855 28359 raft_consensus.cc:606] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:33.609247 28359 raft_consensus.cc:614] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:33.628245  1803 raft_consensus.cc:993] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
W20260709 12:19:33.631194  2210 log.cc:927] Time spent T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.052s	user 0.002s	sys 0.000s
I20260709 12:19:33.636857  2023 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.609977  2062 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:33.645905  2062 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:33.646880  2062 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.647254  1803 raft_consensus.cc:1081] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:33.659253 28361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4036b67205e41c597049a10a71e9467"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:33.660061 28361 raft_consensus.cc:606] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:33.660571 28361 raft_consensus.cc:614] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:33.665647 28436 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "014b394597e7430489bd093d8616dcad"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:19:33.666308 28436 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:33.666747 28436 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:33.672067 28366 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:33.673753 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:33.678738 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:33.679719 28438 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:33.686851 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:33.687521 28286 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.690121 28285 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:33.699673  2023 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:33.701576 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:33.702457 28439 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 21: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:33.708258 28358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:33.708976 28358 raft_consensus.cc:606] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:33.709448 28358 raft_consensus.cc:614] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:33.710678 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:19:33.712690 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:33.713634 28361 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 21: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:33.724293  2161 raft_consensus.cc:993] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:33.724874  2161 raft_consensus.cc:1081] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:33.726452  2167 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.728601  2167 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:33.729200 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 55865.9us
I20260709 12:19:33.730835 28436 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.740082 28285 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1410767867a5470e81621556e44c6560"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:39592
I20260709 12:19:33.741452 28285 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:33.742029 28285 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.745002 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:33.749341 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:33.755884 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49532.3us
I20260709 12:19:33.768368 28436 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:33.769212  2167 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.772054 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:33.772324 28286 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:19:33.772905 28358 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:33.780095 28285 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:33.771097  1999 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:19:33.782632 28285 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:33.788041 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:33.788808 28358 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.787321 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:33.790867 28990 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:33.791179 28990 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:33.801182 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:33.802055 28361 raft_consensus.cc:3055] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:33.802364 28361 raft_consensus.cc:740] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:33.803617 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:33.798520 28990 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:33.803107 28361 consensus_queue.cc:260] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.806593 28361 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:33.801266  2193 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:19:33.809825  2192 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:33.810427 28358 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:33.812282  2182 ts_tablet_manager.cc:1929] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.10
I20260709 12:19:33.812803  2182 log.cc:1199] T 114fa146124747358c997ac1008e1b93 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/114fa146124747358c997ac1008e1b93
I20260709 12:19:33.816545  2182 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33064 (request call id 0) took 1149 ms. Trace:
I20260709 12:19:33.817308 28436 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:33.817827 28361 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:33.815029 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:33.822171  2084 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:33.822695  2084 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:33.823525  2084 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:33.832491 28990 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.836352 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:33.838563 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:33.839706 28286 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 21: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:19:33.858466 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:19:33.863237  2215 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
I20260709 12:19:33.864359  2193 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:33.865912  2134 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:33.814188  2062 consensus_queue.cc:1175] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a3de34b6f4cf4606a61944db6e2739c8" responder_term: 16 status { last_received { term: 16 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 16 index: 9 } }
I20260709 12:19:33.873862  2193 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:33.874842  2193 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.877838 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:33.879505 28404 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:33.888320 28984 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:19:33.897782 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54458.9us
I20260709 12:19:33.898589 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:33.897684 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44591.9us
I20260709 12:19:33.891757  2209 raft_consensus.cc:2804] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:33.816960  2182 rpcz_store.cc:276] 0709 12:19:32.667356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.667642 (+   286us) service_pool.cc:224] Handling call
0709 12:19:32.959557 (+291915us) ts_tablet_manager.cc:1151] Deleting tablet 114fa146124747358c997ac1008e1b93
0709 12:19:32.959662 (+   105us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:19:33.544460 (+584798us) tablet_metadata.cc:665] Metadata flushed
0709 12:19:33.812129 (+267669us) tablet_metadata.cc:665] Metadata flushed
0709 12:19:33.816436 (+  4307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48988,"tablet-delete.queue_time_us":290647,"thread_start_us":201056,"threads_started":1}
I20260709 12:19:33.910024 28990 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:19:33.900460  1987 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:19:33.918231 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:33.918804 28288 raft_consensus.cc:3060] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:33.921936  2218 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:19:33.908128  2209 raft_consensus.cc:697] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:33.904335  2203 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:33.924974 28361 raft_consensus.cc:1240] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:19:33.931257 28288 raft_consensus.cc:2468] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:33.933559 28330 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:33.938230 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:33.942726 28288 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:33.931564  2203 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:33.944696  2134 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:19:33.949561  2209 consensus_queue.cc:237] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.936509 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 114fa146124747358c997ac1008e1b93 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:33.940362  2023 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:33.940182  2027 consensus_queue.cc:1059] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:19:33.967271  2058 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:19:33.962411  2062 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:19:33.993181  2203 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:33.978261  2058 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:33.999573 28984 raft_consensus.cc:1275] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:33.999418  2058 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:34.000929  2058 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:34.002456 28284 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:34.007046  2062 consensus_queue.cc:1048] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:34.008749  2062 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:34.119151 28438 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:34.166991  2216 log.cc:927] Time spent T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.058s	user 0.001s	sys 0.005s
I20260709 12:19:34.173032 28288 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:34.175124  2062 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
W20260709 12:19:34.179265  2219 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.068s	user 0.000s	sys 0.006s
I20260709 12:19:34.191998 28438 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:34.192657 28439 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:34.204574  2065 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:34.206110  2065 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:19:34.233326 28190 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 220) took 44003 ms (client timeout 15000 ms). Trace:
W20260709 12:19:34.242813 28190 rpcz_store.cc:269] 0709 12:18:50.229612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:50.238536 (+  8924us) service_pool.cc:224] Handling call
0709 12:18:50.855274 (+616738us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b5400002490. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb7630b2 00007f85c46dc759 00007f85c4674a76 00007f85c4645d58 00007f85c474404e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc
0709 12:19:34.233206 (+43377932us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:33.577524 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:33.578173 (+   649us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:33.578211 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:19:33.578230 (+    19us) tablet.cc:662] Decoding operations
0709 12:19:33.579271 (+  1041us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:33.579341 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:33.579366 (+    25us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:19:33.580304 (+   938us) tablet.cc:701] Row locks acquired
0709 12:19:33.580318 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:19:33.580487 (+   169us) write_op.cc:270] Start()
0709 12:19:33.580594 (+   107us) write_op.cc:276] Timestamp: P: 1783599573580437 usec, L: 0
0709 12:19:33.580609 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 12:19:33.618297 (+ 37688us) log.cc:844] Serialized 4769 byte log entry
0709 12:19:33.946536 (+328239us) op_driver.cc:464] REPLICATION: finished
0709 12:19:33.976435 (+ 29899us) write_op.cc:301] APPLY: starting
0709 12:19:33.976530 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:34.046192 (+ 69662us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:34.046289 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:34.110097 (+ 63808us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:34.110344 (+   247us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 12:19:34.110384 (+    40us) write_op.cc:312] APPLY: finished
0709 12:19:34.110666 (+   282us) log.cc:844] Serialized 113 byte log entry
0709 12:19:34.110838 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:34.110995 (+   157us) write_op.cc:454] Released schema lock
0709 12:19:34.111044 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:34.120047 (+  9003us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:34.120081 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":52,"spinlock_wait_cycles":70127488,"child_traces":[["op",{"apply.queue_time_us":28774,"apply.run_cpu_time_us":16485,"apply.run_wall_time_us":140845,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54263,"num_ops":12,"prepare.queue_time_us":201073,"prepare.run_cpu_time_us":21841,"prepare.run_wall_time_us":161436,"raft.queue_time_us":51705,"raft.run_cpu_time_us":14011,"raft.run_wall_time_us":26866,"replication_time_us":362767,"thread_start_us":255145,"threads_started":4,"wal-append.queue_time_us":164595}]]}
I20260709 12:19:34.241537 28182 catalog_manager.cc:5697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.264880 28182 catalog_manager.cc:5697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 7 to 14, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 14 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.272204 28182 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 10 to 16, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 16 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:34.282058 28182 catalog_manager.cc:5697] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 10 to 15, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 15 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.299021 28182 catalog_manager.cc:5697] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 8 to 15, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 15 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.300766 28182 catalog_manager.cc:5697] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 12 to 17, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 17 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.302510 28182 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 12 to 13, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 13 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.306445 28182 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 11 to 13, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 13 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.308912 28182 catalog_manager.cc:5697] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.310415 28182 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.324086 28359 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:34.325978  2203 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:34.325809 28182 catalog_manager.cc:5697] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.335012 28182 catalog_manager.cc:5697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 12 to 15, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 15 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.338948 28182 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 11 to 12, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 12 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:34.342338 28182 catalog_manager.cc:5697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.350687 28182 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 9 to 18, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.358726 28361 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:34.361879  2192 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:34.367244 28984 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:34.370069  2084 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:34.384214 28182 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.389874 28182 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.395074 28182 catalog_manager.cc:5697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 10 to 14, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 14 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.409296 28182 catalog_manager.cc:5697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 12 to 15, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 15 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.411288 28182 catalog_manager.cc:5697] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 10 to 13, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 13 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.413761 28182 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:34.420596 28288 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:34.423563  2173 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:34.494094 28439 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:34.510887  2084 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:34.511226 28361 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:34.521412  2192 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:34.524073 28359 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:19:34.525825  2084 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:34.745723 28291 raft_consensus.cc:1275] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:19:34.763332 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 84070.2us
I20260709 12:19:34.775532  2188 consensus_queue.cc:1048] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
W20260709 12:19:34.799526 28238 auto_rebalancer.cc:629] Failed to schedule move for tablet 610cd41ac66c4074be8e2aa7b22d5e6c: Illegal state: Replica 6634fa3fbd594726a5ec3c1686900284 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:34.823009 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:34.848176 28289 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:34.916464 28438 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:34.918888  2261 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:19:34.927527 28255 consensus_peers.cc:597] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bea7beef1d4c49159d75e5f61eaaeb5a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:34.938282 28366 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:34.951563  2261 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:34.988240  2261 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:34.990758 28366 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:19:35.015372 28238 auto_rebalancer.cc:629] Failed to schedule move for tablet 916dfa76a5254f63bfc746dbe9c5fde0: Illegal state: Replica 6634fa3fbd594726a5ec3c1686900284 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
W20260709 12:19:35.021500 28238 auto_rebalancer.cc:629] Failed to schedule move for tablet 97c4af01d02243b2b0727baa5bbacf44: Illegal state: Replica a3de34b6f4cf4606a61944db6e2739c8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.058775 28439 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:35.157166 28181 catalog_manager.cc:5697] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.159566 28181 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 22, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 22 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.161239 28181 catalog_manager.cc:5697] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 9 to 18, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 18 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.164492 28181 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:35.178167 28181 catalog_manager.cc:5697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 12 to 14, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 14 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.189195 28181 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.188417 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 114fa146124747358c997ac1008e1b93 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:33064
I20260709 12:19:35.192328 28366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:35.202405 28366 raft_consensus.cc:606] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:35.203531 28181 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 13 to 14, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 14 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.211555 28984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "f526369c0d0743b192be6e205540a059"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:37070
I20260709 12:19:35.212527 28984 raft_consensus.cc:606] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:35.205426 28181 catalog_manager.cc:5697] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 12 to 17, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 17 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.223409 28358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:35.224094 28358 raft_consensus.cc:606] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:35.223320 28181 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:35.224488 28358 raft_consensus.cc:614] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:35.227859 28358 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:35.228497 28358 raft_consensus.cc:606] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:35.228881 28358 raft_consensus.cc:614] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:35.231603 28366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:35.232317 28366 raft_consensus.cc:606] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:35.237680  2065 raft_consensus.cc:993] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:35.241837  2065 raft_consensus.cc:1081] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:35.241128 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:35.244959 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:35.246008 28181 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 11 to 16, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.248054 28181 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:35.250186 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:35.250111 28181 catalog_manager.cc:5697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 13 to 19, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 19 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.256157 28181 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 12 to 14, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 14 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:35.258414 28181 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:35.260694 28439 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:19:35.260260 28181 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 13 to 17, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 17 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.261372 28439 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:35.261871 28439 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:35.188040 28182 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 241) took 16309 ms (client timeout 15000 ms). Trace:
W20260709 12:19:35.270291 28182 rpcz_store.cc:269] 0709 12:19:18.878671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.879812 (+  1141us) service_pool.cc:224] Handling call
0709 12:19:35.187907 (+16308095us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:34.616956 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:34.617593 (+   637us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:34.617648 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:19:34.617669 (+    21us) tablet.cc:662] Decoding operations
0709 12:19:34.619394 (+  1725us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:34.619499 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:34.619531 (+    32us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:19:34.621098 (+  1567us) tablet.cc:701] Row locks acquired
0709 12:19:34.621119 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:19:34.621286 (+   167us) write_op.cc:270] Start()
0709 12:19:34.621416 (+   130us) write_op.cc:276] Timestamp: P: 1783599574621229 usec, L: 0
0709 12:19:34.621437 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:19:34.630596 (+  9159us) log.cc:844] Serialized 8229 byte log entry
0709 12:19:34.774071 (+143475us) op_driver.cc:464] REPLICATION: finished
0709 12:19:34.827307 (+ 53236us) write_op.cc:301] APPLY: starting
0709 12:19:34.827404 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:34.855953 (+ 28549us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:34.856112 (+   159us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:34.871263 (+ 15151us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:34.873840 (+  2577us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 12:19:34.873889 (+    49us) write_op.cc:312] APPLY: finished
0709 12:19:34.874339 (+   450us) log.cc:844] Serialized 188 byte log entry
0709 12:19:34.874531 (+   192us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:34.874768 (+   237us) write_op.cc:454] Released schema lock
0709 12:19:34.874829 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:34.875662 (+   833us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:34.875694 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":48072,"apply.run_cpu_time_us":26134,"apply.run_wall_time_us":48066,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91865,"num_ops":21,"prepare.queue_time_us":134260,"prepare.run_cpu_time_us":6101,"prepare.run_wall_time_us":18010,"replication_time_us":152457,"thread_start_us":134798,"threads_started":2}]]}
I20260709 12:19:35.289867 28439 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:35.299741  2023 raft_consensus.cc:993] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:35.300100 28439 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:35.289875 28181 catalog_manager.cc:5697] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 11 to 16, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.301180  2251 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:35.302040 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:35.302413  2251 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:35.300352  2023 raft_consensus.cc:1081] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:35.306731 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:35.307083 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:35.307752 28358 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:35.308084 28358 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:35.308697 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:35.309399 28286 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 24.
I20260709 12:19:35.310171 28358 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:35.302683 28984 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:35.306751 28438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:19:35.315333 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:35.316136 28358 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:35.318321 28366 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 24: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:35.302423 28181 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 14 to 16, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:35.315526 28438 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:35.321364 28438 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:35.304945  2251 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:35.339732 28984 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:19:35.334703 28438 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:35.356997 28358 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 10.
I20260709 12:19:35.346372  2188 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:35.374810 28438 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:35.369840  2188 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:19:35.393473 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:35.394227 28286 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:35.395819 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:35.396507 28358 raft_consensus.cc:3055] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:35.396840 28358 raft_consensus.cc:740] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:35.397639 28358 consensus_queue.cc:260] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:35.399019 28358 raft_consensus.cc:3060] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:35.411681 28358 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:35.413555 28286 raft_consensus.cc:2468] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:35.414628 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:35.397886  2188 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:35.445545  2265 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:35.446241  2265 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:35.447232  2265 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:35.463716 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 114fa146124747358c997ac1008e1b93 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
W20260709 12:19:35.466585 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 52069.6us
I20260709 12:19:35.494422 28404 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:35.493634 28181 catalog_manager.cc:5697] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 18 to 19, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 19 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:35.502074  2266 raft_consensus.cc:2804] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:35.502547 28181 catalog_manager.cc:5697] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 7 to 23, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 23 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:35.504096  2266 raft_consensus.cc:697] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:35.505177  2266 consensus_queue.cc:237] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:35.518096 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45094.7us
I20260709 12:19:35.533453 28181 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:35.540313  2188 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:35.574409  2188 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:35.580757 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:35.582837 28366 raft_consensus.cc:2393] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 24: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:19:35.589777 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:35.601295 28291 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 24.
I20260709 12:19:35.615558  2065 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.619434  2065 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:35.621906 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 74613.4us
I20260709 12:19:35.623392 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:35.624336 28438 raft_consensus.cc:2410] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:19:35.625790  2065 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:35.625865 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:35.631518  2251 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:35.632091  2251 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:35.632987  2251 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:35.632109 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 101505us
I20260709 12:19:35.651481 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:19:35.652489 28286 raft_consensus.cc:2410] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:19:35.653828 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:35.655582  2023 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:19:35.709838  2277 ts_tablet_manager.cc:933] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:19:35.718094  2277 tablet_copy_client.cc:323] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:19:35.720150 28302 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet bea7beef1d4c49159d75e5f61eaaeb5a from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:19:35.720764 28302 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet bea7beef1d4c49159d75e5f61eaaeb5a from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-bea7beef1d4c49159d75e5f61eaaeb5a
I20260709 12:19:35.739570  2255 raft_consensus.cc:993] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:35.740267  2255 raft_consensus.cc:1081] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:35.744725 28439 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:53280
I20260709 12:19:35.745299 28439 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:35.745662 28439 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.761922 28439 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:35.763695 28302 tablet_copy_source_session.cc:215] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:35.764703 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:35.765543 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:35.766109 28361 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.766186 28984 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:35.766649 28984 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:19:35.767501 28984 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:35.768667 28984 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.772699  2277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea7beef1d4c49159d75e5f61eaaeb5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:35.774111 28361 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:35.775193 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:35.776103  2266 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:35.776573  2266 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:35.777434  2266 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:35.832583 28366 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 12. (index mismatch)
I20260709 12:19:35.834709  2188 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:35.843715 28439 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:35.853835 28984 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:35.890547 28185 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:35.919495 28185 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:35.921466 28185 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:35.929786 28185 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:35.932679 28185 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 21 to 24, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 24 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:35.935082 28984 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 12. (index mismatch)
I20260709 12:19:35.943370  2188 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:35.959561 28185 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 12 to 20, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 20 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:35.963560 28185 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 13 to 14, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 14 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:35.967425 28358 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:19:35.955202  2277 tablet_copy_client.cc:806] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:35.975485  2277 tablet_copy_client.cc:670] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:35.979789  2277 tablet_copy_client.cc:538] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:35.990651  2250 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:36.005580  2277 tablet_bootstrap.cc:492] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:19:36.026813 28185 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 11 to 19, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 19 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.036618 28291 raft_consensus.cc:1275] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:19:36.042004  2188 consensus_queue.cc:1048] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:36.062738 28181 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 222) took 7145 ms. Trace:
W20260709 12:19:36.133531  2291 log.cc:927] Time spent T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:19:36.153926 28181 rpcz_store.cc:276] 0709 12:19:28.917372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.949575 (+ 32203us) service_pool.cc:224] Handling call
0709 12:19:36.062635 (+7113060us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:35.718062 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:35.718660 (+   598us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:35.718695 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:19:35.718713 (+    18us) tablet.cc:662] Decoding operations
0709 12:19:35.747507 (+ 28794us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:35.747604 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:35.747629 (+    25us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:19:35.749056 (+  1427us) tablet.cc:701] Row locks acquired
0709 12:19:35.751057 (+  2001us) write_op.cc:260] PREPARE: finished
0709 12:19:35.751290 (+   233us) write_op.cc:270] Start()
0709 12:19:35.751424 (+   134us) write_op.cc:276] Timestamp: P: 1783599575751183 usec, L: 0
0709 12:19:35.751443 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:19:35.752210 (+   767us) log.cc:844] Serialized 7940 byte log entry
0709 12:19:35.817759 (+ 65549us) op_driver.cc:464] REPLICATION: finished
0709 12:19:35.830496 (+ 12737us) write_op.cc:301] APPLY: starting
0709 12:19:35.830593 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:35.848439 (+ 17846us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:35.848553 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:35.865096 (+ 16543us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:35.865425 (+   329us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 12:19:35.865480 (+    55us) write_op.cc:312] APPLY: finished
0709 12:19:35.865911 (+   431us) log.cc:844] Serialized 180 byte log entry
0709 12:19:35.866104 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:35.866316 (+   212us) write_op.cc:454] Released schema lock
0709 12:19:35.866376 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:35.892469 (+ 26093us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:35.892791 (+   322us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":69358,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":12467,"apply.run_cpu_time_us":23319,"apply.run_wall_time_us":55885,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87687,"num_ops":20,"prepare.queue_time_us":87759,"prepare.run_cpu_time_us":5552,"prepare.run_wall_time_us":41103,"replication_time_us":66137,"thread_start_us":57379,"threads_started":2}]]}
I20260709 12:19:36.191179 28990 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:36.200807 28185 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 119) took 3398 ms. Trace:
I20260709 12:19:36.201530 28185 rpcz_store.cc:276] 0709 12:19:32.802265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.803546 (+  1281us) service_pool.cc:224] Handling call
0709 12:19:36.200658 (+3397112us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:36.045079 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:36.045605 (+   526us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:36.045655 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:19:36.045672 (+    17us) tablet.cc:662] Decoding operations
0709 12:19:36.046603 (+   931us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:36.046682 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:36.046712 (+    30us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:19:36.047386 (+   674us) tablet.cc:701] Row locks acquired
0709 12:19:36.047399 (+    13us) write_op.cc:260] PREPARE: finished
0709 12:19:36.047550 (+   151us) write_op.cc:270] Start()
0709 12:19:36.047671 (+   121us) write_op.cc:276] Timestamp: P: 1783599576047505 usec, L: 0
0709 12:19:36.047689 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:19:36.048352 (+   663us) log.cc:844] Serialized 3194 byte log entry
0709 12:19:36.054522 (+  6170us) op_driver.cc:464] REPLICATION: finished
0709 12:19:36.054907 (+   385us) write_op.cc:301] APPLY: starting
0709 12:19:36.055009 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:36.059066 (+  4057us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:36.059160 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:36.063808 (+  4648us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:36.064038 (+   230us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 12:19:36.064085 (+    47us) write_op.cc:312] APPLY: finished
0709 12:19:36.064343 (+   258us) log.cc:844] Serialized 81 byte log entry
0709 12:19:36.069907 (+  5564us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:36.070052 (+   145us) write_op.cc:454] Released schema lock
0709 12:19:36.070104 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:36.070484 (+   380us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:36.070516 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":75340,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":10203,"apply.run_wall_time_us":15555,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33424,"num_ops":8,"prepare.queue_time_us":108,"prepare.run_cpu_time_us":3613,"prepare.run_wall_time_us":5880,"replication_time_us":6656}]]}
I20260709 12:19:36.207748 28984 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:36.212821  2265 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:36.232811 28292 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:19:36.239774  2266 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:36.240454 28361 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:19:36.241694 28432 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:36.248581 28361 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:36.252080  2278 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:36.274523  2188 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:19:36.278098 28184 catalog_manager.cc:5697] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 8 to 14, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 14 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:36.260281  2276 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:19:36.318199 28366 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:36.375047 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:36.383612  2250 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 12:19:36.477253 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 134060us
I20260709 12:19:36.479223  2301 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index 7 to 12, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.490803 28365 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 7 to 12, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.518906 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3ee4696ba90a447fa07c4d09407217f6 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:36.516948 28439 raft_consensus.cc:2955] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 7 to 12, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.624830  2277 tablet_bootstrap.cc:492] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:36.625991  2277 tablet_bootstrap.cc:492] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:36.626780 28185 catalog_manager.cc:5697] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.629078 28185 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 20 to 21, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 21 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.642832 28185 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 9 to 10, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 10 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.644642 28185 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 16 to 17, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 17 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.646231 28185 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 12 to 13, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 13 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.649096 28185 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 16 to 18, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 18 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.626859  2277 ts_tablet_manager.cc:1403] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.622s	user 0.057s	sys 0.004s
I20260709 12:19:36.719744 28185 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 16 to 17, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:36.705420  2277 raft_consensus.cc:359] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:36.731400  2277 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:36.732264  2277 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:36.735484  2277 ts_tablet_manager.cc:1434] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:19:36.737468 28302 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-bea7beef1d4c49159d75e5f61eaaeb5a received from {username='slave'} at 127.0.0.1:39592
I20260709 12:19:36.738170 28302 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-bea7beef1d4c49159d75e5f61eaaeb5a on tablet bea7beef1d4c49159d75e5f61eaaeb5a with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:36.746984 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 114fa146124747358c997ac1008e1b93 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:33064
W20260709 12:19:36.747622 28184 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 240) took 38954 ms (client timeout 15000 ms). Trace:
W20260709 12:19:36.748185 28184 rpcz_store.cc:269] 0709 12:18:57.793015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.793360 (+   345us) service_pool.cc:224] Handling call
0709 12:19:36.747525 (+38954165us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:36.316974 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:36.317701 (+   727us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:36.318028 (+   327us) write_op.cc:435] Acquired schema lock
0709 12:19:36.318070 (+    42us) tablet.cc:662] Decoding operations
0709 12:19:36.318526 (+   456us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:36.318595 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:36.318626 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:19:36.318852 (+   226us) tablet.cc:701] Row locks acquired
0709 12:19:36.318870 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:19:36.319044 (+   174us) write_op.cc:270] Start()
0709 12:19:36.319185 (+   141us) write_op.cc:276] Timestamp: P: 1783599576318995 usec, L: 0
0709 12:19:36.319208 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:19:36.319983 (+   775us) log.cc:844] Serialized 538 byte log entry
0709 12:19:36.326510 (+  6527us) op_driver.cc:464] REPLICATION: finished
0709 12:19:36.327014 (+   504us) write_op.cc:301] APPLY: starting
0709 12:19:36.327114 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:36.328031 (+   917us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:36.328132 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:36.329388 (+  1256us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:36.329570 (+   182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:19:36.329617 (+    47us) write_op.cc:312] APPLY: finished
0709 12:19:36.329878 (+   261us) log.cc:844] Serialized 25 byte log entry
0709 12:19:36.330090 (+   212us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:36.330180 (+    90us) write_op.cc:454] Released schema lock
0709 12:19:36.330223 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:36.342036 (+ 11813us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:36.343208 (+  1172us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10156,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":255,"apply.run_cpu_time_us":3570,"apply.run_wall_time_us":3570,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":651,"prepare.run_cpu_time_us":3403,"prepare.run_wall_time_us":15618,"replication_time_us":7141}]]}
I20260709 12:19:36.799417 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 114fa146124747358c997ac1008e1b93 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:36.840186 28185 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 15 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:36.842288 28185 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 7 to 12, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 18 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:36.873828 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 114fa146124747358c997ac1008e1b93 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:33064
I20260709 12:19:36.875737 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 114fa146124747358c997ac1008e1b93 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
W20260709 12:19:36.885612 28188 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 239) took 58672 ms (client timeout 15000 ms). Trace:
W20260709 12:19:36.894722 28188 rpcz_store.cc:269] 0709 12:18:38.212781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.213924 (+  1143us) service_pool.cc:224] Handling call
0709 12:19:36.885361 (+58671437us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33360,"reported_tablets":52,"spinlock_wait_cycles":43136}
I20260709 12:19:36.915306 28186 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 14 to 17, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:36.920449 28186 catalog_manager.cc:5697] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:36.931049 28186 catalog_manager.cc:5697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 14 to 17, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:36.962596 28186 catalog_manager.cc:5697] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 13 to 14, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 14 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:36.974355 28432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:36.975135 28432 raft_consensus.cc:606] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:37.004546 28432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:37.005473 28432 raft_consensus.cc:606] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:37.033152 28432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:37.034416 28432 raft_consensus.cc:606] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:37.036759 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 58169.5us
I20260709 12:19:37.042076 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:19:37.049866 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:37.051213 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:37.073000  2304 raft_consensus.cc:993] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:37.085292  2302 raft_consensus.cc:1081] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:37.092433 28289 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59792
I20260709 12:19:37.093209 28289 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:37.093641 28289 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:37.102989 28289 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:37.167240 28984 raft_consensus.cc:1217] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.9->[15.10-15.10]   Dedup: 15.10->[]
I20260709 12:19:37.170413 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ee4696ba90a447fa07c4d09407217f6 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:36572
I20260709 12:19:37.176507 28289 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:37.178045 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:37.212945 28432 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:37.213366 28432 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:37.214229 28432 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:37.215246 28432 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:37.189070 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:37.217830 28358 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:37.217854  2319 tablet_replica.cc:333] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:19:37.219415  2319 raft_consensus.cc:2243] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:37.192502 28181 catalog_manager.cc:5697] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 17, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 17 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.228740 28181 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 18, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 18 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.230592 28181 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:37.232765 28181 catalog_manager.cc:5697] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 17 to 20, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 20 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.234190 28181 catalog_manager.cc:5697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 17 to 20, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 20 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.235325 28358 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:19:37.236459  2319 raft_consensus.cc:2272] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:37.235562 28181 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 11 to 12, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 12 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.248826 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:37.249856  2261 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:37.250592  2261 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:37.251600  2261 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:37.262106 28181 catalog_manager.cc:5697] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 14 to 17, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 17 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.229504 28432 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:19:37.272635 28181 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 17 to 20, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 20 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:37.279687 28186 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 227) took 9256 ms. Trace:
I20260709 12:19:37.282049  2319 ts_tablet_manager.cc:1916] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:37.278791 28181 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.298856 28181 catalog_manager.cc:5697] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 13 to 17, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 17 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.280246 28186 rpcz_store.cc:276] 0709 12:19:28.023762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.024314 (+   552us) service_pool.cc:224] Handling call
0709 12:19:37.279505 (+9255191us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:36.993674 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:36.998846 (+  5172us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:36.998892 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:19:36.998907 (+    15us) tablet.cc:662] Decoding operations
0709 12:19:36.999552 (+   645us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:36.999628 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:36.999660 (+    32us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:19:37.000107 (+   447us) tablet.cc:701] Row locks acquired
0709 12:19:37.000127 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:19:37.000306 (+   179us) write_op.cc:270] Start()
0709 12:19:37.000455 (+   149us) write_op.cc:276] Timestamp: P: 1783599577000249 usec, L: 0
0709 12:19:37.000486 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:19:37.001238 (+   752us) log.cc:844] Serialized 1694 byte log entry
0709 12:19:37.013570 (+ 12332us) op_driver.cc:464] REPLICATION: finished
0709 12:19:37.019976 (+  6406us) write_op.cc:301] APPLY: starting
0709 12:19:37.020085 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:37.022505 (+  2420us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:37.022602 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:37.025193 (+  2591us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:37.025416 (+   223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 12:19:37.025469 (+    53us) write_op.cc:312] APPLY: finished
0709 12:19:37.053554 (+ 28085us) log.cc:844] Serialized 49 byte log entry
0709 12:19:37.060146 (+  6592us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:37.078164 (+ 18018us) write_op.cc:454] Released schema lock
0709 12:19:37.083235 (+  5071us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:37.158717 (+ 75482us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:37.168497 (+  9780us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28892,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":1033,"apply.run_cpu_time_us":6984,"apply.run_wall_time_us":67963,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16712,"num_ops":4,"prepare.queue_time_us":3328,"prepare.run_cpu_time_us":3626,"prepare.run_wall_time_us":16238,"replication_time_us":9876}]]}
W20260709 12:19:37.301602 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 109588us
I20260709 12:19:37.312875 28181 catalog_manager.cc:5697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 13 to 16, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 16 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.314716 28181 catalog_manager.cc:5697] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 19, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 19 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.316684 28181 catalog_manager.cc:5697] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 15 to 17, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:37.325682 28181 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 18 to 21, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 21 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.327592 28181 catalog_manager.cc:5697] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 15 to 17, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:37.331409 28181 catalog_manager.cc:5697] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 15 to 17, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:37.338231 28181 catalog_manager.cc:5697] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 14 to 16, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 16 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.355746  2319 ts_tablet_manager.cc:1929] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.11
I20260709 12:19:37.371869  2319 log.cc:1199] T 3ee4696ba90a447fa07c4d09407217f6 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/3ee4696ba90a447fa07c4d09407217f6
I20260709 12:19:37.379364  2304 raft_consensus.cc:993] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:37.378831  2268 raft_consensus.cc:993] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:37.380040  2265 raft_consensus.cc:1081] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:37.380123  2304 raft_consensus.cc:1081] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:37.381582 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 3ee4696ba90a447fa07c4d09407217f6 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:37.382851 28358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:37.383421 28358 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:37.383782 28358 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:37.385990 28362 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:37.386616 28362 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:37.386981 28362 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.412077 28362 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:37.420378 28358 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:37.450232 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:37.450922 28439 raft_consensus.cc:3055] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:37.451718 28439 raft_consensus.cc:740] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:37.462817 28439 consensus_queue.cc:260] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:37.466399 28439 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:37.434710 28362 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:37.438004 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:37.482395 28991 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.442895 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:37.483662 28432 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:37.484026 28432 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:37.484862 28432 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:37.486213 28432 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.490620 28358 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:37.495148 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:37.499462 28991 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:37.500341 28984 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:37.502279 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:37.503239  2023 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:37.503758  2023 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:37.504624  2023 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:37.514425 28439 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:37.518065 28332 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:37.528589 28984 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:37.530304 28432 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:37.520970 28186 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:19:37.538177 28181 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 223) took 1331 ms. Trace:
I20260709 12:19:37.538873 28181 rpcz_store.cc:276] 0709 12:19:36.198423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.198732 (+   309us) service_pool.cc:224] Handling call
0709 12:19:37.529410 (+1330678us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:37.352711 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:37.353238 (+   527us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:37.353272 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:19:37.353290 (+    18us) tablet.cc:662] Decoding operations
0709 12:19:37.354634 (+  1344us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:37.354693 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:37.354721 (+    28us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:19:37.355908 (+  1187us) tablet.cc:701] Row locks acquired
0709 12:19:37.355925 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:19:37.363568 (+  7643us) write_op.cc:270] Start()
0709 12:19:37.363708 (+   140us) write_op.cc:276] Timestamp: P: 1783599577363498 usec, L: 0
0709 12:19:37.363735 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:19:37.364509 (+   774us) log.cc:844] Serialized 6695 byte log entry
0709 12:19:37.372393 (+  7884us) op_driver.cc:464] REPLICATION: finished
0709 12:19:37.373498 (+  1105us) write_op.cc:301] APPLY: starting
0709 12:19:37.373593 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:37.396607 (+ 23014us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:37.396714 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:37.464184 (+ 67470us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:37.464468 (+   284us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 12:19:37.464516 (+    48us) write_op.cc:312] APPLY: finished
0709 12:19:37.464903 (+   387us) log.cc:844] Serialized 156 byte log entry
0709 12:19:37.479011 (+ 14108us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:37.479214 (+   203us) write_op.cc:454] Released schema lock
0709 12:19:37.479274 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:37.504731 (+ 25457us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:37.504767 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":43679,"reported_tablets":33,"spinlock_wait_cycles":1920,"child_traces":[["op",{"apply.queue_time_us":856,"apply.run_cpu_time_us":21748,"apply.run_wall_time_us":118494,"cfile_cache_hit":68,"cfile_cache_hit_bytes":71026,"num_ops":17,"prepare.queue_time_us":129,"prepare.run_cpu_time_us":4743,"prepare.run_wall_time_us":33151,"replication_time_us":8509}]]}
I20260709 12:19:37.520534  2318 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:37.699143  2318 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:37.700706  2318 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:37.861905 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55871.2us
W20260709 12:19:37.865103 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 182462us
I20260709 12:19:37.894203 28181 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 21 to 22, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 22 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:37.902874 28436 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:19:38.039284 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 174791us
W20260709 12:19:38.041816 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 57869.4us
W20260709 12:19:38.049561 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65084.5us
I20260709 12:19:38.057929  2261 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:38.078162  2322 raft_consensus.cc:1064] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:38.081312  2322 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:38.099833 28438 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:38.101624  2065 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:38.125162 28358 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:19:38.143074 28984 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:19:38.150127 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 82860.3us
I20260709 12:19:38.153640  2023 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:19:38.145095  2261 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
W20260709 12:19:38.224524 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51775.4us
I20260709 12:19:38.237089 28991 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:19:38.240439 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 83320.1us
I20260709 12:19:38.285109 28438 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:19:38.358913 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66909.6us
I20260709 12:19:38.345978 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:38.370616  2317 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:19:38.412781 28186 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 228) took 1009 ms. Trace:
W20260709 12:19:38.481137 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 174662us
W20260709 12:19:38.466637 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 167931us
I20260709 12:19:38.413606 28186 rpcz_store.cc:276] 0709 12:19:37.403501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.403664 (+   163us) service_pool.cc:224] Handling call
0709 12:19:37.413612 (+  9948us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b5400002490. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb7630b2 00007f85c46dc759 00007f85c4674a76 00007f85c4645d58 00007f85c474404e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc
0709 12:19:38.412425 (+998813us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:37.542682 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:37.548973 (+  6291us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:37.549022 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:19:37.549043 (+    21us) tablet.cc:662] Decoding operations
0709 12:19:37.661403 (+112360us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:37.661471 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:37.661504 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:19:37.661729 (+   225us) tablet.cc:701] Row locks acquired
0709 12:19:37.661767 (+    38us) write_op.cc:260] PREPARE: finished
0709 12:19:37.661919 (+   152us) write_op.cc:270] Start()
0709 12:19:37.662086 (+   167us) write_op.cc:276] Timestamp: P: 1783599577661872 usec, L: 0
0709 12:19:37.662109 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:19:37.662750 (+   641us) log.cc:844] Serialized 538 byte log entry
0709 12:19:37.664868 (+  2118us) op_driver.cc:464] REPLICATION: finished
0709 12:19:37.665394 (+   526us) write_op.cc:301] APPLY: starting
0709 12:19:37.665476 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:37.666375 (+   899us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:37.666479 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:37.667220 (+   741us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:37.667380 (+   160us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:19:37.667424 (+    44us) write_op.cc:312] APPLY: finished
0709 12:19:37.667608 (+   184us) log.cc:844] Serialized 25 byte log entry
0709 12:19:37.667805 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:37.667896 (+    91us) write_op.cc:454] Released schema lock
0709 12:19:37.667943 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:37.669285 (+  1342us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:37.669316 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17878,"reported_tablets":34,"spinlock_wait_cycles":11231488,"child_traces":[["op",{"apply.queue_time_us":299,"apply.run_cpu_time_us":2962,"apply.run_wall_time_us":2961,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4178,"num_ops":1,"prepare.queue_time_us":5226,"prepare.run_cpu_time_us":14276,"prepare.run_wall_time_us":126577,"replication_time_us":2603}]]}
I20260709 12:19:38.577055 28362 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:19:38.612533 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 247174us
I20260709 12:19:38.628552 28439 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:38.629271 28984 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 15 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:38.642323 28186 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 17 to 18, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:38.704766  2366 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:19:38.710811  2065 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:38.718222  2367 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:38.719195  2367 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:38.782155  2367 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:38.817312  2308 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:38.822317 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:38.837414 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:38.851819  2361 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:38.856676 28257 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:38.857997  2357 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:38.918399 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 55163.1us
I20260709 12:19:39.009085 28181 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 224) took 1298 ms. Trace:
I20260709 12:19:39.009559 28181 rpcz_store.cc:276] 0709 12:19:37.711057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.711229 (+   172us) service_pool.cc:224] Handling call
0709 12:19:38.357403 (+646174us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54001960a8. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c21f1161 00007f85c227b3af 00007f85c227cb11 00007f85c477cfa1 00007f85c477d7ff 00007f85c463ca1b 00007f85c474490e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc
0709 12:19:39.008876 (+651473us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:38.514632 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:38.515587 (+   955us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:38.515645 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:19:38.515664 (+    19us) tablet.cc:662] Decoding operations
0709 12:19:38.516245 (+   581us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:38.516362 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:38.516395 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:19:38.517427 (+  1032us) tablet.cc:701] Row locks acquired
0709 12:19:38.517495 (+    68us) write_op.cc:260] PREPARE: finished
0709 12:19:38.517771 (+   276us) write_op.cc:270] Start()
0709 12:19:38.592442 (+ 74671us) write_op.cc:276] Timestamp: P: 1783599578517654 usec, L: 0
0709 12:19:38.592501 (+    59us) op_driver.cc:348] REPLICATION: starting
0709 12:19:38.593820 (+  1319us) log.cc:844] Serialized 538 byte log entry
0709 12:19:38.773257 (+179437us) op_driver.cc:464] REPLICATION: finished
0709 12:19:38.982120 (+208863us) write_op.cc:301] APPLY: starting
0709 12:19:38.982248 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:38.986617 (+  4369us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:38.986749 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:38.987546 (+   797us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:38.987868 (+   322us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:19:38.987928 (+    60us) write_op.cc:312] APPLY: finished
0709 12:19:38.988233 (+   305us) log.cc:844] Serialized 25 byte log entry
0709 12:19:38.999246 (+ 11013us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:39.000680 (+  1434us) write_op.cc:454] Released schema lock
0709 12:19:39.001680 (+  1000us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":138147,"reported_tablets":3,"spinlock_wait_cycles":23604480,"child_traces":[["op",{"apply.queue_time_us":180343,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8305,"num_ops":1,"prepare.queue_time_us":95488,"prepare.run_cpu_time_us":8752,"prepare.run_wall_time_us":109102,"replication_time_us":180410,"thread_start_us":183032,"threads_started":2}]]}
I20260709 12:19:39.014508  2361 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:39.053110 28986 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:39.086585 28360 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:39.181613 28433 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:39.238154 28186 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 15, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 15 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:39.280442 28181 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:39.339749 28290 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:39.358269 28433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:39.359148 28433 raft_consensus.cc:606] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:39.365991 28187 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 238) took 77912 ms (client timeout 15000 ms). Trace:
W20260709 12:19:39.368099 28187 rpcz_store.cc:269] 0709 12:18:21.508993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.524376 (+ 15383us) service_pool.cc:224] Handling call
0709 12:19:39.365772 (+77841396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":648529,"reported_tablets":52}
I20260709 12:19:39.372272 28437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:39.376557 28437 raft_consensus.cc:606] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:39.368085 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 114fa146124747358c997ac1008e1b93 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:33064
I20260709 12:19:39.380622 28986 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:39.386186  2361 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:39.394313 28433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:39.394929 28433 raft_consensus.cc:606] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:39.395390 28433 raft_consensus.cc:614] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:39.397570 28433 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:39.401691 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 114fa146124747358c997ac1008e1b93 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:39.403673 28433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:39.404495 28433 raft_consensus.cc:606] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:39.411135 28286 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:39.411795 28286 raft_consensus.cc:606] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:39.412963  2084 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:39.425335 28433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:39.424778 28986 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:39.426031 28433 raft_consensus.cc:606] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:39.427641 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:19:39.435300 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:39.436626 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:39.442415  2308 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:39.455295 28438 raft_consensus.cc:2955] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:39.494091  2308 raft_consensus.cc:993] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.495633  2308 raft_consensus.cc:1081] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.498155 28360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55954
I20260709 12:19:39.499063 28360 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.499534 28360 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:39.506646 28181 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 11 to 12, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 15 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:39.522905 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet bea7beef1d4c49159d75e5f61eaaeb5a with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:39084
I20260709 12:19:39.532769  2385 tablet_replica.cc:333] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:19:39.538663  2385 raft_consensus.cc:2243] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:39.558106  2385 raft_consensus.cc:2272] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:39.560745 28360 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:39.566448 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bea7beef1d4c49159d75e5f61eaaeb5a with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:39.570801 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:39.571534 28986 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:39.572495 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:39.573177 28288 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:39.573473 28288 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:39.574450 28288 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:39.575873 28288 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:39.577929 28360 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:39.580943 28986 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:19:39.588275 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:39.589865  2318 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:39.590920  2385 ts_tablet_manager.cc:1916] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:39.626611 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ee4696ba90a447fa07c4d09407217f6 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 12)) from {username='slave'} at 127.0.0.1:36572
I20260709 12:19:39.635500  2385 ts_tablet_manager.cc:1929] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 12:19:39.636209  2385 log.cc:1199] T bea7beef1d4c49159d75e5f61eaaeb5a P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/bea7beef1d4c49159d75e5f61eaaeb5a
I20260709 12:19:39.638330 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet bea7beef1d4c49159d75e5f61eaaeb5a (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:39.642840 28288 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
W20260709 12:19:39.652324 28183 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 221) took 29889 ms (client timeout 15000 ms). Trace:
W20260709 12:19:39.652890 28183 rpcz_store.cc:269] 0709 12:19:09.786783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.814907 (+ 28124us) service_pool.cc:224] Handling call
0709 12:19:39.652077 (+29837170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3900891,"reported_tablets":52}
I20260709 12:19:39.645028  2318 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:39.657538  2318 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:39.701550 28189 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 226) took 31832 ms (client timeout 15000 ms). Trace:
W20260709 12:19:39.702212 28189 rpcz_store.cc:269] 0709 12:19:07.869034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.869233 (+   199us) service_pool.cc:224] Handling call
0709 12:19:39.701332 (+31832099us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38839,"reported_tablets":52}
I20260709 12:19:39.702497 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet bea7beef1d4c49159d75e5f61eaaeb5a with delete_type TABLET_DATA_TOMBSTONED (Replica has no consensus available (current committed config index is 12)) from {username='slave'} at 127.0.0.1:39084
I20260709 12:19:39.704452 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet bea7beef1d4c49159d75e5f61eaaeb5a (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:39.704861  2389 raft_consensus.cc:993] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.706241  2302 raft_consensus.cc:1081] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.711365 28366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:39.711964 28366 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.712289  2302 raft_consensus.cc:993] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.712357 28366 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:39.712898  2389 raft_consensus.cc:1081] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.721539 28360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:39.722232 28360 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.722609 28360 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.728044  2302 raft_consensus.cc:993] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.728642  2302 raft_consensus.cc:1081] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.732506 28365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:39.733181 28365 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.733539 28365 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:39.734746 28360 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:39.737536 28360 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:39.738673 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:39.739467 28288 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.747195  2339 raft_consensus.cc:993] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.747802  2339 raft_consensus.cc:1081] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:39.751471 28365 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:39.753685 28359 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:39.754346 28359 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.754501 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:39.754846 28359 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:39.755115 28290 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:39.755220 28365 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:39.769272 28359 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:39.770584 28288 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:39.780965 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:39.809065 28433 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:39.809417 28433 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:39.810277 28433 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:39.811520 28433 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.793352 28181 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 20 to 21, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 21 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.803872 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:39.814409 28438 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:39.814735 28438 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:39.815616 28438 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:39.817008 28438 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:39.805119 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:39.818125 28432 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:39.818424 28432 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:39.819343 28432 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.12, Last appended by leader: 12, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:39.820585 28432 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:39.803275 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 3ee4696ba90a447fa07c4d09407217f6 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:39.839154 28366 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:39.842645 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:39.843304 28436 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:39.843631 28436 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:39.844447 28436 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:39.844787 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:39.845430 28285 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:39.845636 28436 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:39.841564 28366 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:39.862342 28285 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:39.862596 28436 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:39.865100 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:39.866055  2386 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:39.866627  2386 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:39.867619  2386 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:39.868954 28289 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:39.872684 28290 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:39.877287 28433 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:19:39.878333 28359 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:39.878773 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:39.879359 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 11 candidate_status { last_received { term: 10 index: 12 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:39.879825  2318 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:39.880082 28986 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:39.880925  2318 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:39.886132  2318 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:39.884140 28438 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:19:39.890005 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:39.891185  2023 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:39.891598 28432 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:19:39.892930 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:39.893582 28986 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 11.
I20260709 12:19:39.911988  2394 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:39.912746  2394 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:39.919320  2023 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:39.958494 28190 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 242) took 5873 ms. Trace:
I20260709 12:19:39.959180 28190 rpcz_store.cc:276] 0709 12:19:34.085362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.249717 (+164355us) service_pool.cc:224] Handling call
0709 12:19:39.958262 (+5708545us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13312,"reported_tablets":52}
I20260709 12:19:39.917882  2394 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.12, Last appended by leader: 12, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:39.971662  2023 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:40.026562 28181 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 18 to 19, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 19 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.036619 28181 catalog_manager.cc:5697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 14 to 15, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 15 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.038928 28181 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 12 to 13, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 13 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.050235 28991 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:19:40.052492  2023 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:19:40.146112 28432 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:40.172518 28991 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:19:40.174012  2339 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:40.175247 28290 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:19:40.177781  2339 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:40.184092 28285 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:19:40.187271  2394 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:40.198112  2329 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index 6 to 13, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:40.200248 28990 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 6 to 13, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:40.230327 28190 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 10 to 11, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 11 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.246601 28189 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 6 to 13, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 21 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:40.285953 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6c8442db4dc44c619fd67179aa212063 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:40.287098 28288 raft_consensus.cc:2955] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 6 to 13, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:40.297068 28433 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:40.311403  2023 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:19:40.324355 28432 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:40.326013  2023 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:40.334414 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c8442db4dc44c619fd67179aa212063 with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:39084
I20260709 12:19:40.358708  2410 tablet_replica.cc:333] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:19:40.359915  2410 raft_consensus.cc:2243] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:40.362963  2410 raft_consensus.cc:2272] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:40.371868  2410 ts_tablet_manager.cc:1916] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:40.395010  2410 ts_tablet_manager.cc:1929] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.12
I20260709 12:19:40.395447  2410 log.cc:1199] T 6c8442db4dc44c619fd67179aa212063 P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/6c8442db4dc44c619fd67179aa212063
I20260709 12:19:40.397213 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet 6c8442db4dc44c619fd67179aa212063 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:40.409471 28438 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 11 index: 13. (index mismatch)
I20260709 12:19:40.413051  2393 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:40.421468 28290 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:40.434691  2393 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:19:40.443048 28289 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:40.458647 28990 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 11 index: 13. (index mismatch)
I20260709 12:19:40.474004  2318 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:40.486249  2023 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:40.516218 28285 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:40.528170  2318 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:19:40.574275 28438 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:40.577363  2386 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:40.623842 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:40.822923 28432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:40.824301 28432 raft_consensus.cc:606] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:40.830821 28432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:40.831491 28432 raft_consensus.cc:606] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:40.832906 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:40.836359 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:40.837503 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:40.846488  2329 raft_consensus.cc:993] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:40.847285  2390 raft_consensus.cc:1081] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:40.849179 28289 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59792
I20260709 12:19:40.849653 28289 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:40.850010 28289 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.863193 28289 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:40.866082 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:40.866264 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:40.866735 28366 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.866878 28432 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:40.867324 28432 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:40.868196 28432 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:40.869436 28432 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.865298 28289 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:40.891383 28432 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:40.895390 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:40.896878  2308 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:40.897687  2308 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:40.898870  2308 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:40.926470 28366 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:40.945545 28286 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:41.017908 28181 catalog_manager.cc:5697] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 15 to 16, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 16 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:41.057703 28255 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9cb896260dae4cf6bb09ba7728c603f3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:41.058116 28438 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:41.060273  2308 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:41.061887 28359 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:41.066334  2361 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:41.119891  2383 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.123291 28366 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.130338  2302 raft_consensus.cc:993] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:41.130975  2302 raft_consensus.cc:1081] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:41.133137 28360 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:41.135077 28360 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:41.135504 28360 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:41.147346 28432 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.169418 28360 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:41.176533 28360 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:41.179567 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:41.180292 28290 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:41.184278 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:41.185047 28433 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:19:41.185361 28433 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:41.187209 28433 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:41.188824 28433 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:41.191728 28290 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:19:41.195191 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:41.196146  2318 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:19:41.196856  2318 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:41.197810  2318 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:41.201700 28433 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:19:41.210300 28183 catalog_manager.cc:5697] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:19:41.330268 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 63999.6us
I20260709 12:19:41.349586 28433 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:41.353621  2383 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:41.365974 28433 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:41.414970 28366 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:41.420351  2383 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:41.424525 28190 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 24 to 25, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 25 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:41.431077 28360 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:19:41.433941 28404 consensus_peers.cc:597] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 956fb13fa38243fd862789f3046aba97. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:41.435284  2329 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:41.441481 28286 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:41.447024  2329 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:41.447569  2439 ts_tablet_manager.cc:933] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:19:41.496243 28366 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.489212  2302 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:19:41.496574  2436 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.062s	user 0.003s	sys 0.000s
I20260709 12:19:41.545670  2439 tablet_copy_client.cc:323] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:19:41.566942 28302 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet 9cb896260dae4cf6bb09ba7728c603f3 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:19:41.568989 28302 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet 9cb896260dae4cf6bb09ba7728c603f3 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-9cb896260dae4cf6bb09ba7728c603f3
I20260709 12:19:41.566273 28190 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.580083 28438 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:19:41.590540  2386 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:41.604628 28436 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:41.620535 28290 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.620592 28302 tablet_copy_source_session.cc:215] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:19:41.631726 28404 consensus_peers.cc:597] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e3e7c85ff90423aa9f9d234d992379b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:41.627727 28360 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:41.635893  2302 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:41.652133 28286 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:41.654415  2439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb896260dae4cf6bb09ba7728c603f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:41.658061  2302 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:19:41.762781  2339 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.780215 28288 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:19:41.780372 28286 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.790891 28366 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.801724  2417 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:41.820232 28181 catalog_manager.cc:5697] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 18 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.822540 28286 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:41.863526  2453 ts_tablet_manager.cc:933] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:41.878357  2439 tablet_copy_client.cc:806] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:41.886732  2453 tablet_copy_client.cc:323] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:19:41.886816  2439 tablet_copy_client.cc:670] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:41.891541 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:41.899591 28360 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:41.899888  2439 tablet_copy_client.cc:538] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:41.901467  2383 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:19:41.906956 28255 consensus_peers.cc:597] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5235a8de06744285b8c3fe8f9f857223. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:41.913590 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 43963.9us
I20260709 12:19:41.915727 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 956fb13fa38243fd862789f3046aba97 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:19:41.935700 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 956fb13fa38243fd862789f3046aba97 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-956fb13fa38243fd862789f3046aba97
I20260709 12:19:41.944371 28438 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:41.954995  2383 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:41.968039  2361 raft_consensus.cc:2955] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:41.980748  2439 tablet_bootstrap.cc:492] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:19:41.990751 28451 tablet_copy_source_session.cc:215] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:41.994774  2453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956fb13fa38243fd862789f3046aba97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:42.013375 28360 raft_consensus.cc:2955] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:42.031942 28181 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 14, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 22 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:42.035030 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 116769us
I20260709 12:19:42.074093 28436 raft_consensus.cc:2955] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:42.189982  2453 tablet_copy_client.cc:806] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:42.190729  2453 tablet_copy_client.cc:670] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:42.207612  2453 tablet_copy_client.cc:538] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:42.214159 28360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:42.214900 28360 raft_consensus.cc:606] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:42.218807 28288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:42.219684 28288 raft_consensus.cc:606] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:42.221275 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:42.223495 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:42.224400 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:42.237012  2460 ts_tablet_manager.cc:933] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:42.250829  2439 tablet_bootstrap.cc:492] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:42.258023  2439 tablet_bootstrap.cc:492] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:42.258263  2460 tablet_copy_client.cc:323] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:19:42.260217  2439 ts_tablet_manager.cc:1403] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.280s	user 0.045s	sys 0.007s
I20260709 12:19:42.270571  2439 raft_consensus.cc:359] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:42.271582  2439 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:42.274219  2439 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:42.277141  2439 ts_tablet_manager.cc:1434] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.004s
I20260709 12:19:42.289135 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 2e3e7c85ff90423aa9f9d234d992379b from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:19:42.290153 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 2e3e7c85ff90423aa9f9d234d992379b from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-2e3e7c85ff90423aa9f9d234d992379b
I20260709 12:19:42.300732 28302 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-9cb896260dae4cf6bb09ba7728c603f3 received from {username='slave'} at 127.0.0.1:39592
I20260709 12:19:42.301385 28302 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-9cb896260dae4cf6bb09ba7728c603f3 on tablet 9cb896260dae4cf6bb09ba7728c603f3 with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:42.335093  2439 ts_tablet_manager.cc:933] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:19:42.341955 28451 tablet_copy_source_session.cc:215] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:42.352531  2460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3e7c85ff90423aa9f9d234d992379b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:42.367074  2453 tablet_bootstrap.cc:492] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
W20260709 12:19:42.373565 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52801.3us
I20260709 12:19:42.380365 28990 raft_consensus.cc:1217] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.10->[17.11-17.11]   Dedup: 17.11->[]
I20260709 12:19:42.405589  2439 tablet_copy_client.cc:323] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:19:42.409313 28302 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet 5235a8de06744285b8c3fe8f9f857223 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:19:42.409986 28302 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet 5235a8de06744285b8c3fe8f9f857223 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-5235a8de06744285b8c3fe8f9f857223
I20260709 12:19:42.518158  2460 tablet_copy_client.cc:806] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:42.534993  2460 tablet_copy_client.cc:670] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:42.531143  2361 raft_consensus.cc:993] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:42.519244 28302 tablet_copy_source_session.cc:215] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:42.541684  2361 raft_consensus.cc:1081] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:42.544116 28359 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55954
I20260709 12:19:42.544765 28359 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:42.545123 28359 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:42.547304  2439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5235a8de06744285b8c3fe8f9f857223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:42.552618  2460 tablet_copy_client.cc:538] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:42.603320 28359 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:42.619535  2453 tablet_bootstrap.cc:492] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:42.627416  2453 tablet_bootstrap.cc:492] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:42.628386  2453 ts_tablet_manager.cc:1403] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.262s	user 0.038s	sys 0.008s
I20260709 12:19:42.630532 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:42.631215 28285 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:42.631549 28285 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:42.631949 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:42.632570 28436 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:42.632404 28285 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:42.633653 28285 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:42.680366 28359 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:42.714159 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 73301.9us
I20260709 12:19:42.742806 28285 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:42.746186  2460 tablet_bootstrap.cc:492] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: Bootstrap starting.
W20260709 12:19:42.756269 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 113722us
I20260709 12:19:42.776747 28330 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:42.778556 28436 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:42.791447  2417 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:42.792080  2417 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:42.792985  2417 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:42.823424  2453 raft_consensus.cc:359] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:42.824406  2453 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:42.825125  2453 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:19:42.837667 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 194839us
I20260709 12:19:42.844410  2453 ts_tablet_manager.cc:1434] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.216s	user 0.008s	sys 0.001s
W20260709 12:19:42.884524 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 80473.5us
W20260709 12:19:42.891234 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 86295.1us
I20260709 12:19:42.884027 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-956fb13fa38243fd862789f3046aba97 received from {username='slave'} at 127.0.0.1:53280
I20260709 12:19:42.913726 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-956fb13fa38243fd862789f3046aba97 on tablet 956fb13fa38243fd862789f3046aba97 with peer f526369c0d0743b192be6e205540a059
W20260709 12:19:42.949488 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 64127.9us
I20260709 12:19:42.988652  2439 tablet_copy_client.cc:806] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:42.989478  2439 tablet_copy_client.cc:670] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
W20260709 12:19:42.990577 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 103959us
I20260709 12:19:43.053504  2439 tablet_copy_client.cc:538] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:43.086284  2417 raft_consensus.cc:993] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:43.122454  2417 raft_consensus.cc:1081] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:43.127424 28189 catalog_manager.cc:5697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 17 to 18, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:43.145769 28437 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:19:43.146488 28437 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:43.146874 28437 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.166960 28437 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:43.195082  2308 raft_consensus.cc:1064] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:43.237048  2439 tablet_bootstrap.cc:492] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
W20260709 12:19:43.249382 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 57913.6us
I20260709 12:19:43.250480 28437 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:43.263013 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:43.269788 28285 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.277659 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:43.278623 28358 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:43.278964 28358 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:43.279883 28358 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:43.330340 28285 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
W20260709 12:19:43.337159 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 68129.7us
I20260709 12:19:43.341614 28437 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:43.349167 28990 raft_consensus.cc:1217] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.9->[16.10-16.10]   Dedup: 16.10->[]
W20260709 12:19:43.353024 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.354933 28358 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.357430  2482 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:19:43.386932 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:43.401825 28358 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
W20260709 12:19:43.417449 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 239029us
I20260709 12:19:43.397277  2308 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:43.431710 28286 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:43.434237  2496 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:43.434899  2496 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:43.435842  2496 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:43.470793 28284 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:43.501055 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:43.537283  2488 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:43.597198 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 73776.8us
W20260709 12:19:43.643309 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 102364us
I20260709 12:19:43.662839 28361 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:43.671192  2490 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:43.677208 28437 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:43.711148  2308 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:43.728701 28991 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:43.747848  2487 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:19:43.809080  2501 raft_consensus.cc:1064] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:43.832957 28366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:43.833778 28366 raft_consensus.cc:606] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:43.829334  2501 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:43.858465 28284 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:43.859506 28284 raft_consensus.cc:606] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:43.864183 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:43.868026 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:43.869056 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:43.890487  2469 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:43.893105 28437 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:43.908234 28358 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:43.918892 28360 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:43.923518  2507 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:43.931903 28986 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:19:43.954780  2513 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.050s	user 0.003s	sys 0.003s
W20260709 12:19:43.967491  2439 log.cc:927] Time spent T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 12:19:43.968663  2460 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:19:43.969379  2460 tablet_bootstrap.cc:492] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:43.970947 28189 catalog_manager.cc:5697] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:43.977926  2460 tablet_bootstrap.cc:492] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:43.978888  2460 ts_tablet_manager.cc:1403] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.233s	user 0.049s	sys 0.018s
I20260709 12:19:43.986172  2460 raft_consensus.cc:359] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:44.003257 28986 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:44.005481 28358 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:44.010538  2507 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:44.012620  2507 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:44.007697  2460 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:44.018550  2460 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:44.021206 28284 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:44.028836 28181 catalog_manager.cc:5697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 16 to 17, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 17 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:44.032718 28288 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:44.060398  2460 ts_tablet_manager.cc:1434] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 12:19:44.053907  2507 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:19:44.062295  2302 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:19:44.070020 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-2e3e7c85ff90423aa9f9d234d992379b received from {username='slave'} at 127.0.0.1:53280
I20260709 12:19:44.082003 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-2e3e7c85ff90423aa9f9d234d992379b on tablet 2e3e7c85ff90423aa9f9d234d992379b with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:44.083026 28288 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:44.092722 28986 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:44.101871 28360 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:44.112174  2308 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:44.135826  2439 tablet_bootstrap.cc:492] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=3 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:44.156359  2318 raft_consensus.cc:993] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:44.157035  2318 raft_consensus.cc:1081] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:44.167529 28437 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:19:44.168251 28437 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:44.168670 28437 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:44.126724  2477 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:19:44.186179  2439 tablet_bootstrap.cc:492] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:44.187230  2439 ts_tablet_manager.cc:1403] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.010s	user 0.066s	sys 0.008s
I20260709 12:19:44.201084  2490 raft_consensus.cc:993] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:44.201776  2490 raft_consensus.cc:1081] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:44.211423 28361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55954
I20260709 12:19:44.212108 28361 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:44.212482 28361 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:44.251981  2495 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:44.260228 28360 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:44.286711  2469 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:19:44.286603  2522 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:19:44.326838 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44077.7us
I20260709 12:19:44.356068 28366 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:44.358564 28437 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:44.359504 28986 raft_consensus.cc:1217] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Deduplicated request from leader. Original: 18.10->[18.11-18.11]   Dedup: 18.11->[]
I20260709 12:19:44.362589 28361 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:44.369120 28437 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:44.374132 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:44.374912 28358 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:44.375314 28358 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:44.375587 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:44.376217 28986 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:44.376163 28358 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:44.383915 28361 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:44.386408 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:44.388820 28437 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:44.398272  2439 raft_consensus.cc:359] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:44.399297  2439 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:44.403558  2439 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:44.421340 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:44.422132 28286 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:44.422439 28286 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:44.423269 28286 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:44.424531 28286 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:44.450963 28990 raft_consensus.cc:2955] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:44.462878  2439 ts_tablet_manager.cc:1434] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.275s	user 0.004s	sys 0.003s
I20260709 12:19:44.472015 28286 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:44.476641 28991 raft_consensus.cc:1217] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.13->[22.14-22.14]   Dedup: 22.14->[]
I20260709 12:19:44.484694 28437 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:44.486887 28332 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:44.501159 28986 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:44.507565 28358 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:44.507798 28991 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:44.521000 28181 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:44.531533  2527 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:44.532240  2527 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:44.543179 28302 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-5235a8de06744285b8c3fe8f9f857223 received from {username='slave'} at 127.0.0.1:39592
I20260709 12:19:44.543746 28302 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-5235a8de06744285b8c3fe8f9f857223 on tablet 5235a8de06744285b8c3fe8f9f857223 with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:44.533264  2527 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:44.570456 28436 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:44.573889 28189 catalog_manager.cc:5697] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:44.587350 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:44.589486  2302 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:44.590016  2302 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:44.595104 28986 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:44.598464  2302 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:44.605382 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 100454us
I20260709 12:19:44.609696 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9cb896260dae4cf6bb09ba7728c603f3 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:44.620326 28358 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:19:44.632371 28289 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:44.628244  2507 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
W20260709 12:19:44.691807 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 92287.7us
I20260709 12:19:44.696048 28366 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:19:44.767329 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 98934.8us
W20260709 12:19:44.814389 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64214.1us
W20260709 12:19:44.815022 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64286.4us
I20260709 12:19:44.837383 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 9cb896260dae4cf6bb09ba7728c603f3 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:33064
I20260709 12:19:44.872398  2507 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:44.896898 28181 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 18 to 19, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 19 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:44.902710  2535 tablet_replica.cc:333] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:19:44.903155 28186 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 17 to 18, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:44.904101  2535 raft_consensus.cc:2243] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 12:19:44.941859 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 48424.1us
I20260709 12:19:44.944809  2535 raft_consensus.cc:2272] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:44.950078  2525 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:44.956238 28366 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:44.962642  2535 ts_tablet_manager.cc:1916] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:44.967092 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 956fb13fa38243fd862789f3046aba97 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:44.980783 28185 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 11 to 12, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:44.994571  2535 ts_tablet_manager.cc:1929] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.12
I20260709 12:19:44.996788  2535 log.cc:1199] T 9cb896260dae4cf6bb09ba7728c603f3 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/9cb896260dae4cf6bb09ba7728c603f3
I20260709 12:19:45.000648 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 9cb896260dae4cf6bb09ba7728c603f3 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:45.006073 28289 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:45.009227 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 956fb13fa38243fd862789f3046aba97 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:36572
I20260709 12:19:45.010692  2482 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:45.023397 28986 raft_consensus.cc:2955] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.050071  2361 raft_consensus.cc:1064] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:45.050644 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:45.053653  2361 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 12, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:45.068156  2526 raft_consensus.cc:1064] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:45.071504  2536 tablet_replica.cc:333] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:19:45.072230  2526 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:45.083958  2536 raft_consensus.cc:2243] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:45.084468 28986 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:45.096413  2536 raft_consensus.cc:2272] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:45.100078 28433 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:45.103569  2483 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:19:45.111902  2536 ts_tablet_manager.cc:1916] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:45.101400  2501 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:19:45.120010 28437 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:45.122236 28991 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:45.132570  2477 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:45.135084  2418 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:45.136212 28361 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:19:45.136229 28289 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:45.141934  2477 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:45.143635 28366 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:45.147217  2465 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:45.148980  2501 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:45.161528 28984 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:45.176077  2308 raft_consensus.cc:2955] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 LEADER]: Committing config change with OpId 22.15: config changed from index 14 to 15, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.184916  2530 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:45.188505 28991 raft_consensus.cc:2955] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.228507 28437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:45.229243 28437 raft_consensus.cc:606] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:45.236989 28436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:45.237668 28436 raft_consensus.cc:606] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:45.243257 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:45.243958 28359 raft_consensus.cc:606] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:45.247123 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:19:45.249246 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:45.249336  2536 ts_tablet_manager.cc:1929] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.11
I20260709 12:19:45.249866  2536 log.cc:1199] T 956fb13fa38243fd862789f3046aba97 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/956fb13fa38243fd862789f3046aba97
I20260709 12:19:45.242717 28436 raft_consensus.cc:2955] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.252275 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 956fb13fa38243fd862789f3046aba97 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:45.264441 28186 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 14 to 15, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:45.269415 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:45.302635 28286 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:45.309350 28361 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:19:45.321528  2302 raft_consensus.cc:993] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:45.328660  2465 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:19:45.323523  2507 raft_consensus.cc:1081] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:45.345860 28991 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:19:45.354287  2549 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:19:45.355691 28286 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59792
I20260709 12:19:45.355687  2308 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:19:45.358484 28286 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:45.359843 28286 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:45.388334 28436 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:19:45.419538  2493 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:45.420469 28286 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:45.426159 28286 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:45.429522 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:45.430210 28361 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:45.431716 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:45.432380 28436 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:45.432688 28436 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:45.428745 28984 raft_consensus.cc:2955] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.433434 28436 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:45.435227 28986 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.436349 28433 raft_consensus.cc:2955] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:19:45.426395  2548 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:19:45.473284 28361 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:45.475091 28255 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:45.476162  2462 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:45.476783  2462 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:45.477820  2462 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:45.428525  2507 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.484263 28436 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:45.498427 28436 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
W20260709 12:19:45.589320 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 68599.6us
I20260709 12:19:45.628139  2533 raft_consensus.cc:2955] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 LEADER]: Committing config change with OpId 22.16: config changed from index 15 to 16, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:19:45.634627 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 49687.4us
I20260709 12:19:45.694087 28185 catalog_manager.cc:5697] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 15 to 16, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 16 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:45.696208 28181 catalog_manager.cc:5697] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.717303  2525 raft_consensus.cc:993] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:45.717926  2525 raft_consensus.cc:1081] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
W20260709 12:19:45.726815 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 49137.1us
I20260709 12:19:45.766043 28286 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.803694 28359 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:45.804340 28359 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:45.804733 28359 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:45.824528 28366 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.828884 28181 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 15 to 16, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 22 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.835975 28359 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:45.845932 28437 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:45.860137 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5235a8de06744285b8c3fe8f9f857223 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:45.868018  2524 raft_consensus.cc:993] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:45.868626  2524 raft_consensus.cc:1081] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:45.882802 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet 5235a8de06744285b8c3fe8f9f857223 with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:39084
I20260709 12:19:45.882905 28361 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:45.883687 28359 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:45.885035 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:45.885629 28433 raft_consensus.cc:3055] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:45.888885  2569 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:45.885934 28433 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:45.891710 28433 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:45.892973 28433 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:45.904834 28433 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:19:45.906162 28332 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:45.909812 28361 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:45.910899 28984 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:45.913197  2483 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:45.914610  2483 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:45.916093  2483 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:45.922884 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:45.941648  2567 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:19:45.953971  2526 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:45.963951  2567 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:45.988904 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:45.994050 28288 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:46.001683 28433 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:45.989020  2575 tablet_replica.cc:333] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:19:45.990849 28289 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59778
I20260709 12:19:45.978564 28984 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:46.003407 28289 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:46.003819 28289 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.003911  2575 raft_consensus.cc:2243] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:46.006072  2575 pending_rounds.cc:70] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Trying to abort 1 pending ops.
I20260709 12:19:46.006345  2575 pending_rounds.cc:77] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Aborting op as it isn't in flight: id { term: 22 index: 15 } timestamp: 7305623900370731008 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "5235a8de06744285b8c3fe8f9f857223" old_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } } new_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } } }
I20260709 12:19:46.012676  2575 raft_consensus.cc:2917] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Aborting config change with OpId 22.15: Aborted: Op aborted
I20260709 12:19:46.013161  2575 raft_consensus.cc:2889] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:19:46.003162 28360 raft_consensus.cc:2955] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:46.013492  2575 raft_consensus.cc:2272] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:46.024812 28288 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:19:46.030830 28289 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:46.037665 28289 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:46.038399 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:46.047354 28183 catalog_manager.cc:5697] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 12 to 13, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 18 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:46.037953 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2e3e7c85ff90423aa9f9d234d992379b with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:46.051786 28366 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:46.052199 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:46.063405 28433 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.055954  2533 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:46.065577  2575 ts_tablet_manager.cc:1916] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:46.086665 28433 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:46.089646 28366 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:46.111837 28366 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:46.113190 28366 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.114878 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:46.115826  2569 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:46.117035  2569 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:46.117946  2569 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:46.161259 28186 catalog_manager.cc:5697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 15 to 16, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 16 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:46.162617 28185 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 21 to 22, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 22 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:46.166772 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e3e7c85ff90423aa9f9d234d992379b with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:36572
I20260709 12:19:46.175524 28366 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:19:46.200492  2575 ts_tablet_manager.cc:1929] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.15
I20260709 12:19:46.201023  2575 log.cc:1199] T 5235a8de06744285b8c3fe8f9f857223 P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/5235a8de06744285b8c3fe8f9f857223
I20260709 12:19:46.218194 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet 5235a8de06744285b8c3fe8f9f857223 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:46.302414  2582 tablet_replica.cc:333] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:19:46.303550  2582 raft_consensus.cc:2243] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:46.304394  2582 raft_consensus.cc:2272] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:46.329602  2582 ts_tablet_manager.cc:1916] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:46.375036 28288 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:19:46.377885  2574 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:46.419792  2582 ts_tablet_manager.cc:1929] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.12
I20260709 12:19:46.431432  2582 log.cc:1199] T 2e3e7c85ff90423aa9f9d234d992379b P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/2e3e7c85ff90423aa9f9d234d992379b
I20260709 12:19:46.447023 28437 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:19:46.455588  2483 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:19:46.484812 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 2e3e7c85ff90423aa9f9d234d992379b (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:46.568089 28437 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:46.570891  2533 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:46.604828 28359 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:46.606525  2493 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:46.827167 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:47.026925 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:47.027622 28359 raft_consensus.cc:606] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:47.034948 28286 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:47.035518 28286 raft_consensus.cc:606] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:47.038767 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:47.046434 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:47.047587 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:47.411669 28366 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:47.524950  2483 raft_consensus.cc:993] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:47.526081  2483 raft_consensus.cc:1081] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:47.592401 28433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:19:47.593248 28433 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:47.593703 28433 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:47.663201 28437 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:47.683032  2563 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:47.682983 28433 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:19:47.698819 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49732.1us
I20260709 12:19:47.708532 28433 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:47.717913  2577 raft_consensus.cc:993] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:47.718487  2493 raft_consensus.cc:1081] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
W20260709 12:19:47.711287 28330 consensus_peers.cc:597] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e6bd3eccb6c4f4c8fedbc76fa829a96. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:47.742620 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:47.743844 28360 raft_consensus.cc:3055] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:47.744354 28360 raft_consensus.cc:740] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:47.745362 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:47.746341 28284 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:47.745329 28360 consensus_queue.cc:260] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:47.754385 28360 raft_consensus.cc:3060] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:47.759734 28286 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:47.768253 28433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:19:47.768916 28433 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:47.769388 28433 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:47.778625 28433 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:47.780824 28433 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:47.769194 28284 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:47.783502 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:47.784194 28991 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:47.786532  2483 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:47.791152  2563 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:47.800830 28433 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:47.806315 28991 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:47.810434 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:47.811162 28292 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:47.811466 28292 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:47.812316 28292 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:47.814901 28438 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:47.808034 28404 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:47.817795  2567 raft_consensus.cc:2804] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:47.818351  2567 raft_consensus.cc:697] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:47.819264  2567 consensus_queue.cc:237] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:47.834317 28292 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:47.843565 28360 raft_consensus.cc:2468] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:19:47.843875 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:47.850181  2561 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:47.850783  2561 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:47.851660  2561 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:47.853440 28292 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:19:47.874511 28359 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:47.888335 28288 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:47.916512 28185 catalog_manager.cc:5697] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 19 to 20, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 20 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:47.923074 28437 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:19:47.925253 28284 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:19:47.926137  2563 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:47.927282  2483 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:19:47.928905 28330 consensus_peers.cc:597] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ab1586b52be49808b90fcc88bdf2d60. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:47.935753 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 68621.5us
W20260709 12:19:47.947252  2601 log.cc:927] Time spent T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:47.957859  2563 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Committing config change with OpId 22.12: config changed from index -1 to 12, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:47.956945 28183 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 9, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 14 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:47.973944 28284 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Committing config change with OpId 22.12: config changed from index -1 to 12, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:47.979192 28432 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Committing config change with OpId 22.12: config changed from index -1 to 12, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:48.002956 28359 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:19:48.013484 28330 consensus_peers.cc:597] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d68008d2b3fd4f7bb72bf69f612b2998. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:48.014632 28432 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:48.016366  2483 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:48.017287 28284 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:48.020460  2483 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:48.059279 28185 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 15 to 16, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:48.062915 28185 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 12, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 22 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:48.076996  2563 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:48.083235 28432 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:48.082643 28284 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:48.088630  2614 ts_tablet_manager.cc:933] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:48.108685 28432 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:48.131567 28183 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:48.142128 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:48.176851 28404 consensus_peers.cc:597] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c474098832a04762b58521bdc2a92b5a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:48.178360 28359 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:48.181265  2589 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:48.193862 28284 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:48.196707  2614 tablet_copy_client.cc:323] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:19:48.199281 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:19:48.199944 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-8e6bd3eccb6c4f4c8fedbc76fa829a96
I20260709 12:19:48.232232 28376 tablet_copy_source_session.cc:215] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:48.234051  2567 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:48.247759  2589 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:48.254480 28360 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:48.261356  2614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:48.288662  2614 tablet_copy_client.cc:806] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:48.289379  2614 tablet_copy_client.cc:670] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:48.328199  2614 tablet_copy_client.cc:538] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:48.336187 28185 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:48.344198  2614 tablet_bootstrap.cc:492] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:19:48.347295 28432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:48.350178 28432 raft_consensus.cc:606] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:48.352448 28360 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:48.353861 28432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:48.354676 28432 raft_consensus.cc:606] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:48.358268 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:48.358928  2530 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:48.360422 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:48.361320 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:48.365346  2619 ts_tablet_manager.cc:933] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:48.383286  2619 tablet_copy_client.cc:323] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:19:48.392691 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 8ab1586b52be49808b90fcc88bdf2d60 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:19:48.393527 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 8ab1586b52be49808b90fcc88bdf2d60 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-8ab1586b52be49808b90fcc88bdf2d60
I20260709 12:19:48.395198 28288 raft_consensus.cc:1275] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:48.413324  2561 consensus_queue.cc:1048] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:48.416152  2589 raft_consensus.cc:993] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:48.416774  2589 raft_consensus.cc:1081] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:48.420542 28376 tablet_copy_source_session.cc:215] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:48.480259  2619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab1586b52be49808b90fcc88bdf2d60. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:19:48.582836 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 93040us
W20260709 12:19:48.583084 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 103312us
I20260709 12:19:48.695034 28991 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:48.718528  2567 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:48.731343  2631 ts_tablet_manager.cc:933] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:48.733402 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 109723us
W20260709 12:19:48.846995 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 90111.1us
I20260709 12:19:48.851397  2632 ts_tablet_manager.cc:933] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:48.880154  2631 tablet_copy_client.cc:323] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:19:48.882733 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet d68008d2b3fd4f7bb72bf69f612b2998 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:19:48.883463 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet d68008d2b3fd4f7bb72bf69f612b2998 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-d68008d2b3fd4f7bb72bf69f612b2998
I20260709 12:19:48.901365 28376 tablet_copy_source_session.cc:215] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:48.903154 28284 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:48.905570 28288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59792
I20260709 12:19:48.945402 28289 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:48.950475  2631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d68008d2b3fd4f7bb72bf69f612b2998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:48.973513 28288 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:48.974196 28288 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.997403 28288 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:49.007203 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:49.007989 28361 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:49.023425 28288 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:49.041821  2530 raft_consensus.cc:993] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:49.042570  2530 raft_consensus.cc:1081] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:49.029561 28361 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:19:49.078312 28359 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:49.078975 28359 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:49.079356 28359 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:49.092775  2632 tablet_copy_client.cc:323] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:19:49.093039 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:49.096899  2629 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:49.097534  2629 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:49.098793  2629 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:49.147475 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet c474098832a04762b58521bdc2a92b5a from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:19:49.148141 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet c474098832a04762b58521bdc2a92b5a from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-c474098832a04762b58521bdc2a92b5a
W20260709 12:19:49.241937  2636 log.cc:927] Time spent T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.133s	user 0.002s	sys 0.001s
I20260709 12:19:49.252051 28359 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:49.254658 28359 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:49.256685 28361 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:49.259271 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:49.259995 28432 raft_consensus.cc:3055] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:49.260272 28432 raft_consensus.cc:740] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:49.261128 28432 consensus_queue.cc:260] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:49.262502 28432 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:49.262739 28451 tablet_copy_source_session.cc:215] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:49.267764  2631 tablet_copy_client.cc:806] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:49.276340 28432 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:49.276751 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:49.277472 28435 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:49.277838 28435 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:49.278843 28435 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:49.279834  2632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c474098832a04762b58521bdc2a92b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:49.280097 28435 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:49.281672 28332 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:49.287744  2563 raft_consensus.cc:2804] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:49.288417  2563 raft_consensus.cc:697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:49.280687  2631 tablet_copy_client.cc:670] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:49.289564  2563 consensus_queue.cc:237] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:49.320582 28435 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
W20260709 12:19:49.330888 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50176.8us
I20260709 12:19:49.339264  2530 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:49.341355  2631 tablet_copy_client.cc:538] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:49.331928  2619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46950 (request call id 1063) took 1024 ms. Trace:
I20260709 12:19:49.416445  2614 tablet_bootstrap.cc:492] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:49.428535  2631 tablet_bootstrap.cc:492] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:19:49.420884  2614 tablet_bootstrap.cc:492] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:49.431367  2614 ts_tablet_manager.cc:1403] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.087s	user 0.044s	sys 0.018s
I20260709 12:19:49.368021  2619 rpcz_store.cc:276] 0709 12:19:48.307258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.307414 (+   156us) service_pool.cc:224] Handling call
0709 12:19:48.365977 (+ 58563us) ts_tablet_manager.cc:934] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
0709 12:19:48.478934 (+112957us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:19:49.064076 (+585142us) tablet_metadata.cc:665] Metadata flushed
0709 12:19:49.064313 (+   237us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:19:49.316732 (+252419us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:19:49.317190 (+   458us) tablet_replica.cc:206] Creating consensus instance
0709 12:19:49.331605 (+ 14415us) ts_tablet_manager.cc:989] Replica registered.
0709 12:19:49.331749 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":964,"tablet-copy.queue_time_us":57323,"thread_start_us":62258,"threads_started":2}
I20260709 12:19:49.450091  2614 raft_consensus.cc:359] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:49.452337  2614 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:49.450531 28186 catalog_manager.cc:5697] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 17 to 18, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:49.452523 28183 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 17, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:49.461879  2619 tablet_copy_client.cc:806] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:49.476188  2619 tablet_copy_client.cc:670] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:49.471710  2614 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:49.494792 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:49.503775  2614 ts_tablet_manager.cc:1434] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.001s
I20260709 12:19:49.516716 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-8e6bd3eccb6c4f4c8fedbc76fa829a96 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:19:49.517304 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-8e6bd3eccb6c4f4c8fedbc76fa829a96 on tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96 with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:49.546766  2619 tablet_copy_client.cc:538] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:49.582440  2632 tablet_copy_client.cc:806] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:49.583225  2632 tablet_copy_client.cc:670] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:49.588059  2619 tablet_bootstrap.cc:492] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
W20260709 12:19:49.600477 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 254561us
I20260709 12:19:49.619159  2632 tablet_copy_client.cc:538] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:49.648859 28359 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:49.652496  2647 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:49.701493 28435 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:19:49.718714 28255 consensus_peers.cc:597] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:19:49.768873 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 50880.7us
I20260709 12:19:49.768352 28991 raft_consensus.cc:1217] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.8->[14.9-14.9]   Dedup: 14.9->[]
W20260709 12:19:49.819145 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 48988.5us
W20260709 12:19:49.869922 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 58611.7us
I20260709 12:19:49.888906 28433 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:49.903154 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:49.904665 28292 raft_consensus.cc:3060] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:49.906652 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 48574.5us
I20260709 12:19:49.925181  2647 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:49.928319  2665 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:49.958300  2632 tablet_bootstrap.cc:492] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: Bootstrap starting.
W20260709 12:19:49.986882  2631 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:19:50.019925 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 74901.1us
I20260709 12:19:50.021530 28292 raft_consensus.cc:2468] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
W20260709 12:19:50.023034 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 79218.9us
I20260709 12:19:50.046132  2631 tablet_bootstrap.cc:492] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:50.088984 28291 raft_consensus.cc:1275] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:50.089838  2631 tablet_bootstrap.cc:492] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:50.111847  2631 ts_tablet_manager.cc:1403] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.684s	user 0.042s	sys 0.025s
I20260709 12:19:50.126907  2664 consensus_queue.cc:1048] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:50.155948 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 65798.1us
W20260709 12:19:50.267174  2619 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 12:19:50.286355 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 64644.9us
I20260709 12:19:50.310287  2631 raft_consensus.cc:359] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:50.311244  2631 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:50.311998  2631 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:50.315546  2631 ts_tablet_manager.cc:1434] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.203s	user 0.006s	sys 0.003s
I20260709 12:19:50.339564  2626 raft_consensus.cc:1064] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:50.343156  2626 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:50.383698 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-d68008d2b3fd4f7bb72bf69f612b2998 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:19:50.384208 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-d68008d2b3fd4f7bb72bf69f612b2998 on tablet d68008d2b3fd4f7bb72bf69f612b2998 with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:50.399612 28986 raft_consensus.cc:1217] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Deduplicated request from leader. Original: 18.10->[18.11-18.11]   Dedup: 18.11->[]
W20260709 12:19:50.438396 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 148317us
W20260709 12:19:50.450116 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 156343us
W20260709 12:19:50.526233 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 59947us
I20260709 12:19:50.531217 28986 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:50.549527 28366 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:50.555323 28366 raft_consensus.cc:606] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:50.559116 28437 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:50.565881 28438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:50.566633 28438 raft_consensus.cc:606] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:50.569939 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:50.572510 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:50.579547  2483 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:50.613250 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:50.623613 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 96019.9us
I20260709 12:19:50.644434  2530 raft_consensus.cc:993] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:50.645047  2644 raft_consensus.cc:1081] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:50.668991 28288 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:50.700382 28361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:50.701076 28361 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:50.701452 28361 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.704876  2574 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:50.718761 28361 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:50.736382  2664 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:50.758965 28361 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:19:50.760171 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 112915us
I20260709 12:19:50.760505 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:50.761968 28438 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:50.762326 28438 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:50.767767 28438 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:50.769174 28438 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.770268 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:50.779048 28287 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.810808  2619 tablet_bootstrap.cc:492] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=2 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:50.812033  2619 tablet_bootstrap.cc:492] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:50.812903  2619 ts_tablet_manager.cc:1403] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.225s	user 0.054s	sys 0.029s
I20260709 12:19:50.840576 28438 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:50.851492 28332 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:50.888577  2483 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:50.903595 28287 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:19:50.895712  2687 raft_consensus.cc:993] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:50.903080  2483 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:50.911535  2596 raft_consensus.cc:1081] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:50.919587  2483 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:50.920346  2661 raft_consensus.cc:1064] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
W20260709 12:19:50.920516  2632 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.061s	user 0.002s	sys 0.001s
I20260709 12:19:50.883317  2619 raft_consensus.cc:359] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:50.926199  2619 raft_consensus.cc:740] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:50.927049  2619 consensus_queue.cc:260] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:50.934000 28287 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59778
I20260709 12:19:50.934713 28287 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:50.935101 28287 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:50.954032 28288 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:50.942176  2574 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:50.956077  2619 ts_tablet_manager.cc:1434] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.143s	user 0.010s	sys 0.001s
I20260709 12:19:50.965283 28376 tablet_copy_service.cc:342] P 6634fa3fbd594726a5ec3c1686900284: Request end of tablet copy session f526369c0d0743b192be6e205540a059-8ab1586b52be49808b90fcc88bdf2d60 received from {username='slave'} at 127.0.0.1:56626
I20260709 12:19:50.966302 28376 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-8ab1586b52be49808b90fcc88bdf2d60 on tablet 8ab1586b52be49808b90fcc88bdf2d60 with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:50.966892 28989 raft_consensus.cc:1217] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.11->[22.12-22.12]   Dedup: 22.12->[]
I20260709 12:19:50.955247  2661 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:51.007754  2632 tablet_bootstrap.cc:492] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=2 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:51.008833  2632 tablet_bootstrap.cc:492] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:51.031023 28287 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:51.009716  2632 ts_tablet_manager.cc:1403] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.052s	user 0.076s	sys 0.017s
I20260709 12:19:51.048508 28287 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:51.017864 28984 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.055136 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:51.055773 28361 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:51.056286 28361 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:51.069399 28361 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:51.073387 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:51.074156 28989 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.077714 28361 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.086370 28185 catalog_manager.cc:5697] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 17 to 18, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:51.107424  2632 raft_consensus.cc:359] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:51.131160 28986 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:51.143639 28989 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:51.141098 28181 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.119474  2632 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:51.158888 28288 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:51.157793  2596 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:19:51.170197  2632 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:51.171326 28436 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:51.174541  2665 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:51.176851  2632 ts_tablet_manager.cc:1434] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.133s	user 0.009s	sys 0.000s
I20260709 12:19:51.180591 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-c474098832a04762b58521bdc2a92b5a received from {username='slave'} at 127.0.0.1:53280
I20260709 12:19:51.181075 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-c474098832a04762b58521bdc2a92b5a on tablet c474098832a04762b58521bdc2a92b5a with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:51.170538 28437 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.194088 28361 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:19:51.210825 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:51.216507  2483 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:51.241865  2690 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:51.245885  2690 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:51.246968  2690 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:51.384459  2663 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.389694 28439 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.429288 28989 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.444804 28288 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.458663 28185 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.474772  2626 raft_consensus.cc:1064] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:51.478281  2626 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 10, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:51.485437 28989 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Advancing to term 17
I20260709 12:19:51.515359 28287 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:51.520131 28358 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:51.527437 28359 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:51.542356  2596 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:51.546279 28984 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:51.546731 28439 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:51.548362  2663 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:51.556638  2665 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:51.575225  2574 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.583323 28986 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.590806 28984 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:51.591786 28288 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:51.603193 28292 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:19:51.613270  2596 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:19:51.660154 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 50165.4us
I20260709 12:19:51.669903  2596 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:51.675197  2665 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:51.665530 28186 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 19 to 20, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 20 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:51.699623  2596 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.701591 28439 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:19:51.704804  2624 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:51.713879 28292 raft_consensus.cc:2955] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.742409 28984 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 LEARNER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:19:51.742198 28991 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.755889  2665 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:51.779399 28292 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:19:51.781580 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d68008d2b3fd4f7bb72bf69f612b2998 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:51.782648 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:51.793648 28283 raft_consensus.cc:2955] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.813638  2663 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:19:51.851212 28186 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:51.856782 28989 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:19:51.858072 28359 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:19:51.865849  2577 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:19:51.867444  2682 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:19:51.883939  2626 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Committing config change with OpId 22.13: config changed from index 12 to 13, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.899324 28984 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index 12 to 13, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.931293 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet d68008d2b3fd4f7bb72bf69f612b2998 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:33064
I20260709 12:19:51.934736 28183 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 10 to 11, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 14 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:51.934021 28439 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index 12 to 13, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:19:51.977106  2706 log.cc:927] Time spent T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.053s	user 0.002s	sys 0.009s
I20260709 12:19:52.028573 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:52.021749 28185 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 12 to 13, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.054590 28292 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index 12 to 13, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:19:52.058743 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 45838us
I20260709 12:19:52.064632  2719 tablet_replica.cc:333] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:19:52.065963  2719 raft_consensus.cc:2243] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:52.069052  2719 raft_consensus.cc:2272] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:52.121850  2682 raft_consensus.cc:1064] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:52.124784  2719 ts_tablet_manager.cc:1916] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:52.140579 28360 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:52.124886  2682 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:52.150004 28989 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:52.152725 28437 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:52.152532  2574 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:52.156718  2626 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:52.176574  2661 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index 13 to 14, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.175239 28984 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.186473 28438 raft_consensus.cc:2955] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.208195 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:33064
W20260709 12:19:52.242079 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47244.8us
I20260709 12:19:52.276736 28183 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 13 to 14, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 22 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.287395 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8ab1586b52be49808b90fcc88bdf2d60 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:52.345759 28360 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:52.346779 28438 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:52.359377 28984 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:19:52.361932 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 155622us
I20260709 12:19:52.357954  2724 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:19:52.369350  2577 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:52.388788 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ab1586b52be49808b90fcc88bdf2d60 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:36572
I20260709 12:19:52.393417  2577 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:52.417523  2729 tablet_replica.cc:333] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:19:52.442804  2729 raft_consensus.cc:2243] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:52.458329  2729 raft_consensus.cc:2272] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:52.479806 28283 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:52.480567 28283 raft_consensus.cc:606] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:52.481786  2729 ts_tablet_manager.cc:1916] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:52.484364 28283 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:52.485049 28283 raft_consensus.cc:606] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
W20260709 12:19:52.490105 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46805.2us
I20260709 12:19:52.500350 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:52.501124 28359 raft_consensus.cc:606] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:52.484390  2719 ts_tablet_manager.cc:1929] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.12
I20260709 12:19:52.505715  2719 log.cc:1199] T d68008d2b3fd4f7bb72bf69f612b2998 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/d68008d2b3fd4f7bb72bf69f612b2998
W20260709 12:19:52.512768 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 52214.3us
I20260709 12:19:52.515774 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet d68008d2b3fd4f7bb72bf69f612b2998 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:52.509831  2719 tablet_replica.cc:333] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:19:52.542068 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:19:52.544486 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:52.547503 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:52.553890  2729 ts_tablet_manager.cc:1929] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.13
I20260709 12:19:52.554490  2729 log.cc:1199] T 8ab1586b52be49808b90fcc88bdf2d60 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/8ab1586b52be49808b90fcc88bdf2d60
I20260709 12:19:52.556627  2647 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 10 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.559671 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 8ab1586b52be49808b90fcc88bdf2d60 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:52.583329 28989 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 10 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.609997  2719 raft_consensus.cc:2243] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:52.612125  2719 raft_consensus.cc:2272] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:52.619477  2719 ts_tablet_manager.cc:1916] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:52.635842 28359 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 10 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.676198 28439 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 10 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.686162  2647 raft_consensus.cc:993] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:52.689334  2727 raft_consensus.cc:1081] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:52.698352 28437 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:19:52.698998 28437 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:52.706164 28437 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:52.705613 28181 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 10 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:52.726677  2719 ts_tablet_manager.cc:1929] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.10
I20260709 12:19:52.727234  2719 log.cc:1199] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/8e6bd3eccb6c4f4c8fedbc76fa829a96
I20260709 12:19:52.736213 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet 8e6bd3eccb6c4f4c8fedbc76fa829a96 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:52.739873  2663 raft_consensus.cc:993] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:52.740494  2663 raft_consensus.cc:1081] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:52.743402 28438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:19:52.743981 28438 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:52.744340 28438 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:19:52.759171 28437 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:52.763195 28437 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:52.764309 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:52.765452 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:52.774080 28287 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:52.774412 28287 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:52.773557 28359 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:52.772404 28283 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:52.776579 28438 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:52.778512 28287 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:52.781930 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:52.782666 28288 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:19:52.795066 28287 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:52.796339 28439 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:52.807675 28989 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:52.808537  2724 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:52.809892  2732 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:52.817516 28359 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:52.820297 28287 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:19:52.821038 28438 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:19:52.821972 28288 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 26.
I20260709 12:19:52.824074 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:52.824771 28358 raft_consensus.cc:3055] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:19:52.825062 28358 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:52.833012 28358 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:52.827237 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:52.835569  2530 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:52.836216  2530 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:52.837291  2530 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:52.839468 28989 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.837543  2732 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.861768  2647 raft_consensus.cc:993] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:52.862493  2724 raft_consensus.cc:1081] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:52.866343 28359 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55954
I20260709 12:19:52.867012 28359 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:52.867372 28359 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:52.886226 28438 raft_consensus.cc:2955] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.890937 28358 raft_consensus.cc:3060] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:19:52.877511 28404 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:19:52.899230  2731 raft_consensus.cc:2804] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:19:52.899825  2731 raft_consensus.cc:697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 26 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:52.900781  2731 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:52.909849 28181 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 12 to 13, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:52.926640 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c474098832a04762b58521bdc2a92b5a with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:19:52.935376 28358 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 26.
I20260709 12:19:52.938742 28359 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:52.943065 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:52.944725 28438 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:52.949136 28359 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:52.956173 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:52.956884 28288 raft_consensus.cc:3055] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:52.957190 28288 raft_consensus.cc:740] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:52.958122 28288 consensus_queue.cc:260] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:52.959609 28288 raft_consensus.cc:3060] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:52.976576 28438 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:52.978103 28332 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:52.978962  2663 raft_consensus.cc:2804] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:52.980011 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet c474098832a04762b58521bdc2a92b5a with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:39084
I20260709 12:19:52.980593  2663 raft_consensus.cc:697] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:52.981810  2663 consensus_queue.cc:237] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:52.991679 28186 catalog_manager.cc:5697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 16 to 17, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 17 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:53.003499  2737 tablet_replica.cc:333] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:19:53.011593  2737 raft_consensus.cc:2243] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:53.028005  2737 raft_consensus.cc:2272] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:53.035669  2737 ts_tablet_manager.cc:1916] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:53.042954 28288 raft_consensus.cc:2468] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:19:53.051061 28181 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 25 to 26, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 26 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:53.057541 28186 catalog_manager.cc:5697] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 15 to 16, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 16 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:53.066833  2737 ts_tablet_manager.cc:1929] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.12
I20260709 12:19:53.070700  2737 log.cc:1199] T c474098832a04762b58521bdc2a92b5a P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/c474098832a04762b58521bdc2a92b5a
I20260709 12:19:53.088001 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet c474098832a04762b58521bdc2a92b5a (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:19:53.340436 28288 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:53.342280  2721 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:53.360576 28360 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:53.375049  2720 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:53.397575 28288 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:53.399477  2596 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:53.416155 28358 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:19:53.439392  2645 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:53.517675 28287 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:19:53.543022 28438 raft_consensus.cc:1275] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:53.562881  2734 consensus_queue.cc:1048] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:19:53.579274  2731 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
W20260709 12:19:53.633225 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 67300.3us
W20260709 12:19:53.698474 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47943.9us
W20260709 12:19:53.701887 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 66306us
W20260709 12:19:53.746498 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 76569.5us
W20260709 12:19:53.765990 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50309.5us
W20260709 12:19:53.788983 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46132.8us
W20260709 12:19:53.879515 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50466.6us
W20260709 12:19:53.906566 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 72860.3us
W20260709 12:19:53.916142 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 102937us
W20260709 12:19:53.960816 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 152223us
W20260709 12:19:53.947278  2742 log.cc:927] Time spent T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:19:53.993167 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:54.146797 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 100235us
I20260709 12:19:54.164546 28360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:54.165539 28360 raft_consensus.cc:606] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:19:54.223789 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45764.6us
W20260709 12:19:54.284516 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50700.2us
W20260709 12:19:54.287497 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 125729us
W20260709 12:19:54.291585 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 364887us
W20260709 12:19:54.345999 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 57720.8us
I20260709 12:19:54.349833  2755 raft_consensus.cc:993] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:54.350368  2755 raft_consensus.cc:1081] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:54.355556 28435 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:19:54.364665 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 47360.8us
I20260709 12:19:54.402943 28437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:54.403633 28437 raft_consensus.cc:606] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:54.405457 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:54.407693 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:54.408560 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:54.410382 28366 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:19:54.411654 28404 consensus_peers.cc:597] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9cdd1e39ef1841c382e814af04ea5bc9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:54.416910  2782 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:54.462599 28287 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59778
I20260709 12:19:54.463275 28287 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:54.463665 28287 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:54.464888 28284 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:54.479481 28287 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:54.491314  2766 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:54.496024 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:54.496773 28361 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:54.497067 28361 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:54.497898 28361 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:54.499034 28361 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:54.510665 28287 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:54.513056 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:54.513775 28435 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:54.519630 28361 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:19:54.523360 28435 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
W20260709 12:19:54.499239  2789 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.064s	user 0.005s	sys 0.012s
I20260709 12:19:54.529793 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:19:54.586902 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 61363us
W20260709 12:19:54.587420 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:54.590651 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 60991.3us
I20260709 12:19:54.606726  2754 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:54.610183 28366 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:54.649154  2770 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:54.671396  2770 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:54.691566 28181 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 9, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 13 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:54.716434  2770 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:54.707914 28290 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:54.752705 28291 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:54.763149 28186 catalog_manager.cc:5697] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 20 to 21, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 21 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:54.776207 28435 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:19:54.776665 28255 consensus_peers.cc:597] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b4036b67205e41c597049a10a71e9467. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:54.778048  2770 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:54.778765 28361 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:54.784433  2774 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:54.855399  2770 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Committing config change with OpId 19.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:54.868021 28360 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:54.884961  2530 raft_consensus.cc:993] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:54.885550  2530 raft_consensus.cc:1081] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:54.888504 28366 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:54.889171 28366 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:54.889575 28366 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.019470 28366 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:55.052966 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:55.053779 28437 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:55.054136 28437 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:55.054981 28437 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:55.055918 28437 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.052539 28366 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:55.071763 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:55.072625 28989 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.098749 28989 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:19:55.106065  2799 ts_tablet_manager.cc:933] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:55.164990 28330 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:55.168315  2784 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:55.169310  2784 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:55.170509  2784 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:55.188656 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 141411us
I20260709 12:19:55.196990 28437 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:19:55.221372 28360 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:19:55.253324  2774 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:55.258818 28439 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:55.244381 28186 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index -1 to 11, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 19 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:55.278744  2799 tablet_copy_client.cc:323] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
W20260709 12:19:55.329501 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 67573us
I20260709 12:19:55.352398 28185 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 19 to 20, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 20 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:55.427039 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 185535us
I20260709 12:19:55.430663 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 9cdd1e39ef1841c382e814af04ea5bc9 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:19:55.431380 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 9cdd1e39ef1841c382e814af04ea5bc9 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-9cdd1e39ef1841c382e814af04ea5bc9
I20260709 12:19:55.460031  2815 ts_tablet_manager.cc:933] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:19:55.494747 28291 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:55.498399 28451 tablet_copy_source_session.cc:215] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:19:55.512624 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45450.4us
W20260709 12:19:55.546644 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 72342.3us
I20260709 12:19:55.554620  2799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdd1e39ef1841c382e814af04ea5bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:55.596814  2815 tablet_copy_client.cc:323] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
W20260709 12:19:55.594640 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 138519us
I20260709 12:19:55.604012 28302 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet b4036b67205e41c597049a10a71e9467 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:19:55.604831 28302 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet b4036b67205e41c597049a10a71e9467 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-b4036b67205e41c597049a10a71e9467
W20260709 12:19:55.625300 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 156844us
I20260709 12:19:55.646251 28302 tablet_copy_source_session.cc:215] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:55.665012 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:55.671815  2815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4036b67205e41c597049a10a71e9467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:55.679401 28361 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:55.683660 28435 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:19:55.690364 28439 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:19:55.691098  2806 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:55.692744 28435 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:55.692577  2752 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:19:55.690395 28255 consensus_peers.cc:597] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bce32c3c49414e228fe6d912d95aeaac. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:55.710183  2799 tablet_copy_client.cc:806] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:55.710930  2799 tablet_copy_client.cc:670] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:55.744141 28366 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:55.736487  2803 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:55.754208  2806 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:55.759212  2806 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:55.760434 28984 raft_consensus.cc:1275] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:19:55.765921  2752 consensus_queue.cc:1048] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:55.769439  2799 tablet_copy_client.cc:538] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:55.836539 28185 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 13, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 20 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:55.854844  2799 tablet_bootstrap.cc:492] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:19:55.902124 28439 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:55.860168  2815 tablet_copy_client.cc:806] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:55.910312  2815 tablet_copy_client.cc:670] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:55.917112  2815 tablet_copy_client.cc:538] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:55.940388 28438 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:19:55.975947 28404 consensus_peers.cc:597] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c99f01ab7374acfa37f7669b8fcdeb2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:19:55.980271 28366 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:19:55.987320  2530 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:55.999715 28288 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:19:56.012171  2754 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:19:56.059341 28361 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:19:56.060012 28361 raft_consensus.cc:606] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 24 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:19:56.067006 28291 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:56.067641 28291 raft_consensus.cc:606] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:56.069890 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:56.073537 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:56.078099  2815 tablet_bootstrap.cc:492] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:19:56.083060 28366 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Committing config change with OpId 26.17: config changed from index -1 to 17, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:56.086232 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:56.094760  2820 raft_consensus.cc:993] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:56.095397  2784 raft_consensus.cc:1081] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 24 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:19:56.080113  2754 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 26 LEADER]: Committing config change with OpId 26.17: config changed from index -1 to 17, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:56.138870 28183 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 17, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 26 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:56.152058 28288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59778
I20260709 12:19:56.154320 28288 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:56.154692 28288 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:19:56.156384  2799 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:19:56.180882 28287 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Committing config change with OpId 26.17: config changed from index -1 to 17, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:19:56.232751 28288 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:56.235905  2799 tablet_bootstrap.cc:492] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:56.238462 28288 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 25 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:56.245817 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:56.246403 28986 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:56.247558 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:56.248153 28366 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:19:56.248441 28366 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:56.250640 28366 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:56.252918 28366 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:56.262480  2799 tablet_bootstrap.cc:492] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:56.264132  2799 ts_tablet_manager.cc:1403] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.410s	user 0.041s	sys 0.027s
I20260709 12:19:56.275403  2799 raft_consensus.cc:359] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:56.287227  2799 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:56.288132  2799 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:56.308037  2799 ts_tablet_manager.cc:1434] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.003s
I20260709 12:19:56.298264 28986 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 25.
I20260709 12:19:56.285238 28366 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 25.
I20260709 12:19:56.309573 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:56.310499  2727 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:19:56.311005  2727 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:56.311880  2727 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:56.314028 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-9cdd1e39ef1841c382e814af04ea5bc9 received from {username='slave'} at 127.0.0.1:53280
I20260709 12:19:56.314514 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-9cdd1e39ef1841c382e814af04ea5bc9 on tablet 9cdd1e39ef1841c382e814af04ea5bc9 with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:56.297289  2841 ts_tablet_manager.cc:933] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113)
I20260709 12:19:56.348500 28186 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 24 to 25, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 25 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:56.374646  2815 tablet_bootstrap.cc:492] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:56.375783  2815 tablet_bootstrap.cc:492] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:56.376547  2815 ts_tablet_manager.cc:1403] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.300s	user 0.022s	sys 0.026s
I20260709 12:19:56.383625 28984 raft_consensus.cc:1217] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.8->[13.9-13.9]   Dedup: 13.9->[]
I20260709 12:19:56.396919  2799 ts_tablet_manager.cc:933] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:56.398067  2841 tablet_copy_client.cc:323] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.1:42113
I20260709 12:19:56.403802 28302 tablet_copy_service.cc:140] P a3de34b6f4cf4606a61944db6e2739c8: Received BeginTabletCopySession request for tablet bce32c3c49414e228fe6d912d95aeaac from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:39592)
I20260709 12:19:56.404371 28302 tablet_copy_service.cc:161] P a3de34b6f4cf4606a61944db6e2739c8: Beginning new tablet copy session on tablet bce32c3c49414e228fe6d912d95aeaac from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:39592: session id = f526369c0d0743b192be6e205540a059-bce32c3c49414e228fe6d912d95aeaac
I20260709 12:19:56.438721  2799 tablet_copy_client.cc:323] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:19:56.441262 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 9c99f01ab7374acfa37f7669b8fcdeb2 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:19:56.441865 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 9c99f01ab7374acfa37f7669b8fcdeb2 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-9c99f01ab7374acfa37f7669b8fcdeb2
I20260709 12:19:56.456147  2806 raft_consensus.cc:993] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:56.456676  2806 raft_consensus.cc:1081] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:56.455942  2815 raft_consensus.cc:359] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:56.457221  2815 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:56.457950  2815 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:56.468425 28438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:19:56.469125 28438 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:56.469913 28438 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:56.477411 28451 tablet_copy_source_session.cc:215] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:56.493781  2799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c99f01ab7374acfa37f7669b8fcdeb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:56.494455  2815 ts_tablet_manager.cc:1434] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.118s	user 0.007s	sys 0.000s
I20260709 12:19:56.496693 28301 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-b4036b67205e41c597049a10a71e9467 received from {username='slave'} at 127.0.0.1:39592
I20260709 12:19:56.497248 28301 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-b4036b67205e41c597049a10a71e9467 on tablet b4036b67205e41c597049a10a71e9467 with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:56.507297 28438 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:56.511003 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:56.511626 28288 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:56.511914 28288 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:56.512650 28288 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:56.514079 28288 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:56.522707 28302 tablet_copy_source_session.cc:215] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:19:56.526871  2841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bce32c3c49414e228fe6d912d95aeaac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:56.537395 28438 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:56.538574 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:56.539178 28984 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:56.547350 28986 raft_consensus.cc:1217] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: Deduplicated request from leader. Original: 19.10->[19.11-19.11]   Dedup: 19.11->[]
I20260709 12:19:56.583653 28984 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:56.594398 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:56.595268  2530 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:56.595755  2530 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:56.596563  2530 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:56.614308 28288 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:19:56.615921  2799 tablet_copy_client.cc:806] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:56.619534  2799 tablet_copy_client.cc:670] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:56.642262  2799 tablet_copy_client.cc:538] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:56.643826 28185 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 17 to 18, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 18 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:56.655588  2841 tablet_copy_client.cc:806] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:19:56.656270  2841 tablet_copy_client.cc:670] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:19:56.695792  2841 tablet_copy_client.cc:538] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:19:56.708896  2799 tablet_bootstrap.cc:492] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:19:56.717149  2841 tablet_bootstrap.cc:492] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:19:56.746125 28984 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:19:56.748976  2770 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:19:57.039135 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 120855us
W20260709 12:19:57.130627 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 107595us
I20260709 12:19:57.144093 28366 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:19:57.174589  2799 tablet_bootstrap.cc:492] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:57.180529 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:57.191684 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 81078.9us
I20260709 12:19:57.220331  2799 tablet_bootstrap.cc:492] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:57.224117  2799 ts_tablet_manager.cc:1403] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.515s	user 0.054s	sys 0.044s
W20260709 12:19:57.254383 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 74825.1us
I20260709 12:19:57.285977 28984 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:57.260397  2821 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.089s
W20260709 12:19:57.392736 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 130598us
I20260709 12:19:57.437703  2799 raft_consensus.cc:359] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:57.439126  2799 raft_consensus.cc:740] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:57.439814  2799 consensus_queue.cc:260] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:19:57.464257 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 69471.8us
I20260709 12:19:57.467253  2799 ts_tablet_manager.cc:1434] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.243s	user 0.021s	sys 0.011s
I20260709 12:19:57.519398  2875 raft_consensus.cc:1064] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
W20260709 12:19:57.547703 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 92305.2us
W20260709 12:19:57.698071 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 225256us
W20260709 12:19:57.699513  2841 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:19:57.716198 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-9c99f01ab7374acfa37f7669b8fcdeb2 received from {username='slave'} at 127.0.0.1:53280
I20260709 12:19:57.728410 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-9c99f01ab7374acfa37f7669b8fcdeb2 on tablet 9c99f01ab7374acfa37f7669b8fcdeb2 with peer f526369c0d0743b192be6e205540a059
W20260709 12:19:57.744597 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 226546us
I20260709 12:19:57.778564  2862 raft_consensus.cc:1064] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:57.781190  2862 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:57.783169  2869 consensus_queue.cc:1175] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6634fa3fbd594726a5ec3c1686900284" responder_term: 13 status { last_received { term: 13 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 13 index: 9 } }
I20260709 12:19:57.758662  2754 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
W20260709 12:19:57.797405 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44026.9us
W20260709 12:19:57.801704 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44383.7us
I20260709 12:19:57.800112  2875 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 9, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:19:57.840001 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 48439.3us
I20260709 12:19:57.872601 28991 raft_consensus.cc:1217] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 LEARNER]: Deduplicated request from leader. Original: 26.16->[26.17-26.17]   Dedup: 26.17->[]
I20260709 12:19:57.877293  2841 tablet_bootstrap.cc:492] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:19:57.898547  2841 tablet_bootstrap.cc:492] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:19:57.899554  2841 ts_tablet_manager.cc:1403] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 1.183s	user 0.048s	sys 0.036s
I20260709 12:19:57.915983 28288 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:19:57.935990 28439 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:19:57.936942 28439 raft_consensus.cc:606] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:19:58.005029 28288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:19:58.005785 28288 raft_consensus.cc:606] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:58.019804 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:58.022267 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:58.023938 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:58.009718  2841 raft_consensus.cc:359] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:58.046803  2841 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:19:58.047606  2841 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:58.072394 28437 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:58.072394 28985 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:58.082474  2878 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:58.082926 28405 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:58.083926  2890 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:58.087826  2841 ts_tablet_manager.cc:1434] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.188s	user 0.005s	sys 0.004s
I20260709 12:19:58.098316 28302 tablet_copy_service.cc:342] P a3de34b6f4cf4606a61944db6e2739c8: Request end of tablet copy session f526369c0d0743b192be6e205540a059-bce32c3c49414e228fe6d912d95aeaac received from {username='slave'} at 127.0.0.1:39592
I20260709 12:19:58.098838 28302 tablet_copy_service.cc:434] P a3de34b6f4cf4606a61944db6e2739c8: ending tablet copy session f526369c0d0743b192be6e205540a059-bce32c3c49414e228fe6d912d95aeaac on tablet bce32c3c49414e228fe6d912d95aeaac with peer f526369c0d0743b192be6e205540a059
I20260709 12:19:58.111261 28366 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:58.122812  2878 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:19:58.148279 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 172395us
I20260709 12:19:58.158906 28360 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:58.167464 28986 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:58.169145 28292 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:58.178050  2878 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:58.183564  2890 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.187855  2890 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:58.184844 28366 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:58.192391 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:58.194079  2530 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:58.195235  2890 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:19:58.210104 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:19:58.213199  2883 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:58.217193 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:19:58.230592  2890 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:58.231832  2853 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:58.221570 28432 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:58.270289  2887 raft_consensus.cc:993] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:58.271399  2887 raft_consensus.cc:1081] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:19:58.274655 28439 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33110
I20260709 12:19:58.275256 28439 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:58.275616 28439 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:58.299048 28439 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:58.317934 28186 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:58.327258 28439 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:19:58.333932 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 52820.8us
I20260709 12:19:58.355136 28986 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index 11 to 12, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:58.376873 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:19:58.377638 28290 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:19:58.377986 28290 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:19:58.391379 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:19:58.392031 28359 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:58.395537  2883 raft_consensus.cc:1064] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:58.395761 28986 raft_consensus.cc:1217] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.12->[20.13-20.13]   Dedup: 20.13->[]
I20260709 12:19:58.450218 28359 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 24.
W20260709 12:19:58.451582  2905 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:19:58.381075  2853 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:19:58.390434 28290 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:58.502521  2883 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 15, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:58.506676  2869 raft_consensus.cc:993] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:58.507467  2869 raft_consensus.cc:1081] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:19:58.512621 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:19:58.513902  2530 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:19:58.514385  2530 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:58.515218  2530 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:58.517781 28365 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:19:58.518433 28365 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:58.518812 28365 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:58.524544  2754 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:58.526615 28290 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:58.551419 28290 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 24.
I20260709 12:19:58.556066 28986 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:58.554138 28290 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:58.540786 28365 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:58.824108 28365 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:19:58.825786 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 250255us
I20260709 12:19:59.084069 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:19:59.086594 28989 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:59.173921  2905 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.002s
I20260709 12:19:59.165624 28989 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:59.234105 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:59.269358 28990 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:19:59.270124 28985 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:59.276173  2883 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:19:59.296752 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:19:59.298089 28438 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:59.298620 28438 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:19:59.300133 28438 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:59.304322  2863 raft_consensus.cc:1064] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:19:59.312408  2863 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:59.323249  2900 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:59.329276 28438 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:59.350847 28365 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:19:59.351527 28364 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:19:59.351485 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:19:59.360548 28438 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:19:59.359910  2926 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:59.379868  2926 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:19:59.359922 28186 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 320) took 1049 ms. Trace:
I20260709 12:19:59.386507 28364 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:59.381323  2926 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:59.412468  2890 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:59.411558 28183 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 9 to 10, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:59.413012 28185 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 308) took 1044 ms. Trace:
I20260709 12:19:59.421403 28185 rpcz_store.cc:276] 0709 12:19:58.368628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.368904 (+   276us) service_pool.cc:224] Handling call
0709 12:19:59.412812 (+1043908us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":250112}
I20260709 12:19:59.422338  2811 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:19:59.423182  2811 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:19:59.382027 28186 rpcz_store.cc:276] 0709 12:19:58.310334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.310515 (+   181us) service_pool.cc:224] Handling call
0709 12:19:59.359659 (+1049144us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:58.496557 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:58.497158 (+   601us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:58.497199 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:19:58.497222 (+    23us) tablet.cc:662] Decoding operations
0709 12:19:58.497582 (+   360us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:58.497658 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:58.497690 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:19:58.497888 (+   198us) tablet.cc:701] Row locks acquired
0709 12:19:58.497906 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:19:58.498048 (+   142us) write_op.cc:270] Start()
0709 12:19:58.498168 (+   120us) write_op.cc:276] Timestamp: P: 1783599598498002 usec, L: 0
0709 12:19:58.498191 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:19:58.498898 (+   707us) log.cc:844] Serialized 595 byte log entry
0709 12:19:59.162141 (+663243us) op_driver.cc:464] REPLICATION: finished
0709 12:19:59.208408 (+ 46267us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88432a 00007f85be809f14 00007f85be809ec3 00007f85be809e38 00007f85be809df3 00007f85be8090dc 00007f85be832e16 00007f85be81acbb 00007f85be80539d 00007f85be808d75
0709 12:19:59.318647 (+110239us) write_op.cc:301] APPLY: starting
0709 12:19:59.318818 (+   171us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:59.342718 (+ 23900us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:59.342766 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:59.343752 (+   986us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:59.344147 (+   395us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:19:59.344232 (+    85us) write_op.cc:312] APPLY: finished
0709 12:19:59.344616 (+   384us) log.cc:844] Serialized 25 byte log entry
0709 12:19:59.346181 (+  1565us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:59.346324 (+   143us) write_op.cc:454] Released schema lock
0709 12:19:59.346386 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:59.347200 (+   814us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:59.347240 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":156080,"apply.run_cpu_time_us":5578,"apply.run_wall_time_us":28492,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8305,"num_ops":1,"prepare.queue_time_us":163125,"prepare.run_cpu_time_us":84228,"prepare.run_wall_time_us":331208,"raft.queue_time_us":157248,"raft.run_cpu_time_us":33709,"raft.run_wall_time_us":124767,"replication_time_us":663585,"spinlock_wait_cycles":135428352,"thread_start_us":445970,"threads_started":4,"wal-append.queue_time_us":399655}]]}
I20260709 12:19:59.428525 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:19:59.413302  2890 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:19:59.446312 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:19:59.467979 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.481832  2811 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:59.543097 28365 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:59.567549  2890 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:19:59.569869 28986 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 LEARNER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:59.576105  2863 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:59.583772 28285 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:19:59.590363 28365 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:59.592887  2928 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:59.595801  2924 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 12:19:59.600090 28284 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:19:59.610342 28437 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:19:59.610610 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:59.611316 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:19:59.612818  2924 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
W20260709 12:19:59.629920 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50707.1us
I20260709 12:19:59.673369 28288 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:59.678025 28186 catalog_manager.cc:5697] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 23 to 24, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 24 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:59.660831 28185 catalog_manager.cc:5697] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 18 to 19, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 19 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:59.695482 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:59.696921 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:19:59.697825  2890 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:59.699880  2890 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:19:59.701002  2890 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:59.719305 28986 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:59.721005  2928 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:19:59.708865 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 88843.2us
I20260709 12:19:59.756034 28438 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:19:59.774675 28360 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:59.787211  2928 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:59.794994  2754 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:59.799641  2883 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:19:59.821389  2943 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.059s	user 0.001s	sys 0.006s
W20260709 12:19:59.826118  2944 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.062s	user 0.004s	sys 0.007s
W20260709 12:19:59.827562  2942 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.056s	user 0.002s	sys 0.007s
I20260709 12:19:59.864971  2878 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 13 to 14, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:59.901984  2900 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 26 LEADER]: Committing config change with OpId 26.18: config changed from index 17 to 18, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:59.934317 28435 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:59.956279 28435 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:59.964849 28989 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:59.966455 28360 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:59.966670 28358 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:19:59.969285  2752 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:59.956279  2890 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:00.010694  2884 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
W20260709 12:20:00.124795 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.130340  2947 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 12:20:00.170608 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 172214us
I20260709 12:20:00.172480 28991 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:20:00.183660  2954 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.053s	user 0.003s	sys 0.000s
I20260709 12:20:00.203595 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b4036b67205e41c597049a10a71e9467 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:00.206303 28181 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: config changed from index 13 to 14, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:00.216576 28987 raft_consensus.cc:2955] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:20:00.252260 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 161644us
I20260709 12:20:00.260739 28439 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:20:00.261664 28439 raft_consensus.cc:606] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:00.276667 28439 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:20:00.283053 28360 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Committing config change with OpId 26.18: config changed from index 17 to 18, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:00.277894 28439 raft_consensus.cc:606] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:00.298424 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:00.334034 28361 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:20:00.342048 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:00.343439 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:00.352662 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 51859.1us
I20260709 12:20:00.384701 28987 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:20:00.389853 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 76033.6us
W20260709 12:20:00.482839 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 70446.2us
W20260709 12:20:00.483601 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 83443.8us
W20260709 12:20:00.510771 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 63429.2us
W20260709 12:20:00.524549 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 115058us
W20260709 12:20:00.640136  2959 log.cc:927] Time spent T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 12:20:00.645295 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 79923.3us
W20260709 12:20:00.744304 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 78151.1us
W20260709 12:20:00.863317 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 308434us
I20260709 12:20:00.939289  2953 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:00.940121  2953 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:00.987190  2963 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:20:00.997246  2953 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:00.988538  2964 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:00.978459  2926 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:01.025409  2926 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:01.028157  2926 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:01.035246 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 245739us
W20260709 12:20:01.067482 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 187031us
I20260709 12:20:01.073530 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.022374  2964 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:01.085248  2912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.000s	sys 0.011s
I20260709 12:20:01.086123  2912 raft_consensus.cc:993] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:20:01.086666  2964 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:01.086666  2912 raft_consensus.cc:1081] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Signalling peer 6634fa3fbd594726a5ec3c1686900284 to start an election
I20260709 12:20:01.098518  2883 raft_consensus.cc:993] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:20:01.120672  2883 raft_consensus.cc:1081] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
W20260709 12:20:01.175400 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 60222us
W20260709 12:20:01.202446 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 194907us
I20260709 12:20:01.217655  2929 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:01.218470  2929 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:01.221129  2929 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:01.224074 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.151572  2840 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:01.285605 28984 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Committing config change with OpId 26.18: config changed from index 17 to 18, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:01.238454  2953 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
W20260709 12:20:01.333397 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45366.2us
I20260709 12:20:01.301411 28181 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 311) took 1199 ms. Trace:
I20260709 12:20:01.300246 28186 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 12 to 13, VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) evicted. New cstate: current_term: 19 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:01.346415 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:01.363135 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.363425 28362 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:55946
I20260709 12:20:01.323086  2921 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:01.374974 28288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59792
I20260709 12:20:01.375635 28288 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:01.376003 28288 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:01.381676  2921 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.384848  2921 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:01.271957  2840 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.387812  2840 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.403211 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.404939 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.405285  2929 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:01.294941 28290 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Committing config change with OpId 26.18: config changed from index 17 to 18, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:01.411334 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.409622 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.422957 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.431252 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.434713 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.435599 28991 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:20:01.447911 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:01.336889 28181 rpcz_store.cc:276] 0709 12:20:00.102438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.102620 (+   182us) service_pool.cc:224] Handling call
0709 12:20:01.301336 (+1198716us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:00.385836 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:00.386397 (+   561us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:00.386439 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:20:00.386457 (+    18us) tablet.cc:662] Decoding operations
0709 12:20:00.386793 (+   336us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:00.386868 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:00.386900 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:00.387088 (+   188us) tablet.cc:701] Row locks acquired
0709 12:20:00.387105 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:20:00.387249 (+   144us) write_op.cc:270] Start()
0709 12:20:00.387362 (+   113us) write_op.cc:276] Timestamp: P: 1783599600387200 usec, L: 0
0709 12:20:00.387382 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:20:00.388108 (+   726us) log.cc:844] Serialized 595 byte log entry
0709 12:20:00.477036 (+ 88928us) op_driver.cc:464] REPLICATION: finished
0709 12:20:00.878160 (+401124us) write_op.cc:301] APPLY: starting
0709 12:20:00.881270 (+  3110us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:01.024291 (+143021us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:01.024393 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:01.025214 (+   821us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:01.025463 (+   249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:01.025509 (+    46us) write_op.cc:312] APPLY: finished
0709 12:20:01.025764 (+   255us) log.cc:844] Serialized 25 byte log entry
0709 12:20:01.025942 (+   178us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:01.026038 (+    96us) write_op.cc:454] Released schema lock
0709 12:20:01.026080 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:01.026540 (+   460us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:01.026587 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":172005,"reported_tablets":1,"spinlock_wait_cycles":6016,"child_traces":[["op",{"apply.queue_time_us":313579,"apply.run_cpu_time_us":4284,"apply.run_wall_time_us":162161,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8305,"num_ops":1,"prepare.queue_time_us":174156,"prepare.run_cpu_time_us":2604,"prepare.run_wall_time_us":2603,"replication_time_us":89452,"thread_start_us":100866,"threads_started":2}]]}
I20260709 12:20:01.446318  3002 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:01.480110  3002 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.484295 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.418006  2929 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.478255  2934 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:01.528146 28283 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:20:01.488219  2890 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:01.489897  2981 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:01.564699  3002 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:01.565570  2981 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.528565  2934 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.559733  2890 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:01.573089  2934 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:01.574554  3011 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:20:01.588836 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.595981 28362 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:01.596832 28288 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.596477 28362 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:01.595585 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:01.608512 28288 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.618345 28330 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:01.608749  2934 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:01.616168  2993 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:01.627522 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.633514 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:01.636816  2931 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:01.655644  2931 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:01.656059  2931 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:01.635010  2890 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:01.664948 28362 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:01.678285 28332 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:01.680269 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:01.680861 28432 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:01.638404  2934 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:01.681118 28432 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:01.684473 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.686625 28404 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:01.689043 28432 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.695499 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.697113 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:01.699429  2934 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:01.687722  2926 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:01.702942 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.696965  2866 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:01.713517  2926 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.714462  2955 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:01.715217  2955 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.715898 28956 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:01.718720  2977 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:01.690994  3003 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:01.720388  2990 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:01.720898  3003 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.724678 28366 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:01.733502 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:01.740952  2931 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.741726  2981 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.742172  2955 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.743335  2931 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.740732  2933 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:01.755532 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:01.748648 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 55434.3us
I20260709 12:20:01.762944 28432 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:01.772089  2926 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.776161 28432 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:01.789618 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.784312  3003 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.791508 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:01.792863  2929 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.795411 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.795698 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.798344 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.798825 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:01.799103 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.799736 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.791900 28954 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:01.803103  2985 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:01.803506 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.806921  2840 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:01.807569  2840 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.813241  2840 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.817464 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:01.819871 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:01.822166 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.824080 28362 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:01.817255 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:01.825919  2931 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:20:01.826283  2884 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:01.826743 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:01.830230 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.830346  2840 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:01.834452 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:01.834884 28362 raft_consensus.cc:1240] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:01.838359 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:01.840827 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:01.831499  2900 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:01.876699 28286 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.828140 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:20:01.878398 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50346.5us
I20260709 12:20:01.879987  2962 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:01.880595  2962 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:01.881536  2962 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.882863 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:01.886595 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:01.887190 28291 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:01.888396  2986 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:01.870464 28441 raft_consensus.cc:3055] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:20:01.871944 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.891098 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:01.867673 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.890553 28441 raft_consensus.cc:740] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:01.897836 28441 consensus_queue.cc:260] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:01.899081 28441 raft_consensus.cc:3060] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:01.907620 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:01.908995 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:01.910126 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:01.910640 28991 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:01.905666 28332 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:01.920477 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.930285 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.924793  2955 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:01.923167  2993 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:01.936604  2993 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:01.938925  2993 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:01.940816 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:01.948498 28991 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:01.952524 28186 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 321) took 1722 ms. Trace:
I20260709 12:20:01.952986 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet b4036b67205e41c597049a10a71e9467 with delete_type TABLET_DATA_TOMBSTONED (TS 9aea2391841b4888b4e195e5a1f4623b not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:33064
I20260709 12:20:01.953042 28186 rpcz_store.cc:276] 0709 12:20:00.230291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.230699 (+   408us) service_pool.cc:224] Handling call
0709 12:20:01.952449 (+1721750us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:01.558606 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:01.577541 (+ 18935us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:01.577589 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:20:01.577602 (+    13us) tablet.cc:662] Decoding operations
0709 12:20:01.578018 (+   416us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:01.578087 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:01.578112 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:01.578311 (+   199us) tablet.cc:701] Row locks acquired
0709 12:20:01.578329 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:20:01.578492 (+   163us) write_op.cc:270] Start()
0709 12:20:01.578616 (+   124us) write_op.cc:276] Timestamp: P: 1783599601578436 usec, L: 0
0709 12:20:01.578637 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:20:01.610825 (+ 32188us) log.cc:844] Serialized 533 byte log entry
0709 12:20:01.783993 (+173168us) op_driver.cc:464] REPLICATION: finished
0709 12:20:01.820276 (+ 36283us) write_op.cc:301] APPLY: starting
0709 12:20:01.820349 (+    73us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:01.821430 (+  1081us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:01.821512 (+    82us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:01.922930 (+101418us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:01.934218 (+ 11288us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:01.934275 (+    57us) write_op.cc:312] APPLY: finished
0709 12:20:01.934495 (+   220us) log.cc:844] Serialized 25 byte log entry
0709 12:20:01.934690 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:01.934789 (+    99us) write_op.cc:454] Released schema lock
0709 12:20:01.934833 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:01.935323 (+   490us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:01.935358 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":36054,"apply.run_cpu_time_us":3806,"apply.run_wall_time_us":114984,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8305,"num_ops":1,"prepare.queue_time_us":150487,"prepare.run_cpu_time_us":3401,"prepare.run_wall_time_us":57685,"replication_time_us":205194,"thread_start_us":110056,"threads_started":2}]]}
W20260709 12:20:01.979768 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 54002.4us
I20260709 12:20:02.476118  3023 tablet_replica.cc:333] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
W20260709 12:20:02.585126  3022 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 12:20:02.565351  2900 consensus_meta.cc:287] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.002s
I20260709 12:20:02.585403  3023 raft_consensus.cc:2243] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:02.799873  3023 raft_consensus.cc:2272] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:02.916610  3023 ts_tablet_manager.cc:1916] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:20:03.567649  2900 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.006s	sys 0.001s
W20260709 12:20:03.568310  2900 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.006s	sys 0.001s
I20260709 12:20:03.841095  2900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:33064 (request call id 11) took 4117 ms. Trace:
I20260709 12:20:03.841697  2900 rpcz_store.cc:276] 0709 12:19:59.723955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.728966 (+  5011us) service_pool.cc:224] Handling call
0709 12:19:59.754942 (+ 25976us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b58005150d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7e30aa 00007f85be865447 00007f85be88270a 00007f85be88376a 00007f85be88291b 00007f85be88e03c 00007f85be88d6b3 00007f85be8859fa 00007f85c384d4c9 00007f85bcdc9c54
0709 12:19:59.819300 (+ 64358us) log.cc:844] Serialized 494 byte log entry
0709 12:20:00.236115 (+416815us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b54002484a0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be800142 00007f85be7e0eaa 00007f85be7e7d53 00007f85be7e7cd9 00007f85be7e7c69 00007f85be7e7c31 00007f85be7e6dfd 0000000000388634 0000000000388479 00007f85bb79a475
0709 12:20:03.840926 (+3604811us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9945,"raft.run_cpu_time_us":3439,"raft.run_wall_time_us":276631,"spinlock_wait_cycles":36846336,"thread_start_us":158680,"threads_started":1,"wal-append.queue_time_us":267571,"wal-append.run_cpu_time_us":1850,"wal-append.run_wall_time_us":1005561}
I20260709 12:20:03.946151  3023 ts_tablet_manager.cc:1929] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.12
I20260709 12:20:03.953045  3023 log.cc:1199] T b4036b67205e41c597049a10a71e9467 P 9aea2391841b4888b4e195e5a1f4623b: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/wals/b4036b67205e41c597049a10a71e9467
I20260709 12:20:03.955973  3023 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33064 (request call id 12) took 2003 ms. Trace:
I20260709 12:20:03.958406  3023 rpcz_store.cc:276] 0709 12:20:01.952400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.952658 (+   258us) service_pool.cc:224] Handling call
0709 12:20:02.475623 (+522965us) ts_tablet_manager.cc:1151] Deleting tablet b4036b67205e41c597049a10a71e9467
0709 12:20:02.475711 (+    88us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:20:03.930775 (+1455064us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:03.945956 (+ 15181us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:03.955836 (+  9880us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":521846,"threads_started":1}
I20260709 12:20:05.161489  2979 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.162461  2979 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.165087  2979 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.172947 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:05.174258 28286 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.249s	user 0.019s	sys 0.042s
I20260709 12:20:05.174877  3030 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:05.175336 28286 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.253s	user 0.019s	sys 0.042s
I20260709 12:20:05.186870  3026 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.187709  3026 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.189510 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:05.190857  3026 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.192129  3033 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:05.190963 28255 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:05.193708  3026 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.194378  3026 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.195253  3029 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.196575  3029 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.197966  3033 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.201435  3033 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.202908  3029 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.193579  3027 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.200528  3034 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.208180  3035 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.208221  3027 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.208993  3035 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.211547  3027 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.211673  3035 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.203956  3026 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.204185  3033 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.205314  3036 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.208600  3034 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.213011  3036 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.199470  3031 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.225890  3031 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.227397  3034 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.228827  3031 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.230134  3036 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:05.240741 28255 connection.cc:256] RPC call timeout handler was delayed by -1.5737840079998477s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
I20260709 12:20:05.244536  3031 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:05.253057 28255 connection.cc:256] RPC call timeout handler was delayed by -1.5179840079999849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
W20260709 12:20:05.246608 28292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 3512) took 3282 ms (client timeout 1588 ms). Trace:
I20260709 12:20:05.253118  3031 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:05.254057 28255 connection.cc:256] RPC call timeout handler was delayed by -1.311884007999879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
I20260709 12:20:05.249784  3035 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.255432  3035 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.256186  3031 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:05.258435 28292 rpcz_store.cc:269] 0709 12:20:01.964125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.246332 (+3282207us) service_pool.cc:211] Skipping call since client already timed out
0709 12:20:05.246508 (+   176us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:20:05.250193  3033 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.264915  3033 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.267248  3033 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.240854  3037 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.268472  3037 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.270959  3037 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.257800  3035 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.272569  3035 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.274996  3035 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.277287 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:05.279564  3033 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:05.277346  3035 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:05.319865 28441 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 3.420s	user 0.020s	sys 0.019s
I20260709 12:20:05.320456 28441 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:20:05.324621  3043 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.325366  3043 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.328060  3043 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:05.332535 28441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 3850) took 3488 ms (client timeout 2196 ms). Trace:
W20260709 12:20:05.333082 28441 rpcz_store.cc:269] 0709 12:20:01.832140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.833163 (+  1023us) service_pool.cc:224] Handling call
0709 12:20:05.320985 (+3487822us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:20:05.339340 28423 thread.cc:641] tablet-delete [worker] (thread pool) Time spent creating pthread: real 3.385s	user 0.023s	sys 0.049s
W20260709 12:20:05.339936 28423 thread.cc:608] tablet-delete [worker] (thread pool) Time spent starting thread: real 3.386s	user 0.024s	sys 0.050s
I20260709 12:20:05.345973  3033 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.346753  3033 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.349525  3033 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.350020  3042 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.350363  3033 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.355432  3042 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.355600  3033 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.357916  3042 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.351383  3040 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.351855  3041 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.368238  3040 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.368436  3041 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.352552  3039 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.371539  3039 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.372037  3045 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.372634  3045 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.373092  3046 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.373697  3046 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:02.464781 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:05.375401  3041 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.354079  3035 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.376775  3046 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.376644  3035 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.354511  3037 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.358580  3033 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.350661  3044 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.378062  3037 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.378417  3044 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.380635  3044 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.381075  3026 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.370960  3040 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.374028  3039 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.353152  3038 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.381706  3026 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.382503  3038 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.384083  3026 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.388305  3038 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:05.389659 28255 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.4:39985 timed out after 1.929s (SENT)
I20260709 12:20:05.395573  3027 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.398725  3027 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.410502  3037 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.388428  3035 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.413292  3045 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.437276  3048 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.379268  3046 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.434680  3027 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:05.452102 28291 consensus_meta.cc:287] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 3.564s	user 0.034s	sys 0.021s
I20260709 12:20:05.452699 28291 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:20:05.436847  3035 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:05.482648 28291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59778 (request call id 3509) took 3569 ms (client timeout 2196 ms). Trace:
W20260709 12:20:05.483445 28291 rpcz_store.cc:269] 0709 12:20:01.884179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.886281 (+  2102us) service_pool.cc:224] Handling call
0709 12:20:05.453240 (+3566959us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:05.487447  3026 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.439180  3048 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.498672  3026 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.500397  3048 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.500929  3026 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.501024  3048 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.488770  3034 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.501967  3034 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.504109  3034 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.445544  3056 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.468901  3030 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:05.445775  3042 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.484339  3043 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.482435  3027 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.525185  3034 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.521921  3045 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.541034 28189 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 312) took 3880 ms. Trace:
I20260709 12:20:05.525436  3026 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.495923  3035 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.501606  3048 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.557235  3058 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.557972  3058 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.547345 28189 rpcz_store.cc:276] 0709 12:20:01.527234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.534069 (+  6835us) service_pool.cc:224] Handling call
0709 12:20:01.628483 (+ 94414us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f1090 00007f85be88ff3a 00007f85c4645e9c 00007f85c474404e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc 00007f85c2c04392 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:05.407624 (+3779141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46699,"reported_tablets":1,"spinlock_wait_cycles":17296128}
I20260709 12:20:05.522203  3029 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.522471  3036 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.485500  3046 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:05.601912 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 212268us
I20260709 12:20:05.546126  3040 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.613792 28183 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 192) took 5180 ms. Trace:
I20260709 12:20:05.575038  3061 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.641728 28183 rpcz_store.cc:276] 0709 12:20:00.428087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:00.428264 (+   177us) service_pool.cc:224] Handling call
0709 12:20:05.608439 (+5180175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84483,"reported_tablets":3,"spinlock_wait_cycles":88192}
I20260709 12:20:05.611511 28185 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 17 to 18, f526369c0d0743b192be6e205540a059 (127.26.123.4) changed from NON_VOTER to VOTER. New cstate: current_term: 26 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:05.560035  3042 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.605625  3062 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.674958  3062 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.677357  3062 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.561897  3045 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.677980  3051 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.678412  3072 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.678887  3058 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.677968  3054 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.562312  3026 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.679147 28987 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:05.681309  3063 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.681874  3026 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.682480  3026 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.607874  3036 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.560433  3027 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.684734  3047 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.649590  3044 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.661916  3040 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.685387  3047 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.660341  3061 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.562137  3034 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.687207  3065 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.687803  3065 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.605535  3029 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.685527  3044 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.663558  3050 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.683374  3026 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.690302  3050 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.679812  3054 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.663529  3064 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.562880  3056 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.681944  3063 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.563066  3043 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:05.716095 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 134217us
I20260709 12:20:05.679699  3045 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.674113  3049 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.692492  3058 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.709168  3062 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.759018 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:05.760591  3070 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.761412  3070 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.762827  3058 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.766105  3070 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.765622  3058 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.766994  3073 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.773954  3070 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:05.717409  3064 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.775564  3045 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.678637  3051 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.804787  3029 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.806851  3043 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.807595  3043 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:05.808149  3028 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.808985  3028 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.809517  3055 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.810230  3055 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.811822  3028 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.812709  3055 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.814043  3040 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.816138  3045 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.818912  3045 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.819751  3054 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.821141  3069 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.821928  3069 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.824716  3069 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.758402  3049 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.827171  3047 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.827961  3049 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.828329  3044 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.828974  3026 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.829116  3035 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.830255  3046 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.830729  3042 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.776341  3062 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.767673  3073 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.853986 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.679075  3072 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.858106 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.856173  3070 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.865808  3070 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.864830  3032 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.869915  3032 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.870657  3066 consensus_queue.cc:1059] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:20:05.879580  3064 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.880108  3048 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:05.871516 28330 connection.cc:256] RPC call timeout handler was delayed by -1.9098583799998323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
I20260709 12:20:05.882162 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.882956 28288 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:20:05.883042 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:20:05.881367 28330 connection.cc:256] RPC call timeout handler was delayed by -1.4357528809998712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
I20260709 12:20:05.846347  3061 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.851022  3027 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.890246 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.887712 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.895016 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.895774 28286 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:05.883826 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.889264 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.890079 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.902992 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.903822 28284 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:20:05.905498 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:05.906409 28984 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:20:05.908646 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:05.909001 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:05.909618 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:05.891897 28289 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:05.911895 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.908679 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:05.914665  3067 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.914817  3064 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.915431  3064 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:05.915395  3067 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.917905  3064 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.918474  3064 raft_consensus.cc:3055] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:05.867477  3053 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.909137 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:05.868093  3065 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.808204  3043 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.918802  3064 raft_consensus.cc:740] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:05.870049  3070 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.940585  3070 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.942986  3051 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.948470  3073 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.949281  3053 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.951006  3052 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:05.951691  3052 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.952090  3053 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.953115  3034 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.955647  3072 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.957022  3063 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.959214  3067 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.960049  3032 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.888305  3062 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:05.963066 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:05.966677  3046 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:05.967293  3046 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.968237 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.969672 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.969901  3054 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.970235 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.970502  3055 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.970894 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.938473  3071 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.977319 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.971192 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.969457  3046 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.975065  3047 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:05.982196  3047 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.986140  3052 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.990161  3064 consensus_queue.cc:260] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:05.993177  3043 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:05.994671 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.995042 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.995487 28360 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:05.995795 28361 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:05.995893  3050 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:05.996632 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.997951 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.998095 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.998766 28361 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:05.971007 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.978183 28358 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:05.972152 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:05.973824  3061 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:05.973543 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.011029 28283 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:06.010563 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.941304  3070 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:06.018044 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 240093us
I20260709 12:20:06.023898 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.042357 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.985527  3047 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.046591 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.047120  3077 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.010260  3063 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.001479  3036 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:06.047906 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.000023  3071 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.052884 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.053305 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.052083 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.052139 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.053972 28358 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:20:06.054360 28283 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:20:06.018491  3061 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.016464  3044 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.057427 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.041393  2979 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.060415 28358 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:20:06.047802 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:05.971048  3055 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.063871 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.065426 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.063333 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.061913 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.065384 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.067886 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.068681 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.069137 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.066561 28985 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:20:06.070277 28358 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:20:06.011250  3054 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:06.071771 28332 connection.cc:256] RPC call timeout handler was delayed by -1.4949902149999161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
I20260709 12:20:06.074620 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.074793 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.073381  3046 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.047784  3077 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.080356  3046 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.082530  3077 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.082784  3046 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.083146 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:06.084313 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.047911  3063 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.085283 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.072971  3052 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.086272  3052 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.087726  3055 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.048220  3076 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.088963  3076 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.089629  3054 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.069085  3067 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:06.090725  3067 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.091168  3076 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.092449 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.092885  3067 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.093863 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.094041  3074 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.094671  3074 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.094679 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.098119 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.088407  3052 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.099643  3071 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.100714 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.072186  2979 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.073593  3044 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.106997 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.108124  3044 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.108616  2979 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.108879 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.109880 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.111748 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.111979 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.112686 28984 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:06.113646  3033 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:06.084203  3070 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.114107  3033 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.114580  3033 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:06.116001 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.117190 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.118177 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.086341  3063 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.122782 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.124781  3056 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.097563  3074 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.118498 28982 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:06.104548 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.095011 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:06.059986  3061 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.130535 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.131702 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:06.134168 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 168103us
I20260709 12:20:06.112727 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.129155 28988 raft_consensus.cc:2376] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19. Re-sending same reply.
I20260709 12:20:06.280479 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.149667 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.310230 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.206223 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.269235 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.284610 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.284914 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.137272 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.320727 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:06.220729 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 82367us
I20260709 12:20:06.232614 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.311898 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.153690 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.326782 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.323067 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.334681 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.337412 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.339102 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.340317 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.340978 28284 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:20:06.148391  3080 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:06.336229 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:06.258589  3064 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:06.346817 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.209075  3070 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.344638 28364 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:20:06.261965  3068 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:06.149205  3033 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.374557  3068 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.376406  3033 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.376761  3068 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:06.378156 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.378875 28437 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:06.356442  3060 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:06.382177 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.382272  3060 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.382875  3060 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:06.386202 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.387532  3075 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:06.388128  3075 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:06.388568  3075 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:06.342224  3080 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.343458 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:06.146610  3071 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:06.220319  3067 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.220504  3066 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.237671  3081 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:06.345297 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 208100us
I20260709 12:20:06.146929  3028 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:06.253362  3052 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.357599 28439 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:20:06.153488  3062 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.148295  3053 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.358821 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.365928 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.370622 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.394798  3070 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.372859 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.373242 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.368404 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:06.428520 28434 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:06.427927 28436 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
W20260709 12:20:06.394809 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 51446us
I20260709 12:20:06.397802 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.403304 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.404516 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:20:06.430449 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 62136.1us
I20260709 12:20:06.429236 28440 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:20:06.427874 28359 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:06.403847 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.371454 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.233317  3049 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.398598  3076 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:06.436935 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.437119 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.437000 28436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
mode: GRACEFUL
new_leader_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:33074
I20260709 12:20:06.438040 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.438249 28436 raft_consensus.cc:606] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Received request to transfer leadership to TS f526369c0d0743b192be6e205540a059
I20260709 12:20:06.438658 28439 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:06.440114 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.442199 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.441543 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:06.445640 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.448971 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.450089 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:06.450486 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.451144 28986 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:20:06.451364 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.454030 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.450750 28292 raft_consensus.cc:2376] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25. Re-sending same reply.
I20260709 12:20:06.458132 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.436159  3065 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:06.405072  3028 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.459594 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.468750 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.457445 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.457894 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.440320  3030 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:06.404989  3071 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.462553 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.460359 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.463254 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.485167 28292 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:20:06.470098 28988 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:06.464315 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.439891  3081 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.465237 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.398227  3067 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.487748 28441 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:20:06.488365 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.453083  3054 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:06.488967  3067 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.440742  3076 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.432804  3052 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.489064 28437 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:06.489789  3076 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:06.490146 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.490696 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.491297 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.491851 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.492470 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.492983 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.493129 28441 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:06.466907 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.469834 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.458801 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.470585 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:20:06.485531 29012 debug-util.cc:398] Leaking SignalData structure 0x7b080028a540 after lost signal to thread 27119
I20260709 12:20:06.471987 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.435299  3053 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.472718 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.473353 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.475690 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.487529 28438 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:06.479410  3087 raft_consensus.cc:993] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: : Instructing follower f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:20:06.502238 28433 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:06.485865  3071 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.503213  3087 raft_consensus.cc:1081] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Signalling peer f526369c0d0743b192be6e205540a059 to start an election
I20260709 12:20:06.447727  3068 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.504226 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.504691  3070 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.504834  3068 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.506264 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.507200  3068 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.507508 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.508797 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.470609  3030 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.442284  3049 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.405306  3066 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.471108 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.509604 28440 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:06.512511  3066 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:06.455204  3088 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.499879 28434 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 12.
I20260709 12:20:06.477880 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9cdd1e39ef1841c382e814af04ea5bc9 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:06.495523 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.514012  3088 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.495831 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:06.515309  3028 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:06.514671 28176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 72471us
W20260709 12:20:06.497887 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08001dd3c0 after lost signal to thread 28224
I20260709 12:20:06.516592  3088 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.501968  3052 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:06.497638 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:06.493997  3085 log.cc:927] Time spent T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:20:06.518668  3050 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:06.455592  3083 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:06.519366 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800241720 after lost signal to thread 28316
I20260709 12:20:06.483278  3080 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.510318 28438 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:06.505551 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.499895  3054 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.530073 28433 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:20:06.499791  3053 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:06.521004 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5580 after lost signal to thread 28391
W20260709 12:20:06.535243 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800288340 after lost signal to thread 28392
I20260709 12:20:06.493716 28437 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:06.519150  3050 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.510389  3030 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:06.525935 28176 catalog_manager.cc:5028] TS 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): tablet b4036b67205e41c597049a10a71e9467 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
W20260709 12:20:06.534593 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 77425.6us
I20260709 12:20:06.466348  3065 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:06.536552 29012 debug-util.cc:398] Leaking SignalData structure 0x7b080038e200 after lost signal to thread 28465
I20260709 12:20:06.539847 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.488024  3081 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.548406 28359 raft_consensus.cc:2393] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 14: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:06.547691 28330 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.547600 28435 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:06.449134  3062 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.551944 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.552464 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:06.545260 29012 debug-util.cc:398] Leaking SignalData structure 0x7b0800367ea0 after lost signal to thread 28466
I20260709 12:20:06.546712  3065 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:06.538789  3050 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:06.566767  3049 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.535105  3054 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:06.578367 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:06.591017 29012 debug-util.cc:398] Leaking SignalData structure 0x7b08003697e0 after lost signal to thread 29015
W20260709 12:20:06.598232 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 50649.3us
I20260709 12:20:06.545218  3083 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:06.599035 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 44168us
I20260709 12:20:06.579690  3075 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:06.605036 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 150408us
I20260709 12:20:06.603529 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.606377 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.608772 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:06.610983 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:06.532627  3076 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.608299 28330 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.566953  3028 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.618518 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.632894 28439 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:20:06.624536 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:06.566687  3061 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.613214 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:06.599300  3077 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:06.636554 28330 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.650581 28362 raft_consensus.cc:1240] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:06.639955 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.611639  3063 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:06.642256 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.645624 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.646376 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.647215 28365 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 17: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:06.649256 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.651641 28367 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 26.
W20260709 12:20:06.659420 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 46286.8us
I20260709 12:20:06.660358 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:06.631469  3066 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:06.667582 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.668975 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.669618 28366 raft_consensus.cc:2468] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 26.
I20260709 12:20:06.670174 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.645324  3030 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.670535 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.671571  3075 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:06.672400  3030 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.673346  3062 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.673499  3090 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:06.638590  3044 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:06.674050  3090 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.674523  3090 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.675491 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.677959 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.679381 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:06.680759 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.682179 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.682307 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.675837 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:06.691191 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.694679 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.697964 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.699087 28435 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
W20260709 12:20:06.706584 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 67278.6us
I20260709 12:20:06.657104  3048 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:06.709105 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.649863  3077 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.731050  3077 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:06.708935 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:06.664253  3066 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.734117  3066 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.734138 28438 raft_consensus.cc:2376] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25. Re-sending same reply.
I20260709 12:20:06.677017  3053 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:06.738219  3053 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.738548 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.739912 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.741142 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.743552  3090 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.708979 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.747550 28437 raft_consensus.cc:2468] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:20:06.748517 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.749162 28437 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:06.750165  2979 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:06.750586 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.750840  2979 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.751590 28185 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 324) took 5521 ms. Trace:
I20260709 12:20:06.753805 28365 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:20:06.754418 28365 raft_consensus.cc:606] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Received request to transfer leadership to TS a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.756145 28183 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 10 to 11, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New cstate: current_term: 13 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:06.709729 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:06.759398 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 49756.7us
I20260709 12:20:06.737347 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.760221 28954 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.761616  3034 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:06.762156  3051 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:06.712780  3060 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.738619  3053 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:06.770210  3060 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.694144  3061 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.720475  3083 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:06.722182  3076 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.753225  2979 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.674332  3044 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.774212  3044 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:06.673352  3028 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.723747  3052 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:06.709434  3048 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.725332 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.778431  3048 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:06.726106 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.728734 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:06.780280 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 51686.4us
I20260709 12:20:06.782369 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:06.784111 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:06.763515 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 54815us
I20260709 12:20:06.767506 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:06.768002 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:06.770951 28432 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:20:06.786809  3037 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.771533 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:06.787518  3037 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:06.788986 28257 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.784440  3060 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:06.784870  3028 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:06.730095 28358 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:06.785081 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.785946 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:06.792810 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.789297 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:06.795795 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:06.778383 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.798030 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:06.772537 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 118004us
I20260709 12:20:06.804755 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:06.751955 28185 rpcz_store.cc:276] 0709 12:20:01.230432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.241600 (+ 11168us) service_pool.cc:224] Handling call
0709 12:20:01.539858 (+298258us) spinlock_profiling.cc:243] Waited 39.1 ms on lock 0x7b5400002490. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb7630b2 00007f85c46dc759 00007f85c4674a76 00007f85c4645d58 00007f85c474404e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc
0709 12:20:01.639750 (+ 99892us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f1090 00007f85be88ff3a 00007f85c4645e9c 00007f85c474404e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc 00007f85c2c04392 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:06.751520 (+5111770us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:05.886033 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:05.886609 (+   576us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:05.886649 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:20:05.886664 (+    15us) tablet.cc:662] Decoding operations
0709 12:20:05.973172 (+ 86508us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:05.979541 (+  6369us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:06.005549 (+ 26008us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:06.005821 (+   272us) tablet.cc:701] Row locks acquired
0709 12:20:06.005840 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:20:06.006009 (+   169us) write_op.cc:270] Start()
0709 12:20:06.006143 (+   134us) write_op.cc:276] Timestamp: P: 1783599606005950 usec, L: 0
0709 12:20:06.006165 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:20:06.006938 (+   773us) log.cc:844] Serialized 595 byte log entry
0709 12:20:06.674667 (+667729us) op_driver.cc:464] REPLICATION: finished
0709 12:20:06.699060 (+ 24393us) write_op.cc:301] APPLY: starting
0709 12:20:06.699166 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:06.700593 (+  1427us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:06.700710 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:06.701684 (+   974us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:06.702743 (+  1059us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:06.702798 (+    55us) write_op.cc:312] APPLY: finished
0709 12:20:06.703060 (+   262us) log.cc:844] Serialized 25 byte log entry
0709 12:20:06.704054 (+   994us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:06.704154 (+   100us) write_op.cc:454] Released schema lock
0709 12:20:06.704204 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:06.705182 (+   978us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:06.705212 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":74284,"reported_tablets":3,"spinlock_wait_cycles":114048640,"child_traces":[["op",{"apply.queue_time_us":24095,"apply.run_cpu_time_us":4144,"apply.run_wall_time_us":6111,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8305,"num_ops":1,"prepare.queue_time_us":222567,"prepare.run_cpu_time_us":15329,"prepare.run_wall_time_us":235929,"raft.queue_time_us":155022,"raft.run_cpu_time_us":7198,"raft.run_wall_time_us":26060,"replication_time_us":668327,"thread_start_us":386300,"threads_started":4,"wal-append.queue_time_us":392797}]]}
I20260709 12:20:06.810079 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.811095 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.811844 28984 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:06.808743 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.813660 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.815181 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:06.816395 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 54749.1us
I20260709 12:20:06.819162 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.823693 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:06.827160 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:06.822230 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.830873 28365 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:06.814160 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.821532 28954 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.833894 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.837064 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.838400 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.839004 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.839907 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:06.841348 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.839296 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.840889 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.805670  2979 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:06.745457  3050 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.856071 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.823413  3075 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.828948  3027 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:06.862468 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.846582 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.849597 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.849876 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.850355 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.846000 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.852012 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:06.853183 28257 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:06.823127  3052 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:20:06.859079 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 86811.2us
I20260709 12:20:06.858842 28432 consensus_queue.cc:237] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.828819  3091 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.867013 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.868033  3093 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:06.847625 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 141413us
I20260709 12:20:06.878480 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.879277 28332 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.868927 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.861424  3090 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:06.842401  3036 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.881922 28956 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.877140 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.877518 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.866514 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:06.885244 28991 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063"
dest_uuid: "f526369c0d0743b192be6e205540a059"
 from {username='slave'} at 127.0.0.1:46934
I20260709 12:20:06.885910 28991 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:06.858814  3030 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:06.882206 28330 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:06.883496 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:06.880880 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:06.886281 28991 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:06.856844  2979 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.906001  2979 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:06.887354 28990 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:06.887758  3048 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.912170  3048 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.888115  3081 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:06.874800  3060 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.880661  3062 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:06.915627  3076 raft_consensus.cc:993] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:20:06.888835  3030 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.916153  3076 raft_consensus.cc:1081] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Signalling peer a3de34b6f4cf4606a61944db6e2739c8 to start an election
I20260709 12:20:06.917140  3032 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.917778  3032 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.918499 28956 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.919348  3100 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:06.920037  3032 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.920632  3032 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:06.921284  3032 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:06.921753 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:06.845250  3033 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:20:06.918402  3034 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:06.925304  3077 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.878937  3052 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:06.926473  3052 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:06.927731  3054 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.927999  3044 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.928928 28361 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:06.886358  3056 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.930052  3056 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.931885  3077 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.932267  3056 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.888761  3037 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:06.933120  3037 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.933853  3054 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:06.934689  3099 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:06.935448  3037 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.883049  3061 raft_consensus.cc:2804] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:06.889691 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.915198  3053 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.936211  3061 raft_consensus.cc:697] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:06.937206  3061 consensus_queue.cc:237] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:06.938061  3053 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:06.940174 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 58065.5us
I20260709 12:20:06.916160  3030 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:06.942305 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 52719.7us
I20260709 12:20:06.941918 28956 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:06.944053 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.945914  3053 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.941020 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:06.946588  3034 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:06.945914  3048 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:06.946147 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 66991.4us
I20260709 12:20:06.948993 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:06.910320  3050 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:06.909631  3065 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.941140  3066 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.951648 28181 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 322) took 1691 ms. Trace:
I20260709 12:20:06.947118  3034 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:06.950160 28189 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 18 to 19, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 19 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:06.968478 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:06.971843 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 118712us
I20260709 12:20:06.970651 28181 rpcz_store.cc:276] 0709 12:20:05.260614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.261289 (+   675us) service_pool.cc:224] Handling call
0709 12:20:06.951555 (+1690266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":136592,"reported_tablets":3}
I20260709 12:20:06.973758 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:06.948115 28956 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.973069 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:06.967763 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:06.974992 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 94710.3us
I20260709 12:20:06.979698 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.981789 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:06.982909 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:06.983397 28988 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:06.983856 28956 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:06.983296 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:06.984608 28358 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:06.966084  3084 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.974414  3034 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.986178 28364 raft_consensus.cc:1240] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:06.981148  3051 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:06.958002 28991 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:06.970938  3053 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.085265 28988 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:07.086578 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.079469 28358 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:07.080143 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 96927.3us
I20260709 12:20:06.995527  3089 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:07.082243 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 114561us
I20260709 12:20:07.084318  3074 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:07.105124 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:06.979336  3094 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:07.084132  3044 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.093904  3056 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:07.086529  3050 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.086762 28438 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.099279 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:20:07.100809 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 114605us
I20260709 12:20:07.083925 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.086166  3054 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:07.115698 28291 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:07.115893 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:07.116761 28956 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.102401 28332 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:20:07.106210 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 254272us
I20260709 12:20:07.107167 28991 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.121789 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:07.106599 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.102375 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.096750  3076 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.105525  3074 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:07.125892 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:07.112228  3094 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:07.130887 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.118031  3054 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:07.152345 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.146615 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:07.136181 28291 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
I20260709 12:20:07.147710 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:07.147881 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.129840 28332 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.173269 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:07.151069  3033 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:07.143036  3050 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.137010  3063 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:07.136440  3074 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.146106  3061 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:07.190001  3061 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:07.190497  3061 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.119212  3080 consensus_queue.cc:1059] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:20:07.163615  3052 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.153966 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.141522  3094 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:07.195657  3052 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.129048  3096 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:07.196455  3096 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:07.196900  3096 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.128854  3060 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:07.197955  3060 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:07.198406  3060 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:07.198666  3080 raft_consensus.cc:3055] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:07.198935  3065 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:07.199602 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 45727.6us
I20260709 12:20:07.200502  3081 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.202265  2979 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.204551  2979 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.178679  3097 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:07.205629  3097 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:07.206041  3097 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.168915 28330 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.175411 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:07.119079  3056 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.208725  3066 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.209444  3076 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:07.209995  3080 raft_consensus.cc:740] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:07.211018  3080 consensus_queue.cc:260] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.212257  3080 raft_consensus.cc:3060] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:07.212831 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:07.213982  3105 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:07.214408  3105 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:07.214774  3105 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:07.215639  3037 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:07.216121  3037 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:07.216473  3037 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.219169  3030 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.220459 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:07.175853 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:07.222599 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 46834.5us
I20260709 12:20:07.224248 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.225303 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:07.185117  3028 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:07.235658  3028 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:07.236188  3028 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:07.167464  3086 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.138159  3032 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:20:07.239014  3032 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:07.167584  3068 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:07.239831  3068 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:07.240281  3068 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:07.179742  3033 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:07.129585  3089 consensus_queue.cc:1059] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:20:07.241837  3091 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:07.242683  3089 raft_consensus.cc:3055] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:07.243072  3089 raft_consensus.cc:740] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:07.243994  3089 consensus_queue.cc:260] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.246069  3033 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:07.239691  3101 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:07.179607 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:07.248087 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 68547.5us
I20260709 12:20:07.248817 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.171149 28954 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:07.250109 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 79056.8us
I20260709 12:20:07.252252  3060 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.129503  3102 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:07.151338  3054 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:07.253800  3086 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:07.254354  3086 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:07.142617  3062 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:20:07.253970  3102 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:07.255242  3102 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:07.255599  3062 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:07.255982  3062 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:07.256592  3101 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:07.256939  3062 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.166486  3077 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:07.226410  3034 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.253515 28289 raft_consensus.cc:1240] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:07.184628  3048 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:07.257120  3101 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.257944  3076 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:07.258201  3104 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:07.259658  3033 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.353539  3033 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.353605 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.344836 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.336108  3076 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:07.355167  3076 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:07.355417 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:07.261339 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.359108 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.200428 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.262532 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:20:07.360498 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 160176us
I20260709 12:20:07.360925 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.362212 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.362996 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:07.363481 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.361255 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.364699 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.254762  3065 raft_consensus.cc:2764] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 12:20:07.365587 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.263257 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.254721  3086 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:07.367175 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.266264 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.209759  3056 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.368386 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.362468  3088 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:07.268018  3102 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.371119 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.371542  3102 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.269490 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.372486 28290 raft_consensus.cc:2393] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:07.373265 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.374569 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.374886 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.375658 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.376142 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.376806 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.270545 28292 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
 from {username='slave'} at 127.0.0.1:59778
I20260709 12:20:07.271965 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.378054 28292 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:07.378459 28292 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.272643 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 9cdd1e39ef1841c382e814af04ea5bc9 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:36572
I20260709 12:20:07.273005 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.369599 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.273198 28284 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:07.272682 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.276163  3060 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.264858  3105 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.279943  3054 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.273990  3068 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.283973  3094 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.285040  3061 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.285609  3084 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:07.400816  3094 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.400650  3084 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.286119  3096 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.387212  3105 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.388885  3054 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.401247  3068 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.402987  3096 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.405114  3084 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.289201  3089 raft_consensus.cc:3060] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:07.407658  3061 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.402009  3053 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:07.407992 28284 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:07.292522 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:07.304329 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:07.410379 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 117954us
I20260709 12:20:07.308252 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.317180 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:07.411558  3096 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:07.411937  3096 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:07.412317  3096 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.318540 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:07.413268  3097 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.414098 28367 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:20:07.327453 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.338140 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.339097 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:07.221561  3030 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.324055  3048 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:07.416251  3048 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:07.416628 28365 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.416783 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.419799 28332 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.343708  3034 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.344523 28986 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:20:07.421885  3074 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.345372  3062 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:07.344885  3104 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.347556 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:07.349177  3101 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.377581 28291 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:07.382023 28988 raft_consensus.cc:1240] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 21. Current term is 23. Ops: []
I20260709 12:20:07.381712 28288 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:07.383749 28286 raft_consensus.cc:1240] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:07.392426 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.384395 28285 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:07.390455 28432 raft_consensus.cc:1240] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:20:07.437073  3084 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.396992 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.439217  3101 consensus_queue.cc:1059] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:07.440340  3101 raft_consensus.cc:3055] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:07.440717  3101 raft_consensus.cc:740] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:07.440757  3053 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:07.444702  3096 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.440444 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.451126  3053 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.453749  3074 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.454351  3101 consensus_queue.cc:260] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.455109  3034 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:07.455899 28285 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:07.456254  3034 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:07.457052  3065 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:07.457437  3065 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.458349  3065 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.415608  3097 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.340862  3036 consensus_queue.cc:1059] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:07.459904  3065 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:07.459913  3036 raft_consensus.cc:3055] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:07.460494  3036 raft_consensus.cc:740] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.400048  3080 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:07.461549  3036 consensus_queue.cc:260] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.465754  3036 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:07.473153  3108 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:20:07.474129 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54425.3us
I20260709 12:20:07.474725  3050 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:07.503675  3028 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.484242  3101 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:07.505030 28332 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.484829  3104 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.508567  3108 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.509770  3108 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.511224  3108 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:07.486038  3048 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.517457  3048 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.518498  3037 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.487529  3076 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.488751 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.489301 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.398597 28292 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.504151  3050 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:20:07.522315 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65002.4us
I20260709 12:20:07.522785  3050 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.453419 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:07.523810  3050 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.525175  3050 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:07.475839  3102 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:07.526539  3102 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:07.526887  3102 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.527746  3102 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.529486  3102 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:07.531003 28283 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:07.532887 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.534898  3056 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.536391 28255 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.536528  3086 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:07.523801 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 70465.8us
I20260709 12:20:07.543489  3037 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.539070  3028 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:07.540269 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.542955 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:07.543718 28292 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.544963  3036 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:07.577093  3104 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.577569  3036 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:07.545567 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.543648 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:20:07.547062 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:07.548391 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 431679us
I20260709 12:20:07.578869 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:07.546928 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 338054us
I20260709 12:20:07.580057 28956 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.542991  3113 tablet_replica.cc:333] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:07.538213  3076 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:07.558529 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.560354  3096 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.560544 28332 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.571518 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:07.572350 28437 raft_consensus.cc:1813] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Handling vote request from an unknown peer a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:07.576632 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.585004 28365 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:20:07.585686 28437 raft_consensus.cc:2410] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:20:07.582172 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.588415 28292 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:07.582618 28956 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.560233  3110 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:07.590991 28292 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.591747 28956 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.591990 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:07.592404 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.596251 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.593855 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.598821 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.600775 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.602886 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.603485 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.578032  3036 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.607723 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.604550 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.609416 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:07.604837 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.610601 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.612177 28365 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.612471 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.583202  3102 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:07.614627 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.614326 28362 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.616350 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.616511 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.614226 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.617717 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.618574 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.619503 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.620301 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.615410 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:07.616863  3056 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.616606  3062 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:07.623545  3062 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:07.623951  3062 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:07.624204 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.625365 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.584290  3111 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:20:07.596699  3097 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.625833 28367 raft_consensus.cc:1240] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:07.604269  3080 consensus_queue.cc:1059] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:20:07.630187  3080 raft_consensus.cc:3055] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:07.630545  3080 raft_consensus.cc:740] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:07.630702 28954 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.631412  3080 consensus_queue.cc:260] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.631995 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.632617  3109 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.633524 28954 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.590368  3050 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:07.635262  3050 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:07.594630  3034 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:07.635311  3036 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.636749  3036 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:07.634541 28982 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:20:07.620302 28359 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 21: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:07.640240 28440 raft_consensus.cc:2393] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:07.628998 28433 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:07.596501  3089 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:07.607069 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.636685  3100 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:07.632339  3051 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:07.642781 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.642971 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.607322  3076 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:07.615794 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.643761  3076 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:07.644173  3076 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:07.644881 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.645632 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.628912 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.643323  3102 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:20:07.647181  3102 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:07.647495  3102 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.647807 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.648393  3102 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.621276  3086 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:07.595072  3061 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:20:07.650995 28287 raft_consensus.cc:1813] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Handling vote request from an unknown peer 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:07.635175  3113 raft_consensus.cc:2243] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:07.645941  3100 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:07.652470 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.652597  3113 raft_consensus.cc:2272] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:07.654071 28433 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 21: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:07.651391  3061 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:07.634399  3109 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.651459 28436 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:07.651631 28287 raft_consensus.cc:2410] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:20:07.659327 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.668355 28283 raft_consensus.cc:1240] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:07.659179 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.657918  3065 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:20:07.673250 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:07.674042  3051 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:07.674417  3051 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.675343 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.675247  3051 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.676553  3051 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.658712  3113 ts_tablet_manager.cc:1916] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:07.664896 28285 raft_consensus.cc:1275] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:20:07.657693 28439 raft_consensus.cc:2468] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:07.649670  3102 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.690582 28439 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.658313 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.658950  3111 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:07.702389 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.659705  3061 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:07.691120  3101 consensus_queue.cc:1059] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:07.691738  3037 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:20:07.711256 28956 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:07.664316 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.699707 28181 catalog_manager.cc:5697] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 24 to 25, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 25 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:07.692153 28432 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:07.660881 28438 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:07.713590 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.700475 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.697854 28986 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:07.707824 28367 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 21: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:07.718312  3101 raft_consensus.cc:3055] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:07.718719  3101 raft_consensus.cc:740] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:07.707500  3111 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:07.722851 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.694007 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.705574 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.704416 28434 raft_consensus.cc:2468] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:07.709259 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.724792 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.719269  3037 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:20:07.703168 28407 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.717481  3100 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.727089 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.715770  3108 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:07.725629  3053 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.721290  3056 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.698112  3080 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:20:07.731072 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.731374  3065 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:07.737354  3097 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:20:07.718715  3062 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.732607  3030 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.733817  3099 consensus_queue.cc:1059] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:20:07.739519 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.729216  3050 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:07.727807 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.741072 28984 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 21: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:07.741683  3100 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:07.742228  3097 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:07.729169  3074 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.726437  3037 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:07.743453 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.744184 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:07.745508 28286 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.747602 28255 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.737507  3105 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:20:07.740415 28332 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:07.749020  3053 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:07.734549  3101 consensus_queue.cc:260] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.751577  3096 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:07.742982  3028 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.750766 28434 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 21: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:07.754633 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.753927 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.756547 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.749285  3056 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.758280 28332 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.758792 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.751588  3034 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:07.760051 28330 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.746080  3108 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:20:07.761726 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:07.767043 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.755105  3074 raft_consensus.cc:2804] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:07.826169 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:07.765772  3084 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:20:07.750051  3111 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.748148  3061 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.822327 28364 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:07.756280  3030 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:07.757081  3105 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:07.800035 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.801700 28365 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:07.823222 28984 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:20:07.791231 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:07.832945 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.914105  3048 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:07.822448  3037 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:07.782967 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.916153  3048 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:07.916368  3061 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:07.916716  3037 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:07.917161  3048 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.793946 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.918588  3048 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.760481  3028 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:07.834425  3109 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:07.920097  3109 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.921108  3109 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.792260 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.765877  3060 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.156s
I20260709 12:20:07.825544  3104 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:07.923242  3104 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:07.837255  3075 raft_consensus.cc:3055] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:07.768134  3053 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.788949 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:20:07.823175 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54616.9us
I20260709 12:20:07.768271  3054 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:07.923640  3104 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:07.924582 28987 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.925004  3075 raft_consensus.cc:740] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:07.767385 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.789323 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:07.766070  3086 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.160s
I20260709 12:20:07.760196  3096 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:07.938874  3096 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:07.939101  3088 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:07.759874  3101 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:07.939884  3062 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:07.940518 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 173203us
I20260709 12:20:07.941291 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.942071  3062 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.929142  3094 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:07.943591 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.944568  3062 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.944307  3076 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:07.926420  3105 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:07.912899  3111 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:07.945600  3096 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:07.928587 28991 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:20:07.791684 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:07.912106 28986 raft_consensus.cc:1813] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Handling vote request from an unknown peer a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.952100 28986 raft_consensus.cc:2410] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:20:07.945467  3105 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.953498 28983 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:20:07.953346 28362 raft_consensus.cc:1813] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Handling vote request from an unknown peer a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.956408 28362 raft_consensus.cc:2368] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15.
I20260709 12:20:07.930796 28364 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:07.832252 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 65281.5us
I20260709 12:20:07.822314  3044 raft_consensus.cc:2804] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:07.888161  3089 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:07.915488  3074 raft_consensus.cc:697] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:07.968858 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.792070 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:07.969878 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.971254 28982 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:20:07.983829 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:07.984759  3088 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:07.987758 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 115594us
I20260709 12:20:07.988315 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:07.999518 28987 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:08.003495  3075 consensus_queue.cc:260] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.004509 28360 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:08.004901 28360 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:08.005242  3075 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:08.005635 28360 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.006853 28360 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.939925  3033 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:20:08.009361 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.958012  3056 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:08.022584 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:08.023635  3126 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.986972  3062 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:08.037221 28989 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
W20260709 12:20:08.038120 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 44255.9us
I20260709 12:20:08.013005  3053 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:08.041797 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.045367 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:08.040575 28360 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:08.009402  3033 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:07.983014  3100 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
W20260709 12:20:08.043741 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.992857  3089 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.046115 28438 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:08.039541  3096 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:08.062624  3034 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:08.063055  3034 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:08.063902  3034 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.065153  3034 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:08.070384  3100 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:08.070863  3100 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:08.071401  3089 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:08.071712  3105 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.071727  3100 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.075736  3074 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.052302  3118 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:08.080451  3109 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:08.081312 28434 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:08.082005  3076 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:08.082530 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:08.085110  3100 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:08.008705  3104 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.086618 28359 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:08.090096 28360 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:08.092456 28362 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:08.092875 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 123123us
I20260709 12:20:08.094231 28330 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:08.051719  3086 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
W20260709 12:20:08.097838 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 73813.9us
I20260709 12:20:08.092237  3105 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:08.034855  3056 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.109026 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:08.111583 28438 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:20:08.130219 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.133723 28988 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:20:08.138386 28286 raft_consensus.cc:1275] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:08.140177 28330 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.140784 28984 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:08.141283 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:08.141995 28982 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:08.141389 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:08.115320  3048 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:08.135288  3033 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.142518  3127 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:08.153007  3104 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:08.154048 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:08.145545  3113 ts_tablet_manager.cc:1929] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.10
I20260709 12:20:08.147509 28332 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:08.149817 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.143950 28988 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 21: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:08.181037  3033 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:08.181950  3113 log.cc:1199] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/wals/9cdd1e39ef1841c382e814af04ea5bc9
I20260709 12:20:08.183879  3056 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:08.150207  3123 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:08.157894 28432 raft_consensus.cc:1240] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:08.189092 28330 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.185253 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 9cdd1e39ef1841c382e814af04ea5bc9 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:20:08.193496 28440 raft_consensus.cc:2468] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 23.
I20260709 12:20:08.157915  3110 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:08.195580 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.164855 28982 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:20:08.209715  3054 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:08.212288 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:08.213424 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:08.216584 28956 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:08.216646 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:08.217550 28989 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 20: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:08.192142  3048 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:08.226712  3075 raft_consensus.cc:2804] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:08.194610  3104 consensus_queue.cc:1059] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:20:08.227286  3075 raft_consensus.cc:697] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 23 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:08.228236  3075 consensus_queue.cc:237] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.220458  3125 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:08.220355  3088 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:20:08.230541  3125 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:08.230890  3125 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:08.220323 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:08.226141  3089 consensus_queue.cc:1059] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:08.231725  3125 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.232702  3123 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:08.233032  3125 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:08.233773  3104 raft_consensus.cc:3055] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:08.234158  3104 raft_consensus.cc:740] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:20:08.235355 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.193059  3061 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:08.232719  3089 raft_consensus.cc:3055] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:08.236316  3089 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:08.237124  3089 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.238894  3089 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.219231 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.240540 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.242420  3110 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:08.242719  3084 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:08.244033  3048 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:08.242461  3096 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:08.245059  3096 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.234930  3104 consensus_queue.cc:260] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.246624  3104 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:08.247540  3096 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:08.250083 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:08.244971  3048 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.252599 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:08.253866  3088 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:20:08.261415 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.260522 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.265945 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:08.266256  3088 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:08.264493  3096 consensus_queue.cc:1048] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:20:08.271710  3088 consensus_queue.cc:1048] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:20:08.274992 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:08.283399  3088 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:08.284117  3088 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:08.285068  3088 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.198071  3044 raft_consensus.cc:697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:08.292963 28183 catalog_manager.cc:5697] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 21 to 23, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 23 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:08.282721 28330 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:08.299991  3130 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:08.300735  3044 consensus_queue.cc:237] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:08.321065 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 275322us
I20260709 12:20:08.322058 28255 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.324034  3028 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:08.325469 28255 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.328384  3028 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:08.328890  3028 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:08.329779  3028 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.334908 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:08.338982 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:08.348439  3125 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:08.335821 28181 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 16 to 17, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 17 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.339676  3088 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:08.335886 28255 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:20:08.357688 28255 raft_consensus.cc:2686] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:20:08.358354 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.359294  3129 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:08.359889  3129 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:08.365885  3089 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 26 LEADER]: Committing config change with OpId 26.19: config changed from index 18 to 19, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:08.366571 28183 catalog_manager.cc:5697] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 17 to 18, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.368501 28183 catalog_manager.cc:5697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 19 to 20, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 20 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.370901 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.374433 28292 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Committing config change with OpId 26.19: config changed from index 18 to 19, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:08.370061 28183 catalog_manager.cc:5697] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 17 to 18, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.377940 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:08.386063  3037 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:08.386202 28255 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.386775  3037 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:08.387527 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.387686  3033 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:08.388281  3033 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:08.387710  3037 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.389164  3033 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.389483  3060 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:08.390436  3060 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:08.393154  3060 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.393667 28283 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:08.376819 28982 raft_consensus.cc:2955] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Committing config change with OpId 26.19: config changed from index 18 to 19, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:08.385936  3104 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:08.361105  3129 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.404500 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:08.407655 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:08.408993 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:08.416484 28359 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:20:08.409211  3104 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:08.416963 28189 catalog_manager.cc:5697] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 18 to 19, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 26 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:08.418380  3104 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.399767  3044 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:20:08.412177 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:08.419694  3118 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:08.431078 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.431774  3118 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:08.431945  3093 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:08.432286  3118 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:08.433583 28255 leader_election.cc:400] T 9cdd1e39ef1841c382e814af04ea5bc9 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15.
I20260709 12:20:08.433185  3118 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.431067 28982 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:08.437393 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.439378  3118 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:08.444531  3037 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:08.461676 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.436751  3028 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:20:08.463125 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.464010  3105 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:08.467613  3093 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:08.470211 28185 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 17 to 18, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 18 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.489192 28438 raft_consensus.cc:1275] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:08.478447  3037 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:08.484858  3119 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:08.469147 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:08.498733  3028 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:08.499343  3028 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:08.499789  3028 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:08.502256  3037 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.497787  3119 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:08.505425  3119 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:08.506925  3119 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.508137  3089 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:33064 (request call id 13) took 1735 ms. Trace:
I20260709 12:20:08.491796  3130 consensus_queue.cc:1048] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:20:08.508563  3089 rpcz_store.cc:276] 0709 12:20:06.793815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.793972 (+   157us) service_pool.cc:224] Handling call
0709 12:20:06.912104 (+118132us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b2c000ef378. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bd180a11 00007f85bd18fa0a 00007f85bd18f693 00007f85bd18f631 00007f85be8c23f1 00007f85be7fb9fe 00007f85be882749 00007f85be88376a 00007f85be88291b 00007f85be88e03c 00007f85be88d6b3
0709 12:20:06.950340 (+ 38236us) log.cc:844] Serialized 494 byte log entry
0709 12:20:07.196179 (+245839us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b18000b7040. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb714c0a 00007f85c4e54219 00007f85bb79ca40 00007f85bb7974e4 00007f85bb797455 00007f85be7e0b0b 00007f85be865b63 00007f85be88d752 00007f85be8859fa
0709 12:20:07.214206 (+ 18027us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b5400066e20. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be800142 00007f85be7e0eaa 00007f85be7e7d53 00007f85be7e7cd9 00007f85be7e7c69 00007f85be7e7c31 00007f85be7e6dfd 0000000000388634 0000000000388479 00007f85bb79a475
0709 12:20:08.508041 (+1293835us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5957,"raft.queue_time_us":30105,"raft.run_cpu_time_us":2629,"raft.run_wall_time_us":37748,"spinlock_wait_cycles":49319296,"thread_start_us":221311,"threads_started":1,"wal-append.queue_time_us":193044,"wal-append.run_cpu_time_us":1701,"wal-append.run_wall_time_us":1001707}
I20260709 12:20:08.513443 28176 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9c99f01ab7374acfa37f7669b8fcdeb2 with cas_config_opid_index 18: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:08.521543 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c99f01ab7374acfa37f7669b8fcdeb2 with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 19) from {username='slave'} at 127.0.0.1:39084
I20260709 12:20:08.602205 28292 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:08.687775 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:08.691893 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 86908.3us
W20260709 12:20:08.704409  3150 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.106s	user 0.008s	sys 0.010s
W20260709 12:20:08.703886  3149 log.cc:927] Time spent T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.004s
I20260709 12:20:08.685573  3124 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:20:08.812105  3153 tablet_replica.cc:333] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
W20260709 12:20:08.881345 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 134992us
I20260709 12:20:08.894652  3153 raft_consensus.cc:2243] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:08.895895  3153 raft_consensus.cc:2272] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:08.903575  3153 ts_tablet_manager.cc:1916] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:08.908792  3028 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.914296  3075 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.923151 28283 raft_consensus.cc:1240] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:08.927050 28982 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:08.940472 28438 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 12:20:08.920920  3152 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:20:08.921257  3147 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:20:08.944738 28438 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.915000  3075 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:08.947799 28360 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:20:08.953125 28283 raft_consensus.cc:3060] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:08.948499 28360 raft_consensus.cc:606] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.961194  3075 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:08.963167 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:08.963495  3075 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:08.963975 28359 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:20:08.970685 28432 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:08.986219 28440 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:08.987469 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:08.988168 28288 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:20:08.982913 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:08.999083 28288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "777967d6330c4654813298b12226a560"
dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
mode: GRACEFUL
new_leader_uuid: "6634fa3fbd594726a5ec3c1686900284"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:20:08.999758 28288 raft_consensus.cc:606] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Received request to transfer leadership to TS 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:09.003173 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:20:09.003783 28359 raft_consensus.cc:606] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:08.955708  3160 consensus_queue.cc:1059] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:20:09.017176 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:20:09.017815 28359 raft_consensus.cc:606] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:09.021185  3028 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.023710 28359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:20:09.024364 28359 raft_consensus.cc:606] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:09.028983 28982 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:09.027731 28438 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:09.032620 28439 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:09.033375 28283 raft_consensus.cc:1275] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:09.030802 28292 raft_consensus.cc:1275] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:20:09.033671 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 50866.8us
I20260709 12:20:09.048810 28432 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:09.048945 28440 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:09.050470 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 104474us
I20260709 12:20:09.095376 28440 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:09.096736 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:09.097265 28364 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:20:09.096645 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 144787us
I20260709 12:20:09.046633  3118 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:20:09.108395 28439 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:20:09.122611 28292 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:09.123029 28364 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
mode: GRACEFUL
new_leader_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:41314
I20260709 12:20:09.123544 28364 raft_consensus.cc:606] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Received request to transfer leadership to TS 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:09.096155  3119 consensus_meta.cc:287] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.589s	user 0.014s	sys 0.000s
I20260709 12:20:09.137696 28359 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:09.117887 28183 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index 11 to 10, term changed from 13 to 15, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2), VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) added, now has a pending config: VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2), VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3), VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 15 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:20:09.144045 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 89017.7us
I20260709 12:20:09.152570 28434 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:20:09.172857 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 68111.5us
I20260709 12:20:09.119823  3168 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:09.179361 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 142650us
I20260709 12:20:09.179633 28432 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:09.146934  3171 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:20:09.115541  3056 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:20:09.135143  3166 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:09.140055  3028 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 12:20:09.193480 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 106142us
W20260709 12:20:09.193281 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 71963.2us
I20260709 12:20:09.170506  3170 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.134624  3088 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:20:09.206040  3093 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:20:09.219731 28982 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:09.224915 28982 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:09.234627  3171 consensus_queue.cc:1048] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:09.235896 28439 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:09.236316 28989 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:09.233037 28367 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:20:09.194981  3166 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.245608  3166 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:09.190529  3168 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.195461  3160 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.254240  3160 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.195067  3179 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:09.256491  3179 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.258395  3167 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:20:09.239203  3172 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.239753  3169 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:09.239852 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:09.281677 28439 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:09.283504 28989 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:09.284677  3172 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.290343  3172 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.216768  3170 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.292884  3170 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.262171  3174 consensus_queue.cc:1048] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 12:20:09.286685  3169 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.416095  3179 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.442385  3168 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.437376  3169 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.511495 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:20:09.517149 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 281447us
W20260709 12:20:09.521376 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 306433us
I20260709 12:20:09.534870 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:09.536798 28292 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:20:09.541078 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:09.560274  3101 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:20:09.573599  3126 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:09.575049  3126 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.578686 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:09.562785  3159 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:09.583992  3169 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.651753  3037 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.652211  3129 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.652917  3037 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.653954  3186 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:09.655050  3186 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.657351  3158 raft_consensus.cc:3055] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:09.657953  3158 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:09.660094  3105 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:09.663934  3181 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.659696  3158 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.661067  3105 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.671938  3169 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.673972  3186 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:09.674664  3186 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.675458  3186 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.678580  3182 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:09.680292  3170 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.681231  3170 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.684806  3176 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.653265  3129 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.685716  3176 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.686772  3183 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.687656  3183 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.688629  3170 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.665091  3181 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.691824  3181 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:09.670722  3159 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.695927  3159 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.689421  3170 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.698148  3185 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:09.642351 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:09.700479 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 142943us
I20260709 12:20:09.638196  3168 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.702989 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:09.703845 28283 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 26.
I20260709 12:20:09.705909  3111 raft_consensus.cc:993] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:20:09.702631  3168 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.674985  3169 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.728884 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:09.730562  3180 raft_consensus.cc:1081] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:20:09.727015 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:09.733080 28283 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:09.724902  3185 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.737831  3110 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:09.738170  3185 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.738741  3110 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.739773  3174 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:09.740267  3174 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.740675  3174 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:09.742024  3037 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.746109  3178 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:09.746821  3178 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:09.748337  3129 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:09.749218 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 66370.1us
I20260709 12:20:09.750447 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:09.751125  3176 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.753327  3105 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.753381 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:09.755431  3188 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.756460  3155 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:09.756966 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:09.756361  3179 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.758538  3182 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.759789  3183 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.762271 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:09.759553 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.757977  3168 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.767854 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:09.772048 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:09.758092  3157 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.773631  3181 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.782243 28365 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 23: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:09.785439 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:09.756297  3185 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.748647  3178 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.775141  3110 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.793309 28358 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:20:09.736330  3170 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.798866 28362 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 19: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:09.799423 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:09.799793  3157 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.799475 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:09.801023  3183 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.829339  3159 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:09.829576  3183 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.830227  3159 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.832459  3183 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.832877  3159 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.801910  3086 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.802642  3155 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.834082  3179 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.836450  3155 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.834173  3086 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.828986 28359 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:09.798717  3182 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.839304  3086 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.840108 28364 raft_consensus.cc:2393] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 26: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:09.803941  3188 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.841271  3182 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.814432  3181 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.814463 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:09.820042 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:09.798559 28360 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:20:09.810778  3126 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.813364  3168 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:09.822562  3186 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.828680  3127 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.842967  3188 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:09.806603  3160 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:09.857257 28292 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:09.828580  3157 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.836736  3179 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.869100  3126 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:09.869989  3127 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.870366  3174 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.871057  3160 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.899008  3160 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.900009  3127 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.871402  3182 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:09.881464  3089 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.885901  3054 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.886644  3195 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:09.911129  3195 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.881815 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:09.896636 28362 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:20:09.914047  3126 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.897353 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:09.901185  3178 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:09.915932 28366 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:09.899937  3174 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.906119  3194 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:09.907605  3089 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.924386  3089 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.889976  3156 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.925422  3156 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.928719  3156 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.910089  3054 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.931622  3054 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.909536  3182 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.917802  3075 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:09.934427  3075 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.937361  3075 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:09.901325  3160 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.940618 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:09.941999  3168 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.917583  3080 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.947220  3080 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:09.939010  3182 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:09.921949  3195 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.932542 28185 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 18 to 19, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 19 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.953064  3080 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:09.916779  3126 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:09.920064  3181 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:09.953819 28185 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 12 to 13, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 13 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.941485 28434 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:09.959247 28185 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 19 to 20, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 20 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.967178 28185 catalog_manager.cc:5697] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 17, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.975217 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:09.976080 28433 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:09.976812 28185 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 20 to 21, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 21 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.864069  3186 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:09.982918  3186 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:09.983180 28438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:20:09.977919 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:09.984336 28433 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:09.980609 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:09.983474  3186 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:09.985496 28432 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:09.964495  3168 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.990039  3191 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.990815  3191 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.993338  3191 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:09.998832  3110 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:09.999871  3183 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:10.000612  3171 raft_consensus.cc:993] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:20:10.001204  3096 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:10.003038 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 74151.2us
I20260709 12:20:10.002699  3186 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:09.984760 28185 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 19 to 20, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 20 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:10.003599  3168 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.011242  3170 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:09.962081  3158 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:10.028276  3175 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:10.036288 28438 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:10.057794 28438 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:10.035547  3195 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:10.063773  3193 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:10.064280 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:10.067991 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.068768  3171 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:10.069271  3171 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.058974 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.061347 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.038110  3086 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.068944 28987 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:20:10.073724 28292 raft_consensus.cc:2393] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 18: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
W20260709 12:20:10.070358 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 319993us
I20260709 12:20:10.075062 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.076748 28433 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:10.069674  3171 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:10.093461 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 222904us
I20260709 12:20:10.013451  3153 ts_tablet_manager.cc:1929] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 26.18
I20260709 12:20:10.095288 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:10.112157 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.114183 28433 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:10.089998  3170 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:10.089859  3194 raft_consensus.cc:1081] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:20:10.110374  3191 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.151516  3191 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.122130  3104 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.122673  3177 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:10.123109  3163 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.123363  3033 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.123466 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:10.125828  3174 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.128466 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.129554 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.129839  3153 log.cc:1199] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/9c99f01ab7374acfa37f7669b8fcdeb2
I20260709 12:20:10.132045  3054 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.102933  3086 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.135025 28332 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:10.135869  3166 consensus_meta.cc:287] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.889s	user 0.007s	sys 0.014s
I20260709 12:20:10.136253  3195 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.137138 28433 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:10.137249  3074 raft_consensus.cc:993] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:20:10.138046 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.093657  3168 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.109011  3175 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:10.142305 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.142838  3186 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:10.143428 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.144971 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.077064  3096 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.114787  3075 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:10.149717 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.150017  3170 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:10.156248 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 61072.3us
I20260709 12:20:10.153403 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.126072 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.162313 28286 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:20:10.179848 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.182528 28987 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:20:10.186714 28984 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:10.102646  3193 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.192324 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.202663 28285 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:20:10.192591  3033 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:10.193619  3191 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.194656  3153 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39084 (request call id 11) took 1673 ms. Trace:
I20260709 12:20:10.194936 28956 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:10.195114  3201 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:10.199503 28982 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
I20260709 12:20:10.191470 28989 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 26.
I20260709 12:20:10.208499 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.225545 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.192430  3169 raft_consensus.cc:1081] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:20:10.224261 28988 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 23.
I20260709 12:20:10.209425  3074 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:10.215106  3201 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.238936 28434 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
W20260709 12:20:10.229161 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 163635us
I20260709 12:20:10.238238 28985 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:20:10.187410  3166 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.188386  3174 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.228087  3199 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.223795  3186 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:10.258986 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 124065us
I20260709 12:20:10.260428 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.262450  3096 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.264297  3033 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.245781  3177 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:10.266426 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.267338 28289 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 26.
I20260709 12:20:10.267669 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.268355 28437 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:20:10.269210  3193 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:10.269942  3193 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.270560  3193 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.272118 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.246203  3075 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.246723  3201 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:10.212937  3104 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.275691  3075 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.276357  3075 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:10.267827 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 73146us
I20260709 12:20:10.276822  3104 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.278734  3086 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.212706  3153 rpcz_store.cc:276] 0709 12:20:08.520928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:08.521243 (+   315us) service_pool.cc:224] Handling call
0709 12:20:08.734672 (+213429us) ts_tablet_manager.cc:1151] Deleting tablet 9c99f01ab7374acfa37f7669b8fcdeb2
0709 12:20:08.736731 (+  2059us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:20:09.446724 (+709993us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:10.013096 (+566372us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:10.194541 (+181445us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30893,"tablet-delete.queue_time_us":189077,"thread_start_us":60384,"threads_started":1}
I20260709 12:20:10.280301  3175 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.215972  3163 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.226104  3054 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:10.282112 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.282902 28286 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:20:10.283591  3054 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.284020 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet 9c99f01ab7374acfa37f7669b8fcdeb2 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
I20260709 12:20:10.266299 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.287261 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:10.287413 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.287993 28287 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:10.288400 28290 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:10.290063  3163 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.269472 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.291244 28437 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:10.291868  3177 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.292452 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.292629  3199 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.293146 28437 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:20:10.229970  3191 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.294348  3191 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.292778 28956 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.282295  3195 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.296517  3191 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:10.296705  3212 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:10.297230  3212 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:10.257903  3101 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:10.297788  3212 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:10.294842 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.298477  3101 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:10.297268 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.238336  3074 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:10.312304 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.312783 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.299722 28988 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:10.313494 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.294711 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:10.311358 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.335867 28434 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 19
W20260709 12:20:10.337044 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 75971us
I20260709 12:20:10.315449 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.335425 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.335064 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.338482 28366 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:20:10.315068 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.304477 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:10.337822 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.337980 28363 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:20:10.337808 28330 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:10.303782  3206 log.cc:927] Time spent T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.063s	user 0.003s	sys 0.000s
I20260709 12:20:10.342530 28956 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.345481 28433 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:20:10.314860  3205 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.159s	user 0.001s	sys 0.001s
W20260709 12:20:10.343998 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 57083us
I20260709 12:20:10.346024 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.343411 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.344766 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.350107 28359 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:20:10.352623 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.352906 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.353638 28364 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:20:10.354020 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.353531 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.354866 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.302402  3170 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.355475 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.346690 28290 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:20:10.329892  3074 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:10.361899 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 57512.7us
I20260709 12:20:10.356065 28362 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:10.307313  3166 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:10.303556  3174 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.368213 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.370268 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.374612 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.371349 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.377951  3124 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:10.378409  3124 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.378765  3124 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:10.379283 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.378126 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.383694 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.384755 28433 raft_consensus.cc:2410] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:20:10.378611 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.383852  3054 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:10.388952  3054 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.389350  3054 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:10.391011 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.357561 28438 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.396040 28287 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:10.392453 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:10.297050  3186 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.365777 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:10.360430  3191 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:10.375345 28982 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:20:10.404932  3155 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.399976 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:10.370968  3195 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:10.370822  3194 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:10.408114 28438 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.373149  3163 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:10.412578 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:10.416121 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.408759 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:20:10.417018 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 71360.8us
I20260709 12:20:10.414836  3165 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:10.421152 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.299687  3101 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.424149 28990 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 20: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:10.424083 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.407898  3191 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.426301 28985 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:20:10.429507 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.411369  3105 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:10.430251 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.432351 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.418813  3163 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.409797  3195 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.412137  3194 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.406419  3155 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.438766  3168 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.407759  3193 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.442317 28989 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 23.
I20260709 12:20:10.420511  3096 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:10.420334  3161 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.444327 28956 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:10.445945 28404 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:10.427311  3183 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:10.428383  3185 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:20:10.475416 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 80120.6us
I20260709 12:20:10.468339 28982 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:20:10.425807  3165 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:10.441953  3105 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.443504  3191 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:10.448262  3163 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:10.504369 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 63345.9us
I20260709 12:20:10.506145 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
W20260709 12:20:10.509941 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 65699.2us
I20260709 12:20:10.467959  3194 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:10.468307  3195 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:10.531415 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.501235  3183 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.498234  3096 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.531579 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.534759 28437 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:20:10.509229  3165 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:10.511741  3105 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:10.505157  3185 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.527307  3075 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:10.507018  3199 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.512204  3158 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:10.555614 28404 leader_election.cc:400] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
I20260709 12:20:10.563437 28404 leader_election.cc:403] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:10.526520 28439 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:20:10.559624 28434 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:10.547389  3096 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:10.540014  3183 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:10.560245  3185 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:10.559664  3181 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.554162  3188 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:10.556787  3166 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:10.592448 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 64377.6us
I20260709 12:20:10.565786  3169 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:10.597048  3155 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.600476 28989 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:20:10.566398  3161 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.602262 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:10.620342 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.604106  3212 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.610579  3104 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:10.563298  3158 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:10.614203 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:10.623591 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.624004 28982 raft_consensus.cc:1275] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:10.625706  3180 raft_consensus.cc:993] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: : Instructing follower 9aea2391841b4888b4e195e5a1f4623b to start an election
I20260709 12:20:10.626267  3161 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.626087  3188 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.630716  3198 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.621053 28988 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 13.
I20260709 12:20:10.629921  3172 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:10.652216 28290 raft_consensus.cc:2410] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:20:10.613372  3190 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
I20260709 12:20:10.608294  3171 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.659081 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:20:10.662930 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 156886us
I20260709 12:20:10.659677 28986 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:20:10.653191  3158 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:10.673993 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:10.670734 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 56626.6us
I20260709 12:20:10.673308 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.674634 28989 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 26.
I20260709 12:20:10.675474 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.683382 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:10.684092 28989 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
W20260709 12:20:10.685657 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 173009us
I20260709 12:20:10.685979  3233 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:10.687484  3233 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:10.688099  3233 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:10.695132  3180 raft_consensus.cc:1081] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Signalling peer 9aea2391841b4888b4e195e5a1f4623b to start an election
W20260709 12:20:10.700902 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 232480us
I20260709 12:20:10.703022 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.703728 28439 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:20:10.711266 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.713143 28437 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:20:10.716809 28437 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:10.717213 28437 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:10.714769 28432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:20:10.718219 28432 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:10.718582 28432 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:10.715809 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.723990 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.726783 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.727559 28286 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:10.728048 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.728740 28435 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:10.734215 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.735008 28435 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:10.736294 28435 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:10.648608  3181 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.751348 28988 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 22: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:10.751679 28435 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0"
dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
 from {username='slave'} at 127.0.0.1:33102
I20260709 12:20:10.752334 28435 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:10.752704 28435 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:10.754889 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 65282.4us
I20260709 12:20:10.658764  3155 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:10.756160 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.670243  3186 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.756779 28289 raft_consensus.cc:1813] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Handling vote request from an unknown peer 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:10.676613  3178 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:10.779458 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.780547 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.759509 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.760361 28289 raft_consensus.cc:2410] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:20:10.654639  3166 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:10.764012  3219 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:20:10.766041  3236 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:10.774128 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:10.775168 28954 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:10.739805  3169 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.778350 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.737972  3215 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:10.783612 28287 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 26.
I20260709 12:20:10.796264 28361 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 18: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:10.805392 28956 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.799993 28330 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:10.807319 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:10.811545  3234 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:20:10.812424 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.812891 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.813552 28289 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:20:10.759840  3190 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:10.827627 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 152222us
I20260709 12:20:10.822160 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.826486 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.828387 28437 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.779290  3178 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:10.670002  3198 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.796355  3236 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.670003  3172 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:10.773453  3124 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.844197 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.819464  3075 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:10.828632  3186 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:10.828307  3199 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:10.830883  3193 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:20:10.854013  3217 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:20:10.852901  3237 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:10.829181  3190 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:10.903904 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.884497 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.843298  3178 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:10.877792 28437 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:10.896960 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 97049.6us
I20260709 12:20:10.818109  3155 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:10.908830 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 76854.4us
I20260709 12:20:10.845381  3236 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:10.914454 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.855332  3186 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.849742  3213 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:10.924592  3239 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:10.849514  3215 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.926442 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.856899  3170 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:10.913229 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.915148 28441 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:20:10.929848 28437 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:20:10.907266  3225 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:10.910656  3190 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:10.903051  3174 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:10.956143 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:20:10.957115 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 44069.1us
I20260709 12:20:10.961724  3171 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.953523 28435 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:10.969321 28435 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:10.967409 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:10.971362 28291 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:10.973301  3172 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.966346  3185 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:10.975095  3201 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.000s
I20260709 12:20:10.975548  3201 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:10.976274  3185 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.969808  3194 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.941061  3170 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.979017  3155 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:10.979483  3194 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.948436  3212 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.981153  3166 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:10.982095 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.982360  3169 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.855321  3054 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.983714  3124 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 27 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.984596  3054 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:10.985191 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:10.912205  3226 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:10.985405  3195 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.927399  3237 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.986678 28984 raft_consensus.cc:2393] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 20: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:10.987744  3195 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.941442  3096 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:10.990073  3246 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:10.990579  3096 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.935684  3225 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.990543  3246 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.991748  3246 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:10.996225  3191 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.998991 28983 raft_consensus.cc:2393] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 20: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:10.998548  3191 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:10.928953  3186 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:11.003118 28291 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:20:11.002978  3236 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.004774  3246 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:10.985615  3215 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:10.993389 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.020007 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.024901 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:10.991672  3225 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:10.918759  3126 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:10.991246 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.023816 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.027392 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.025619 28441 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:10.987979  3237 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.030936 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.035884 28954 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.036801  3191 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:11.037319  3191 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 26 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:11.005563  3236 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.038386  3195 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:11.038478  3191 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.037240 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:10.969125  3165 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:10.993201  3190 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.029866 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.040324 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:10.979977  3170 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:11.024747  3258 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.052s	user 0.004s	sys 0.000s
I20260709 12:20:10.977665  3201 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:10.925523  3233 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:11.029011  3254 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.066s	user 0.002s	sys 0.008s
I20260709 12:20:11.049360 28435 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:20:11.024106  3193 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:11.061242 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.028249  3215 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:11.027506  3155 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:10.969480  3174 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:11.085004 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 253482us
I20260709 12:20:11.031262  3178 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.022018  3213 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.067044  3170 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:11.091738  3245 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:11.092149  3227 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:20:11.092635  3105 consensus_meta.cc:287] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
W20260709 12:20:11.096283  3204 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.564s	user 0.002s	sys 0.004s
I20260709 12:20:11.099013  3190 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.100567  3165 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.075049  3155 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.042773  3226 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.088246  3201 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:11.065109  3126 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.088201 28432 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.101718  3124 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.132694  3124 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.133777  3175 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:11.134364  3175 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.134766  3124 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.104194 28183 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 325) took 2617 ms. Trace:
I20260709 12:20:11.135329  3110 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:11.111120  3163 consensus_meta.cc:287] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.002s
I20260709 12:20:11.114262 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.104489  3186 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.139222  3075 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:11.138808 28956 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.116312  3213 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:11.140393  3195 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:11.140754  3186 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.116050  3245 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.142273  3245 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.136821  3110 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.119938  3155 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:11.140935  3195 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:11.137648  3163 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.147468  3195 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.121677  3226 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.121206  3201 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.150049 28283 raft_consensus.cc:2368] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15.
I20260709 12:20:11.128731 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:11.139683  3075 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.123535  3126 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:11.118727 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 3466) took 1094 ms. Trace:
I20260709 12:20:11.117244  3105 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.126348  3193 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.153434 28440 rpcz_store.cc:276] 0709 12:20:10.024528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.024673 (+   145us) service_pool.cc:224] Handling call
0709 12:20:10.025050 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:10.025509 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.025545 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:10.026258 (+   713us) log.cc:844] Serialized 26 byte log entry
0709 12:20:10.157579 (+131321us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:10.157610 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.157667 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:10.995010 (+837343us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:20:11.118285 (+123275us) raft_consensus.cc:1685] finished
0709 12:20:11.118308 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.118524 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":162,"spinlock_wait_cycles":133559040,"thread_start_us":130842,"threads_started":1,"wal-append.queue_time_us":369060}
I20260709 12:20:11.155404  3105 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.150695  3201 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:11.112576  3158 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:11.113933  3074 consensus_meta.cc:287] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.706s	user 0.006s	sys 0.006s
I20260709 12:20:11.157338  3188 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.157912 28433 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:11.149606 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.116050  3170 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:11.118223 28436 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 23.
I20260709 12:20:11.124787 28432 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:11.129817 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.150772 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.138104 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.152380 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.163046 28439 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.162283 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:11.163311 28434 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:11.126684 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:11.149461  3163 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.162634 28956 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.168439 28257 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:11.175302  3177 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:11.175806  3177 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.176173  3177 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:11.160619  3158 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.180699 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.160286  3170 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.182443  3110 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.151657  3075 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:11.185091 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.101992  3181 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.155583  3198 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.118949  3178 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:11.188484 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 100034us
I20260709 12:20:11.189767 28432 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 19: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:11.191576 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:11.130326  3174 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:11.136490  3175 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.192529  3191 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:11.114567  3233 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.200220 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.207077 28434 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:11.204254 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.200698 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.172375  3105 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:11.206080 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:11.205001 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.208726 28359 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:11.104475  3183 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.157303  3074 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.207852 28361 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:11.214088  3105 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.207466  3163 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:20:11.223758 28288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 1587) took 1022 ms. Trace:
I20260709 12:20:11.227484 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.227725 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.230490 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.228878 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.229641 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.242360 28362 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:20:11.236064 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.234035 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.237630 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.236171 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.259372 28987 raft_consensus.cc:3060] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:20:11.235520 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.225003 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.234793 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.261364 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.263022 28360 raft_consensus.cc:2368] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15.
I20260709 12:20:11.259778 28985 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:11.261318 28984 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.227133 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:20:11.262738 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 62106.3us
I20260709 12:20:11.235842 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.243057 28982 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:11.276129 28990 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:11.265410 28434 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:11.277946 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.234799 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.219574  3261 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:11.278895 28989 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:20:11.239706 28288 rpcz_store.cc:276] 0709 12:20:10.201363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.201505 (+   142us) service_pool.cc:224] Handling call
0709 12:20:11.223541 (+1022036us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b54000b0254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:11.223692 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2246981888}
I20260709 12:20:11.222934 28185 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 318) took 2012 ms. Trace:
I20260709 12:20:11.229236 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.135830 28183 rpcz_store.cc:276] 0709 12:20:08.486406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:08.486762 (+   356us) service_pool.cc:224] Handling call
0709 12:20:08.687881 (+201119us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f1090 00007f85be88ff3a 00007f85c4645e9c 00007f85c474404e 00007f85c2c052a4 00007f85c2c05232 00007f85c2c05161 00007f85c2c050dc 00007f85c2c04392 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:11.103938 (+2416057us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:09.644620 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:09.732112 (+ 87492us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:09.732169 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:20:09.732201 (+    32us) tablet.cc:662] Decoding operations
0709 12:20:09.732800 (+   599us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:09.732899 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:09.732953 (+    54us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:09.760191 (+ 27238us) tablet.cc:701] Row locks acquired
0709 12:20:09.760250 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:20:09.760496 (+   246us) write_op.cc:270] Start()
0709 12:20:09.760670 (+   174us) write_op.cc:276] Timestamp: P: 1783599609760377 usec, L: 0
0709 12:20:09.760709 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:20:09.761894 (+  1185us) log.cc:844] Serialized 780 byte log entry
0709 12:20:10.126146 (+364252us) op_driver.cc:464] REPLICATION: finished
0709 12:20:10.410889 (+284743us) write_op.cc:301] APPLY: starting
0709 12:20:10.422697 (+ 11808us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:10.762730 (+340033us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:10.762878 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:10.763851 (+   973us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:10.764174 (+   323us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:10.764230 (+    56us) write_op.cc:312] APPLY: finished
0709 12:20:10.764516 (+   286us) log.cc:844] Serialized 25 byte log entry
0709 12:20:10.764738 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:10.764832 (+    94us) write_op.cc:454] Released schema lock
0709 12:20:10.764887 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:10.765470 (+   583us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:10.765498 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":266405888,"child_traces":[["op",{"apply.queue_time_us":240332,"apply.run_cpu_time_us":5796,"apply.run_wall_time_us":366578,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8305,"num_ops":1,"prepare.queue_time_us":463443,"prepare.run_cpu_time_us":4935,"prepare.run_wall_time_us":122121,"replication_time_us":356356,"thread_start_us":584574,"threads_started":2}]]}
I20260709 12:20:11.236822  3105 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.284857  3074 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.216773  3178 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:11.296059 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.296576 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.297310 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.230234 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.295441 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.305866 28364 raft_consensus.cc:2368] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 26. Current term is 27.
I20260709 12:20:11.311726 28983 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:11.325356 28361 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
W20260709 12:20:11.284008  3264 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:20:11.324355 28359 raft_consensus.cc:2393] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 17: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:11.322674 28987 raft_consensus.cc:2468] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:20:11.330186 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:20:11.314245 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 79554.2us
I20260709 12:20:11.331967 28283 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.332294 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.332207 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.332564 28366 raft_consensus.cc:2410] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:20:11.325332 28363 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:20:11.333573 28982 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 24.
I20260709 12:20:11.334548 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.333091 28360 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 14.
I20260709 12:20:11.335153 28287 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:11.337244 28954 leader_election.cc:400] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15.
I20260709 12:20:11.337857 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.338716 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.339510 28332 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.282807  3228 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:11.340622 28365 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:11.338768 28361 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:20:11.340998 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.324731  3195 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:11.340420 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.341701 28954 leader_election.cc:403] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:11.345016 28358 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 17: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:11.233870 28189 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 reported cstate change: config changed from index 10 to 11, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted, no longer has a pending config: VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2), VOTER 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3), VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 15 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:11.340368 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.345969 28359 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.328119 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:11.343181 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.347038 28984 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 23.
I20260709 12:20:11.347353 28360 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.348057 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.348562  3194 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:11.348820 28290 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:11.349547  3074 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.350157  3074 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.350456  3075 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.352355  3074 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.352921 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.353572  3075 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.344429 28364 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:20:11.354962 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.357465 28360 raft_consensus.cc:2468] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:11.344035 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.359191 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.359736  3246 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.353847 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.364639  3201 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.353147  3155 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.366289  3251 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:11.366770  3251 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.367169  3251 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.367664  3236 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:11.368173  3155 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.370307  3163 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:11.371243 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:11.373873 28438 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:11.382457 28439 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:11.383809 28956 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:11.385341 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 45920.6us
I20260709 12:20:11.375193 28991 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 14.
I20260709 12:20:11.389287 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.341823  3261 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.392750 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.394907  3124 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:11.395370  3169 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:11.389653 28363 raft_consensus.cc:1240] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:11.384358  3086 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:11.393355  3104 consensus_queue.cc:1048] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:11.305567 28185 rpcz_store.cc:276] 0709 12:20:09.210567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.214323 (+  3756us) service_pool.cc:224] Handling call
0709 12:20:11.222831 (+2008508us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:10.069028 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:10.069714 (+   686us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:10.069768 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:20:10.069785 (+    17us) tablet.cc:662] Decoding operations
0709 12:20:10.137654 (+ 67869us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:10.137707 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:10.137755 (+    48us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:20:10.138348 (+   593us) tablet.cc:701] Row locks acquired
0709 12:20:10.138364 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:20:10.138542 (+   178us) write_op.cc:270] Start()
0709 12:20:10.138681 (+   139us) write_op.cc:276] Timestamp: P: 1783599610138491 usec, L: 0
0709 12:20:10.138703 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:20:10.297781 (+159078us) log.cc:844] Serialized 2439 byte log entry
0709 12:20:10.372951 (+ 75170us) op_driver.cc:464] REPLICATION: finished
0709 12:20:10.432986 (+ 60035us) spinlock_profiling.cc:243] Waited 82.4 ms on lock 0x7b5400196254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be882e28 00007f85c22bd581 00007f85c22bcd58 00007f85c22c19f1 00007f85c22c19a9 00007f85c22c1939 00007f85c22c1901 00007f85c22c0b2d 0000000000388634 0000000000388479
0709 12:20:10.664411 (+231425us) write_op.cc:301] APPLY: starting
0709 12:20:11.094259 (+429848us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b54001960a8. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4c439e1 00007f85c4c4264c 00007f85c22d2e10 00007f85c22be97c 00007f85c22c2d61 00007f85c22c2d19 00007f85c22c2ca9 00007f85c22c2c71 00007f85c22c1e9d 0000000000388634 0000000000388479
0709 12:20:11.094390 (+   131us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:11.108070 (+ 13680us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:11.108092 (+    22us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:11.127262 (+ 19170us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:11.127569 (+   307us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 12:20:11.127613 (+    44us) write_op.cc:312] APPLY: finished
0709 12:20:11.127932 (+   319us) log.cc:844] Serialized 65 byte log entry
0709 12:20:11.128741 (+   809us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:11.128876 (+   135us) write_op.cc:454] Released schema lock
0709 12:20:11.128928 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:11.129484 (+   556us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:11.129514 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":634,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":273042,"apply.run_cpu_time_us":21461,"apply.run_wall_time_us":472626,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29195,"num_ops":6,"prepare.queue_time_us":413,"prepare.run_cpu_time_us":8049,"prepare.run_wall_time_us":364157,"replication_time_us":201939,"spinlock_wait_cycles":327371008,"thread_start_us":127343,"threads_started":1}]]}
I20260709 12:20:11.553695 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.403481 28989 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:11.492197 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.403142 28288 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.368136  3236 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.526510 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:11.530455 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 202424us
I20260709 12:20:11.556823 28284 raft_consensus.cc:2393] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:11.552985 28283 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 23.
I20260709 12:20:11.557530 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.330065  3226 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.344097  3178 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.543778 28287 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 27.
I20260709 12:20:11.367161  3183 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.569442 28332 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:11.548516 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.522531  3170 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15.
I20260709 12:20:11.536548  3181 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:11.545305  3188 raft_consensus.cc:2764] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:20:11.566438 28330 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.555238 28984 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:11.557405 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:11.530067  3075 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:11.569015 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.573922 28986 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:20:11.570459 28983 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:20:11.551633  3124 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.403704  3194 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.585665 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.576578 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.574141 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.580015 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:11.558914  3236 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:11.576068 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.592028 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.556640  3155 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:11.597128 28287 raft_consensus.cc:2368] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19.
I20260709 12:20:11.373314  3228 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.569123  3239 raft_consensus.cc:2764] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:20:11.605213 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.552578  3225 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.553795  3161 consensus_queue.cc:1059] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 11.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:20:11.553277  3245 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.631559 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.632359 28982 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:11.594288  3242 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.586611  3124 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:11.594642  3180 raft_consensus.cc:2764] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:20:11.642930 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.600776  3075 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:11.637020 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.638656 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.650084 28987 raft_consensus.cc:2368] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 22. Current term is 23.
I20260709 12:20:11.606109  3183 raft_consensus.cc:2804] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:11.611532  3101 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:20:11.617893  3265 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 12:20:11.653038 28257 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:11.582479  3181 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.665693 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:11.614959  3155 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.556104  3169 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.575656  3178 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.569640  3086 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.648558 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: true dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.638624  3261 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.682176  3246 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:11.601349  3126 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:11.695724  3126 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.696192  3126 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:11.580513  3226 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.231585  3110 raft_consensus.cc:2955] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, VOTER a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:11.678121  3183 raft_consensus.cc:697] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:11.585892  3252 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:11.718622 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.628580  3074 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:20:11.659618  3266 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:20:11.685876 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:11.650977  3224 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:11.688140 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.676362  3104 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:20:11.675683 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.728744 28987 raft_consensus.cc:2368] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 23. Current term is 24.
I20260709 12:20:11.694172 28983 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:11.647119 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.730444 28988 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:20:11.676162  3175 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:11.700845 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.656126 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.735489 28990 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:11.708678 28982 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:20:11.740815 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.741638 28362 raft_consensus.cc:3055] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:11.742131 28362 raft_consensus.cc:740] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:11.742885 28362 consensus_queue.cc:260] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.744513 28362 raft_consensus.cc:3060] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:11.754069  3224 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 23 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:11.755223  3224 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.756983  3086 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:11.758499  3177 consensus_meta.cc:287] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.582s	user 0.048s	sys 0.068s
I20260709 12:20:11.755672 28362 raft_consensus.cc:2468] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:20:11.759949 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.762440  3178 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:11.766587  3181 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:11.729542  3074 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 24 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:11.768100  3074 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.726625  3246 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:11.774108 28438 raft_consensus.cc:1240] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:11.779066  3245 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.782889  3225 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.783099  3169 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.783785  3175 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:11.784992  3175 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.785466  3183 consensus_queue.cc:237] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.717875  3201 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.717376  3252 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.645323 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.714811 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.656056  3242 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.714818 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.725409 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:11.723887 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 123848us
I20260709 12:20:11.726790 28366 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:20:11.688905 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.713611 28288 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 23.
I20260709 12:20:11.798959 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:20:11.790306 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 220956us
I20260709 12:20:11.799989 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.800613 28987 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:11.804994 28991 raft_consensus.cc:2393] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 15: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:11.805783 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.807013 28439 raft_consensus.cc:1240] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:11.810276 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.812243 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.813122 28439 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 17: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:11.798185 28366 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:11.819800 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.820803 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:11.820703 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:20:11.822407 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 97085us
I20260709 12:20:11.798341 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:11.823228 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.824481 28954 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.784898  3155 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.834095 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.834861 28440 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:11.825421 28990 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:11.708098  3101 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.826561 28988 raft_consensus.cc:2368] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:20:11.826511 28365 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:20:11.840864 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:11.841217 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.842615 28987 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:20:11.840927 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.827289 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.830541 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:11.842713 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.845502 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.828707 28287 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:11.846588 28290 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 19: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:11.860335 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.862154 28292 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.862637 28289 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:11.847653 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.860673 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.864387 28432 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.864989 28291 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:11.798297  3252 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.866701 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:11.867305 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:11.818398  3201 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:11.890838 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54466.7us
I20260709 12:20:11.889467 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:11.866344  3186 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:11.824512  3158 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:11.890285 28985 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:11.893333 28404 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:20:11.890450 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.868682 28954 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.899986 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.901752 28365 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:11.902125 28365 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:11.902698 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:11.839097  3245 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:11.794922  3177 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.843917  3246 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:11.905366 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:11.908931 28987 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:11.911130  3228 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.860898  3191 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:11.908839 28990 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 12:20:11.907675 28433 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:11.903096 28365 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:11.841023  3260 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:11.860046  3155 raft_consensus.cc:2764] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:20:11.906401 28954 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:11.923691 28359 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.000s
I20260709 12:20:11.871429  3033 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:11.929703 28433 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:11.791106  3075 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.930212 28365 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.935253 28954 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.931620 28359 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:11.828246  3251 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.931046 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.939168 28283 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 21: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:11.937629 28983 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:20:11.945246 28954 leader_election.cc:400] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 14. Current term is 15.
I20260709 12:20:11.946406 28954 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.947721 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.827349  3124 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.912288  3198 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:11.909051  3174 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:11.954447 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.965472 28434 raft_consensus.cc:2368] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19.
I20260709 12:20:11.949530 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:11.908413  3194 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.967615 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:11.934257  3188 raft_consensus.cc:2804] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:11.927601  3233 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:11.968633 28359 raft_consensus.cc:2368] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:20:11.968837 28365 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:11.969719 28954 leader_election.cc:400] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 26. Current term is 27.
I20260709 12:20:11.904537  3201 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:11.951993  3193 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:11.982784 28954 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:11.912210  3246 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.984032 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:11.928658  3260 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:11.997071  3260 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:11.963775  3195 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:12.002316  3195 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:12.004237  3251 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.004822 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.007520 28366 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 11.13, Last appended by leader: 13, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.009948  3260 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.013221  3124 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:11.980084 28364 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:11.985317 28361 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 22: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:11.994419  3242 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:11.985297  3246 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:11.964570  3198 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.018046  3198 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:11.996006  3219 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.432s	user 0.000s	sys 0.003s
I20260709 12:20:11.996518 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:12.023301 28954 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:12.024724 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:12.026052  3195 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:12.026913  3195 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:12.027261  3195 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:12.027668  3195 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:11.961793  3170 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:11.961196  3165 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:12.029911  3174 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:12.030148  3170 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:12.030850  3170 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.030131 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:12.039569 28432 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 23.
I20260709 12:20:12.034623 28289 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:20:12.012779  3260 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.042636 28954 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:12.043576 28441 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
W20260709 12:20:11.985612  3269 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:20:11.981894  3191 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:12.112000 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 69473.4us
I20260709 12:20:12.030577  3174 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:12.122121 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:12.122936 28441 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
W20260709 12:20:12.050797  3275 log.cc:927] Time spent T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.090s	user 0.000s	sys 0.002s
I20260709 12:20:12.156404 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.172111 28359 raft_consensus.cc:3060] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:11.989637  3188 raft_consensus.cc:697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:11.961997  3033 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.059530  3186 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.162984 28440 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 24.
I20260709 12:20:12.171036 28289 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:12.042274  3193 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:11.991981  3233 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:12.190383 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.112107  3191 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.182250 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.123405  3174 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:12.042420  3165 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:12.203857 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 277736us
I20260709 12:20:12.178475  3226 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:12.233497 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 332199us
I20260709 12:20:12.235795 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:12.239800 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:12.191836  3193 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:12.202921  3191 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:12.243546  3126 consensus_meta.cc:287] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 12:20:12.239300 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:12.245223 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:12.239406 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:12.252211 28441 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:12.225638  3186 raft_consensus.cc:2804] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:12.255470 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.264463 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.265237 28361 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:20:12.247156  3193 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:12.262341  3105 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.930s	user 0.012s	sys 0.000s
I20260709 12:20:12.255549  3278 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:12.279924 28330 leader_election.cc:400] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19.
I20260709 12:20:12.280337 28330 leader_election.cc:403] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:20:12.285852 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.286243  3177 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.287282  3233 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:12.287017  3105 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.289506  3105 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.270262  3163 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:12.292783  3163 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.294977  3292 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:12.296769  3110 consensus_meta.cc:287] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.547s	user 0.006s	sys 0.003s
I20260709 12:20:12.297093  3292 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:20:12.298110  3188 consensus_queue.cc:237] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.298738  3165 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:12.300870 28285 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 22: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:12.303468 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:12.304132 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:12.304886 28985 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:12.264856  3101 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.307889 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:20:12.308347  3276 log.cc:927] Time spent T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:12.309621  3291 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:12.309791 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:12.310288  3291 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.311940  3282 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.312465  3252 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.314352  3105 consensus_queue.cc:1059] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:12.312506  3282 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:12.310940  3236 consensus_meta.cc:287] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.002s
I20260709 12:20:12.315333  3175 raft_consensus.cc:3055] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:12.315742  3175 raft_consensus.cc:740] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:12.316385  3201 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.316797  3236 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.315706 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:12.318219 28359 raft_consensus.cc:2468] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:12.318846 28330 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:12.275895  3251 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:12.272753  3178 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:12.261077  3126 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.336393 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.337198 28361 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:12.295045  3163 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.338590 28255 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:12.316663  3282 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:12.323323  3252 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.345273 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:12.323405  3260 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113) with higher term. Message: Invalid argument: T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19.
I20260709 12:20:12.346776 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.379933 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:12.380091 28359 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:20:12.380173 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 165091us
I20260709 12:20:12.380870 28359 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:12.357766 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:12.382151 28432 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:12.365442 28330 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:12.388754 28404 leader_election.cc:400] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 23. Current term is 24.
I20260709 12:20:12.389335 28404 leader_election.cc:403] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 12:20:12.391404 28292 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.005s
I20260709 12:20:12.392932 28292 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:12.346073  3272 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:12.390537 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:12.395406 28359 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.317875  3278 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.399472 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.408350 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:12.412621 28365 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:12.349308  3251 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 27 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:12.408841 28359 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:12.409927 28255 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:12.411129  3155 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:12.346436  3281 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:12.416675 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:12.416996 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:12.385710  3282 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:12.392347  3201 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 23. Current term is 24.
I20260709 12:20:12.426170 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.340500  3178 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.366495  3246 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.448150 28984 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:12.449661 28365 raft_consensus.cc:2468] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:20:12.399783  3124 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:12.455693 28438 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:12.342381  3198 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.467337 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.416293  3155 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.473371 28365 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 17: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:12.430472 28255 leader_election.cc:400] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 22. Current term is 23.
I20260709 12:20:12.474653 28255 leader_election.cc:403] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:20:12.406268  3272 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:12.475800 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 67525us
I20260709 12:20:12.477509 28404 leader_election.cc:400] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:20:12.471036 28359 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:20:12.477991 28404 leader_election.cc:403] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:20:12.477900 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.430670  3033 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:12.381261  3175 consensus_queue.cc:260] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.447857  3233 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:20:12.481797 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 53969.3us
I20260709 12:20:12.482503 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.459453  3124 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:12.417339  3163 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:20:12.460546  3186 raft_consensus.cc:697] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:12.493103 28432 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:12.494889 28989 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 22: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:12.472319  3181 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:12.466831  3101 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:12.480540  3272 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.476686  3155 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:12.406962  3075 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 12:20:12.485792  3105 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 22. Current term is 23.
I20260709 12:20:12.497612  3124 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:12.545707  3201 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:12.557191 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.560393 28255 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:12.497709  3295 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:12.561079 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:12.566736 28255 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:12.569306 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:12.559469 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 82118.5us
I20260709 12:20:12.511142  3302 raft_consensus.cc:3060] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.570544 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:12.546814  3239 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:12.548520  3175 raft_consensus.cc:3060] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:12.551136  3126 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:12.580438  3290 log.cc:927] Time spent T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 12:20:12.581559  3186 consensus_queue.cc:237] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.569044 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:12.586711 28438 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:12.576853 28404 leader_election.cc:400] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:20:12.576115  3306 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:20:12.511152  3282 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:20:12.509301  3101 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.574590 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:12.588411 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:12.508001  3033 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:20:12.568414  3177 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:12.595777 28985 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:20:12.561931  3236 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.566833  3287 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:12.571552  3158 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:20:12.597223 28361 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:12.598486  3105 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:12.599182  3299 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
W20260709 12:20:12.600281  3288 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:20:12.602278  3101 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:12.603426  3289 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:20:12.605046  3198 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:12.576896  3295 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:12.612164 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 13 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.575781  3233 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.578076  3239 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:12.622593 28438 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:20:12.620245 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:12.626171 28332 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:12.630563  3178 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:12.660514  3158 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:12.660660  3239 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.661633  3158 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 11.13, Last appended by leader: 13, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.571982  3201 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.663879 28440 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:12.619270  3287 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.659502  3304 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:12.678203  3287 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:12.680191  3195 consensus_meta.cc:287] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.002s
I20260709 12:20:12.680361  3096 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.680776  3195 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.635654  3291 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:12.636375  3293 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.187s	user 0.001s	sys 0.001s
I20260709 12:20:12.642642  3278 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:12.546098  3251 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.652787 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:12.692679 28436 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
W20260709 12:20:12.653646 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65342us
I20260709 12:20:12.691740 28284 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 26. Current term is 27. Ops: [26.20-26.20]
I20260709 12:20:12.695636 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:12.656412 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.658427 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.658360  3295 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.659472 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 3740) took 1040 ms. Trace:
I20260709 12:20:12.659610 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.659771 28332 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:12.662083  3239 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:12.634698 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:12.666127  3246 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:12.682060 28432 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:20:12.585386  3174 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.684922 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:20:12.686136 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.687152 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:20:12.687664 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 118445us
I20260709 12:20:12.702769 28367 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:12.703428 28361 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
W20260709 12:20:12.704118 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 88343.1us
I20260709 12:20:12.665722  3201 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.699350 28956 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:12.701519 28404 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:12.704473 28364 rpcz_store.cc:276] 0709 12:20:11.619166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.619337 (+   171us) service_pool.cc:224] Handling call
0709 12:20:11.622027 (+  2690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:12.014845 (+392818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.659192 (+644347us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b540027e554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:12.659382 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":803260672}
W20260709 12:20:12.708168 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48510.2us
I20260709 12:20:12.708645 28441 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 22: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:12.722112 28404 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:12.685755  3304 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:12.723259  3178 raft_consensus.cc:2804] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:12.723757 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:12.723625  3304 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.723869  3178 raft_consensus.cc:697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:12.726471  3178 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.724177  3126 raft_consensus.cc:2804] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:12.728524  3246 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.647259  3191 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.734334 28189 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 198) took 1932 ms. Trace:
I20260709 12:20:12.731541 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:12.685813  3282 consensus_queue.cc:1059] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:20:12.708768  3245 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:12.712682  3096 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:12.739688 28255 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:12.735165 28985 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:12.741078 28330 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:12.744803 28332 leader_election.cc:400] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19.
I20260709 12:20:12.742245 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.746253 28362 raft_consensus.cc:2368] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
I20260709 12:20:12.741160 28954 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:12.748126 28987 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:12.748273 28441 raft_consensus.cc:1240] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:12.749066 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:12.688760  3294 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:20:12.730798  3302 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:20:12.751718 28984 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:12.754402 28438 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:12.738803  3245 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.770079 28365 raft_consensus.cc:2393] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 19: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:12.753098  3284 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:12.762252  3231 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:12.781212 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.774107  3124 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.783991  3308 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:12.784619  3308 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.784986  3308 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:12.785480  3124 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:12.787434  3231 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:12.788587  3231 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.789904  3310 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.017s
W20260709 12:20:12.791385  3301 log.cc:927] Time spent T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:20:12.791877  3284 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 23 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:12.792946  3284 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.797058  3174 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:12.793359  3191 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:12.797876  3174 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:12.798627  3231 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:12.799015 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:20:12.806442 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 55941.8us
I20260709 12:20:12.807662 28332 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:12.773115  3245 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.773008  3294 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:12.812861 28985 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:20:12.734836 28189 rpcz_store.cc:276] 0709 12:20:10.802144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.838912 (+ 36768us) service_pool.cc:224] Handling call
0709 12:20:12.734254 (+1895342us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:11.739104 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:11.780459 (+ 41355us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:11.780500 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:20:11.780517 (+    17us) tablet.cc:662] Decoding operations
0709 12:20:11.780880 (+   363us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:11.780948 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:11.780982 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:11.781172 (+   190us) tablet.cc:701] Row locks acquired
0709 12:20:11.781191 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:20:11.781358 (+   167us) write_op.cc:270] Start()
0709 12:20:11.781490 (+   132us) write_op.cc:276] Timestamp: P: 1783599611781300 usec, L: 0
0709 12:20:11.781514 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:20:11.782341 (+   827us) log.cc:844] Serialized 533 byte log entry
0709 12:20:12.264467 (+482126us) op_driver.cc:464] REPLICATION: finished
0709 12:20:12.572241 (+307774us) write_op.cc:301] APPLY: starting
0709 12:20:12.577394 (+  5153us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:12.661319 (+ 83925us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:12.661426 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:12.662304 (+   878us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:12.662524 (+   220us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:20:12.662574 (+    50us) write_op.cc:312] APPLY: finished
0709 12:20:12.662788 (+   214us) log.cc:844] Serialized 25 byte log entry
0709 12:20:12.662970 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:12.663059 (+    89us) write_op.cc:454] Released schema lock
0709 12:20:12.663099 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:12.664116 (+  1017us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:12.664145 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":180071,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":278736,"apply.run_cpu_time_us":4084,"apply.run_wall_time_us":104956,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8305,"num_ops":1,"prepare.queue_time_us":158383,"prepare.run_cpu_time_us":2980,"prepare.run_wall_time_us":61627,"replication_time_us":463315,"thread_start_us":260980,"threads_started":2}]]}
I20260709 12:20:12.812538 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:12.822297 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:12.823510 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:12.819820 28332 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:12.831223 28255 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:12.831632 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.834038 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:20:12.830670 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 106963us
I20260709 12:20:12.836802 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:12.777140  3233 consensus_queue.cc:1059] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:20:12.825098  3169 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:12.851331 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.865787 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:12.854444 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:12.881666 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:12.839325 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:12.856709 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 3652) took 1244 ms. Trace:
I20260709 12:20:12.838184 28982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 1164) took 1144 ms. Trace:
I20260709 12:20:12.889475 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:12.813953  3226 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:12.904599 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55954 (request call id 3749) took 1145 ms. Trace:
I20260709 12:20:12.878432 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:12.917150 28290 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:20:12.896255  3163 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:12.927965 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:12.926259 28365 rpcz_store.cc:276] 0709 12:20:11.759290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.477442 (+718152us) service_pool.cc:224] Handling call
0709 12:20:12.904325 (+426883us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54003d2f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:12.904495 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":668096896}
I20260709 12:20:12.901474 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.892954 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:20:12.896044 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 56825.2us
I20260709 12:20:12.818150  3294 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:12.861294 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:12.889115 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:12.927212 28990 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:20:12.933943 28283 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
W20260709 12:20:12.907218  3300 log.cc:927] Time spent T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.317s	user 0.004s	sys 0.000s
I20260709 12:20:12.833864  3252 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:12.898101 28982 rpcz_store.cc:276] 0709 12:20:11.690147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.789355 (+ 99208us) service_pool.cc:224] Handling call
0709 12:20:11.789820 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:11.794477 (+  4657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:11.794511 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:11.825996 (+ 31485us) log.cc:844] Serialized 26 byte log entry
0709 12:20:12.175100 (+349104us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:12.218777 (+ 43677us) log.cc:844] Serialized 14 byte log entry
0709 12:20:12.757996 (+539219us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b1800182ce0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb797455 00007f85be87f060 00007f85be891716 00007f85be8a2287 00007f85be8a2208
0709 12:20:12.824980 (+ 66984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.825104 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:12.825158 (+    54us) raft_consensus.cc:1685] finished
0709 12:20:12.834360 (+  9202us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:12.834530 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":42952,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":2536,"raft.queue_time_us":89614,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":21351,"spinlock_wait_cycles":101337344,"thread_start_us":343814,"threads_started":2,"wal-append.queue_time_us":414749}
I20260709 12:20:12.939095 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:12.936297 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:12.937004 28287 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:20:12.935729 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:12.940634 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:12.908718  3286 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:12.941275 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:12.941805 28404 leader_election.cc:400] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
I20260709 12:20:12.894565  3194 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:12.945410 28404 leader_election.cc:403] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:20:12.858083  3105 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.858537  3281 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:12.951400 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:12.942274 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:12.910763  3195 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:12.952464 28287 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 15: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:12.953135 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 3747) took 1194 ms. Trace:
I20260709 12:20:12.953483 28365 rpcz_store.cc:276] 0709 12:20:11.758703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.952389 (+1193686us) service_pool.cc:224] Handling call
0709 12:20:12.952712 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:12.952891 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:12.952908 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:12.952938 (+    30us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:12.952967 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.953003 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:12.953086 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:12.944196 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:12.898656  3054 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:12.858357  3177 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:12.948448  3302 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 18. Current term is 19.
I20260709 12:20:12.892283  3169 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:12.950313 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:12.950800 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 3746) took 1192 ms. Trace:
I20260709 12:20:12.897656 28363 rpcz_store.cc:276] 0709 12:20:11.612316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.612468 (+   152us) service_pool.cc:224] Handling call
0709 12:20:11.632951 (+ 20483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.715138 (+ 82187us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:20:12.151713 (+436575us) log.cc:844] Serialized 14 byte log entry
0709 12:20:12.303709 (+151996us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:12.303957 (+   248us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:12.304127 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.856350 (+552223us) spinlock_profiling.cc:243] Waited 81.9 ms on lock 0x7b5400127354. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:12.856412 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:12.856591 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22502,"prepare.queue_time_us":561754,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":1820,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"spinlock_wait_cycles":188311936,"thread_start_us":576185,"threads_started":2,"wal-append.queue_time_us":243577}
I20260709 12:20:12.935774  3294 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.960201 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:12.966701 28365 raft_consensus.cc:2393] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 18: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:12.966863 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:12.959246 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:12.967630 28291 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:12.967906 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.961664 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:12.961899 28362 rpcz_store.cc:276] 0709 12:20:11.758367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.938148 (+1179781us) service_pool.cc:224] Handling call
0709 12:20:12.950252 (+ 12104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:12.950456 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:12.950493 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:12.950536 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:12.950566 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.950625 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:12.950741 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:20:12.964994 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 68097.2us
I20260709 12:20:12.910552  3155 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.944964  3181 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:12.971303 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:12.970695 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:12.939654  3126 raft_consensus.cc:697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:12.955344  3178 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.974350 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:12.974534  3316 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:20:12.988158 28362 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:12.989126 28362 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 1361) took 1170 ms. Trace:
I20260709 12:20:12.989341 28362 rpcz_store.cc:276] 0709 12:20:11.818408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.969192 (+1150784us) service_pool.cc:224] Handling call
0709 12:20:12.986642 (+ 17450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:12.988858 (+  2216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.989068 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:12.975395  3169 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:12.942408  3252 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:12.950466  3299 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:12.973439 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:12.984047 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:12.886829  3272 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:12.970779 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:12.987124 28290 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:12.955469  3054 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:12.975270  3313 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:12.958645  3286 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:12.962832  3194 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:13.000896 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:12.980659  3246 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:13.005441 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.006407  3054 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:13.015384  3313 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:13.016440  3313 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.008648 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:13.003366  3299 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.019606 28990 raft_consensus.cc:1240] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:13.020792  3195 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:20:12.992893  3252 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:13.023404  3194 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.023973  3155 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:13.026554  3193 consensus_meta.cc:287] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.725s	user 0.011s	sys 0.002s
I20260709 12:20:13.027102  3193 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.027992  3272 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:13.028563 28988 raft_consensus.cc:2393] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 19: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:13.029292  3193 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:13.029903  3286 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.031214  3286 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:12.958990  3105 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.013131 28362 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:13.035849 28365 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:13.038877 28283 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 12:20:13.061491 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 56138us
I20260709 12:20:13.063539 28404 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:20:13.064254 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:13.070477 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 119163us
I20260709 12:20:13.007071  3261 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:13.073092 28363 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 27 LEADER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:13.009825  3295 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:13.014370  3177 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:13.084436 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:12.993036  3178 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.069365  3155 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:13.065109  3201 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:12.966578  3281 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.077857  3261 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:13.090382  3308 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.095250  3309 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:13.095815  3309 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:13.096223  3309 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:13.097126  3308 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.101900 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:13.103277 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:13.104029 28984 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 21: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:13.105142 28984 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:20:13.104837  3177 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.107304  3178 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:13.117719 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:13.126825 28364 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 22: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:13.119648 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:13.122709 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:13.116343 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59792 (request call id 3535) took 1316 ms. Trace:
I20260709 12:20:13.129492 28285 rpcz_store.cc:276] 0709 12:20:11.800011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.819524 (+ 19513us) service_pool.cc:224] Handling call
0709 12:20:13.116106 (+1296582us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b54003da4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:13.116252 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1104312064}
W20260709 12:20:13.115617  3170 consensus_meta.cc:287] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.993s	user 0.011s	sys 0.003s
I20260709 12:20:13.133373 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:13.135344 28358 raft_consensus.cc:1240] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:13.142839 28283 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:13.143570 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:20:13.121138  3307 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.515s	user 0.001s	sys 0.001s
I20260709 12:20:13.154800 28359 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:13.129478  3195 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:13.184827 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:13.116755  3261 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:13.205410  3075 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:13.205926  3299 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.176874  3245 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.133806  3126 consensus_queue.cc:237] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.157039 28358 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:13.155699  3170 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.211347  3105 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:13.212011 28990 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:13.213681 28330 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.214064  3281 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.215823 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:13.210137  3245 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:13.186394  3317 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:20:13.219952 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.156198  3252 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.172092 28359 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:13.223220 28364 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:13.215746  3170 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:13.227913 28440 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:13.232722 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:13.246670 28437 raft_consensus.cc:2393] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 21: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:20:13.233069  3239 consensus_meta.cc:287] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.002s
I20260709 12:20:13.250020  3318 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 12:20:13.251319  3252 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.204676  3195 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.253644  3195 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.246762 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.155864  3054 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.187131  3242 raft_consensus.cc:3055] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:13.238231 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:20:13.240584 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 94850.1us
I20260709 12:20:13.241045 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:13.242673 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.321394 28439 raft_consensus.cc:2393] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 16: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:13.243343 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:13.228673 28290 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:13.241389 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:13.332683 28433 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 22: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:20:13.225333  3188 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.013s	sys 0.008s
W20260709 12:20:13.266930  3306 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.008s	sys 0.015s
I20260709 12:20:13.341133 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:13.325798 28330 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:13.259892  3054 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:13.267149  3242 raft_consensus.cc:740] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:20:13.340025  3188 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.013s	sys 0.009s
I20260709 12:20:13.353713 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:13.348062 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:13.349485 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:13.355208 28364 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:13.340948  3306 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.009s	sys 0.015s
I20260709 12:20:13.345897 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:13.325724  3239 consensus_queue.cc:1059] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 12:20:13.352628 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:13.354686 28291 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:13.361236 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:13.363389 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:13.363967 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:13.364688 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:13.365497 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.366336 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:13.357476 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:13.365439 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.367722 28289 raft_consensus.cc:3060] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:13.326045  3291 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:13.368927 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:13.356599  3155 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:13.371944  3155 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.374106  3155 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:13.374464  3239 raft_consensus.cc:3055] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:13.374872  3239 raft_consensus.cc:740] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:13.375697  3239 consensus_queue.cc:260] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.376948  3239 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:13.379102  3322 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:13.379890  3322 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:13.381335  3246 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:13.381799  3246 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:13.382189  3246 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:13.353899  3286 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:13.383623  3286 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.353667  3260 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:13.389793  3310 consensus_queue.cc:1059] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:20:13.392488 28983 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:13.394737  3330 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.395354  3330 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.336858  3294 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.373997 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:13.398880 28984 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:13.390290 28987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1378) took 1088 ms. Trace:
I20260709 12:20:13.400326 28987 rpcz_store.cc:276] 0709 12:20:12.302253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.302410 (+   157us) service_pool.cc:224] Handling call
0709 12:20:13.390031 (+1087621us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400762d54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:13.390192 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1004157184}
I20260709 12:20:13.409338 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:13.397606  3330 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:13.382280 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 9cdd1e39ef1841c382e814af04ea5bc9 with delete_type TABLET_DATA_TOMBSTONED (TS a3de34b6f4cf4606a61944db6e2739c8 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:36572
I20260709 12:20:13.411365 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:13.411721 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 46263.4us
I20260709 12:20:13.416757 28288 raft_consensus.cc:2468] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:20:13.408072  3286 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.418227  3226 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:13.423329 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.424503 28289 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:20:13.405246  3295 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:13.424590 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:13.407081  3306 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:13.429864 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.435748 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:13.440230  3086 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:13.444520  3306 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.445107  3252 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:13.446871  3260 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:13.426538  3226 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:13.448529  3294 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.449528 28987 raft_consensus.cc:1275] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:20:13.424010  3330 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.456347 28991 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:13.458784 28364 raft_consensus.cc:2468] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:20:13.463028 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.494135 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:13.407568  3291 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.463673  3126 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:20:13.468969  3305 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.867s	user 0.002s	sys 0.000s
W20260709 12:20:13.471319  3280 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.159s	user 0.000s	sys 0.004s
I20260709 12:20:13.477013 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.480695 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:13.506036 28433 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:13.481513 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.508226 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.509383 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.447060  3306 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.489672 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.489672 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:13.491616 28440 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:13.492849 28360 raft_consensus.cc:1275] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:13.423306  3261 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.433624  3313 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:13.499485 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.519163 28954 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:13.502230 28330 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.500340 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:13.504199 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.520954 28441 raft_consensus.cc:2393] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 21: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:13.482712 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.521529 28434 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:13.511252 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:13.528781 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:13.528927 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:13.529826 28367 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:13.517776 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.528475  3231 consensus_queue.cc:1059] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:20:13.530691  3054 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.535888 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:13.536240  3054 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.528666  3194 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:13.456574  3330 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.540005  3330 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.540524  3306 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:13.540850  3195 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:13.528956  3262 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:20:13.543279 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 1168) took 1149 ms. Trace:
I20260709 12:20:13.500820  3126 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:13.544178  3308 raft_consensus.cc:3055] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:13.544648  3308 raft_consensus.cc:740] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:13.538172  3324 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:13.542881  3163 raft_consensus.cc:3055] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:13.545207  3054 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.546272  3163 raft_consensus.cc:740] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:13.517590  3295 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:13.547343  3163 consensus_queue.cc:260] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.548378  3163 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:13.528653  3175 consensus_queue.cc:1059] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:20:13.549482  3175 raft_consensus.cc:3055] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:13.549873  3175 raft_consensus.cc:740] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:13.550740  3175 consensus_queue.cc:260] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.551344 28366 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:20:13.552064  3175 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:13.545635  3308 consensus_queue.cc:260] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.556988 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 1363) took 1160 ms. Trace:
I20260709 12:20:13.559711  3308 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:13.557449 28363 rpcz_store.cc:276] 0709 12:20:12.396748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.004569 (+607821us) service_pool.cc:224] Handling call
0709 12:20:13.030864 (+ 26295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:13.081800 (+ 50936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:13.556696 (+474896us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b5400126454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:13.556916 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96620800}
I20260709 12:20:13.531980 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.543713 28986 rpcz_store.cc:276] 0709 12:20:12.393938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.394060 (+   122us) service_pool.cc:224] Handling call
0709 12:20:12.412225 (+ 18165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:12.423080 (+ 10855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:12.423115 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:12.430341 (+  7226us) log.cc:844] Serialized 26 byte log entry
0709 12:20:12.502950 (+ 72609us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:12.541368 (+ 38418us) log.cc:844] Serialized 14 byte log entry
0709 12:20:12.541600 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:12.541703 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:13.468252 (+926549us) spinlock_profiling.cc:243] Waited 9.09 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:20:13.542946 (+ 74694us) raft_consensus.cc:1685] finished
0709 12:20:13.543065 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:13.543207 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7253,"prepare.queue_time_us":42591,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":122642,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":17194,"spinlock_wait_cycles":20996864,"thread_start_us":17206,"threads_started":1,"wal-append.queue_time_us":105307}
I20260709 12:20:13.457234  3309 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.584168 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:13.584604 28983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 1167) took 1191 ms. Trace:
I20260709 12:20:13.592588 28433 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:13.516671 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:13.594071 28364 raft_consensus.cc:2393] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 19: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:13.594681 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:13.586465 28983 rpcz_store.cc:276] 0709 12:20:12.393557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.393720 (+   163us) service_pool.cc:224] Handling call
0709 12:20:13.392260 (+998540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:13.392952 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:13.584379 (+191427us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b5400762d50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:13.584526 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2254808832}
I20260709 12:20:13.544521  3126 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:13.597426 28363 raft_consensus.cc:1240] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:13.455127  3242 consensus_queue.cc:260] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.598472  3298 raft_consensus.cc:3055] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:13.598698 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:13.600070 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:13.625721 28287 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 23.
I20260709 12:20:13.601271 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:20:13.586635 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54755.2us
I20260709 12:20:13.591828 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:13.631170 28366 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:20:13.632488 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:13.643106 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.647727 28954 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:13.645253 28255 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:13.650202 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:13.653245 28255 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:13.659396 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:13.660004 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.667586 28956 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:13.671083 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:13.683341 28255 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:13.695744 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:13.700533 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.701269 28434 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 24.
I20260709 12:20:13.641873  3298 raft_consensus.cc:740] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:20:13.650408  3172 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:20:13.644263  3239 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.710937 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:20:13.712392 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 44931.4us
I20260709 12:20:13.648780  3313 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:13.650758  3233 raft_consensus.cc:2764] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:20:13.650242  3295 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:13.692003  3324 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:13.692350  3330 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:13.692620  3326 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:13.726589 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:13.732133 28983 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 11 index: 13. Preceding OpId from leader: term: 12 index: 14. (index mismatch)
I20260709 12:20:13.736999 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:13.675625  3310 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:20:13.741976 28435 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:13.721690  3294 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:13.720635  3318 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:13.725829  3175 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:13.725270  3224 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:13.730448  3177 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:13.764353  3261 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.765106  3246 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.766438 28436 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 21: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:13.769440  3246 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:13.770537  3291 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:13.733239  3239 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.753378 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:13.775846  3239 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:13.757745 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.743903  3183 consensus_queue.cc:1048] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:20:13.757548  3318 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.759373  3309 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:13.736660  3226 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:20:13.693186 28366 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55946 (request call id 3654) took 1960 ms (client timeout 1733 ms). Trace:
W20260709 12:20:13.787210 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 112082us
I20260709 12:20:13.798625 28404 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:13.800351  3295 raft_consensus.cc:2804] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:13.800935  3295 raft_consensus.cc:697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:13.801952  3295 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:13.806211 28361 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:13.810160 28983 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:20:13.813567 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:13.819176 28361 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:13.823048 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
W20260709 12:20:13.829532 28404 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.123.2:44701 timed out after 1.733s (SENT) [suppressed 4 similar messages]
W20260709 12:20:13.796566 28366 rpcz_store.cc:269] 0709 12:20:11.675353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:11.687851 (+ 12498us) service_pool.cc:224] Handling call
0709 12:20:12.007088 (+319237us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b4c0047f800. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bcb53a20 00007f85bcb64a29 00007f85bcb2a644 00007f85bcb51f2d 00007f85bcb51b93 00007f85be88218b 00007f85be882cb5 00007f85be87df30 00007f85be87f7de 00007f85be88b27c 00007f85c384cb17
0709 12:20:12.795388 (+788300us) spinlock_profiling.cc:243] Waited 8.5 ms on lock 0x7b18000cea60. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8c2646 00007f85be7fc58e 00007f85be87df9e 00007f85be87f7de 00007f85be88b27c 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c
0709 12:20:13.190893 (+395505us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b58004ff1d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7e30aa 00007f85be865447 00007f85be87dfe7 00007f85be87f7de 00007f85be88b27c 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c
0709 12:20:13.551127 (+360234us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b580073f7d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7e30aa 00007f85be865447 00007f85be87dfe7 00007f85be87f7de 00007f85be88b27c 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c
0709 12:20:13.635832 (+ 84705us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103238144}
W20260709 12:20:13.845252 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 122354us
I20260709 12:20:13.846290 28991 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 27 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 20. (term mismatch)
I20260709 12:20:13.846724 28991 pending_rounds.cc:85] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 19
W20260709 12:20:13.876834 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 205822us
I20260709 12:20:13.793907  3292 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:13.795444  3291 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:13.783404  3324 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:13.846949 28991 pending_rounds.cc:107] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 26.20
I20260709 12:20:13.891259 28257 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:13.895486 28991 raft_consensus.cc:2889] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:13.901213 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:13.841755  3198 consensus_queue.cc:1048] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
W20260709 12:20:13.915053  3335 log.cc:927] Time spent T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:20:13.955124  3286 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.017463 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:14.015122 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 185605us
I20260709 12:20:13.782995  3298 consensus_queue.cc:260] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:13.900313  3309 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:13.916937  3239 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 12:20:14.022888 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 120554us
I20260709 12:20:14.018364  3226 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.058537  3226 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.059725 28985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1390) took 1019 ms. Trace:
I20260709 12:20:14.060158 28985 rpcz_store.cc:276] 0709 12:20:13.039973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.566721 (+526748us) service_pool.cc:224] Handling call
0709 12:20:13.567175 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:13.567643 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:13.567674 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:13.568275 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:20:13.784031 (+215756us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:13.784077 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:13.786165 (+  2088us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:14.059484 (+273319us) raft_consensus.cc:1685] finished
0709 12:20:14.059507 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:14.059656 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7856,"raft.queue_time_us":4098,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":9811,"thread_start_us":215294,"threads_started":1,"wal-append.queue_time_us":392674}
I20260709 12:20:14.063138 28954 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:14.064241  3226 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.066038  3286 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.067515  3306 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:14.075783 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:14.077039 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59723us
W20260709 12:20:14.071655  3340 log.cc:927] Time spent T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.177s	user 0.004s	sys 0.000s
I20260709 12:20:14.032749  3033 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:14.083776 28174 catalog_manager.cc:5028] TS a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): tablet 9cdd1e39ef1841c382e814af04ea5bc9 (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
W20260709 12:20:14.083627 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 355820us
I20260709 12:20:14.086781 28956 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:14.089145 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:14.069077  3330 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.081094  3101 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:14.060441  3126 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.080502  3188 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:14.080803  3309 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:14.115200  3324 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:14.087802  3033 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:14.090726  3306 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:20:14.123665 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:14.105997  3163 consensus_meta.cc:287] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.557s	user 0.014s	sys 0.001s
W20260709 12:20:14.127521 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64556.3us
I20260709 12:20:14.129587 28366 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:20:14.132460 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.134452 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:14.118952  3188 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:14.138713 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.140499  3188 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.145534  3163 consensus_queue.cc:1059] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:14.148896  3326 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.149937  3231 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.150560  3231 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.153357  3105 raft_consensus.cc:3055] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:14.153857  3105 raft_consensus.cc:740] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:14.154781 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:14.155158 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:14.156072 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 3748) took 2397 ms. Trace:
I20260709 12:20:14.156512 28365 rpcz_store.cc:276] 0709 12:20:11.759005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.968834 (+1209829us) service_pool.cc:224] Handling call
0709 12:20:12.969121 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:13.059999 (+ 90878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:14.155807 (+1095808us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b5400054954. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:14.156008 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152978304}
I20260709 12:20:14.156960 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:14.154891  3105 consensus_queue.cc:260] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.158560  3105 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.155860  3308 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:14.159979 28291 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:20:14.158589 28290 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:14.161763  3175 consensus_queue.cc:1059] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:14.162873  3175 raft_consensus.cc:3055] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:14.163249  3175 raft_consensus.cc:740] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:14.119166  3033 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:14.157096 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.167755 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:14.168049 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:14.169593 28984 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 27.
I20260709 12:20:14.171595  3175 consensus_queue.cc:260] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.173010  3175 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:14.172098  3126 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.178474 28285 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:14.153790  3298 raft_consensus.cc:3060] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:14.119185  3239 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:20:14.114173  3330 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.184640 28991 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:20:14.129688  3324 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.149473  3326 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.178273  3348 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.178414  3350 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:14.181489  3177 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.198371 28439 raft_consensus.cc:1240] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:20:14.199716 28286 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:14.199847 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.200862 28439 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 22: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:14.204481 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:14.197229  3350 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:14.201345  3177 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.174960  3309 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.216468 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:14.217259 28287 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
I20260709 12:20:14.219630 28287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:14.221165 28287 raft_consensus.cc:2410] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 15 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:20:14.222645 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:14.194188  3347 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:20:14.212409  3286 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.197221  3348 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.215667  3177 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.231083 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:14.223336  3350 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:14.234639  3365 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:14.260977  3365 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:14.261420  3365 raft_consensus.cc:3060] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:14.254112  3330 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.255232 28989 raft_consensus.cc:1240] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:14.265426 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 1181) took 1442 ms. Trace:
I20260709 12:20:14.265842 28989 rpcz_store.cc:276] 0709 12:20:12.823573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.338905 (+515332us) service_pool.cc:224] Handling call
0709 12:20:14.255010 (+916105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:14.264003 (+  8993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:14.265213 (+  1210us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b540048fb50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:14.265355 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2105820928}
I20260709 12:20:14.267175  3363 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:14.267861  3363 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.270620  3326 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.272435  3348 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:14.254601 28988 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46950 (request call id 1381) took 1858 ms (client timeout 2115 ms). Trace:
W20260709 12:20:14.273288 28988 rpcz_store.cc:269] 0709 12:20:12.396118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.396286 (+   168us) service_pool.cc:224] Handling call
0709 12:20:14.254358 (+1858072us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b540048fb54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:14.254523 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390067072}
I20260709 12:20:14.258244 28990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46960 (request call id 1177) took 1528 ms. Trace:
I20260709 12:20:14.274319 28990 rpcz_store.cc:276] 0709 12:20:12.729802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.043570 (+313768us) service_pool.cc:224] Handling call
0709 12:20:13.081338 (+ 37768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:13.212396 (+131058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:14.257989 (+1045593us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400870fd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:14.258178 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300085504}
I20260709 12:20:14.277493  3363 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.251312  3367 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:14.282591  3367 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.282989  3367 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:14.286788 28985 raft_consensus.cc:2410] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:20:14.287274  3324 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.290992 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:20:14.273301 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.293484 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.294478 28360 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:14.296139 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.296779 28361 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:20:14.299800  3180 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:14.312683 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.313441 28360 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
W20260709 12:20:14.314059 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 69484.1us
I20260709 12:20:14.315878 28404 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:14.318110 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.318576 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:14.324641  3350 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.311162  3245 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:14.325544  3367 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.300689  3361 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:14.332852  3245 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.332345  3348 consensus_queue.cc:1059] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 12:20:14.335147  3245 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:14.335748  3245 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:14.336912  3364 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.328231  3101 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:14.331292 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:14.326851  3324 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.345772  3350 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.333611  3361 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.346685 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:14.337421  3347 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.348304 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:14.328555  3365 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.339867 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:14.332860  3326 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:14.359637 28290 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:14.346930 28291 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:14.316725  3175 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:20:14.373088 28433 raft_consensus.cc:1240] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:14.384231 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44467.5us
I20260709 12:20:14.387279 28435 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 11. Current term is 12. Ops: []
I20260709 12:20:14.350134  3368 raft_consensus.cc:2804] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:14.390357 28360 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:20:14.359400  3309 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:14.386121 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:14.342773  3364 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.380336  3326 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:14.391809 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 60623.7us
I20260709 12:20:14.396136 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 12 candidate_status { last_received { term: 11 index: 13 } } ignore_live_leader: true dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:14.397210 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:14.400444 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.403035  3364 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:14.405040  3361 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.406708  3367 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 28 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.396909 28366 raft_consensus.cc:2376] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12. Re-sending same reply.
I20260709 12:20:14.384994  3245 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:14.408182  3326 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:14.416929 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:14.414631  3033 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.421096  3365 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.421911  3033 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.421929 28287 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:14.423727 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.394806  3309 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.425451  3309 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:14.399246  3350 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.432669  3326 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.435616 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:14.417094 28432 raft_consensus.cc:1240] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:14.438937 28360 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:14.432883  3350 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.408447  3368 raft_consensus.cc:697] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:14.443624 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:14.446847 28988 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:14.445842 28332 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:14.449365 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:14.364589  3347 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.450610 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:14.453536  3124 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:14.454331  3124 consensus_queue.cc:1059] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:14.454453  3368 consensus_queue.cc:237] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.455700  3161 raft_consensus.cc:3055] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:14.456090  3161 raft_consensus.cc:740] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:14.456888  3161 consensus_queue.cc:260] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.460541 28283 raft_consensus.cc:2468] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
W20260709 12:20:14.462387 28954 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.463503 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:14.468026  3278 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:14.468698  3278 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.469120  3242 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:14.469514  3347 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.470357  3252 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:20:14.470976  3278 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.474423 28289 raft_consensus.cc:2410] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:20:14.442775  3180 consensus_queue.cc:1059] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:20:14.476101  3180 raft_consensus.cc:3055] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:14.476460  3180 raft_consensus.cc:740] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:14.463201 28954 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:14.478785  3294 raft_consensus.cc:2804] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:14.479271  3294 raft_consensus.cc:697] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 22 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:14.480384  3294 consensus_queue.cc:237] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.477423  3180 consensus_queue.cc:260] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.483259 28366 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:14.483014  3198 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:20:14.492655  3332 log.cc:927] Time spent T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.082s	user 0.006s	sys 0.006s
I20260709 12:20:14.484265  3180 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:14.492508  3198 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:14.495818  3344 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.601s	user 0.006s	sys 0.002s
I20260709 12:20:14.496306  3350 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.500108 28366 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:14.500694 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:14.501983 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:14.503001 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:14.503719 28365 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:14.500479 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.501389 28983 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:20:14.507324 28255 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:14.508245 28988 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 28.
I20260709 12:20:14.509385 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:14.510353 28434 raft_consensus.cc:2393] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 22: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:14.510603 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:14.511041 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.511623 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:14.517426 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.502683 28984 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 14.
I20260709 12:20:14.497262  3318 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.481653  3231 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.532042 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:14.533397 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.530905 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:14.534353 28435 raft_consensus.cc:2410] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:20:14.511965 28437 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:20:14.538458 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:14.538825 28292 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:14.540166 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.543453 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:14.548089 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 85725.8us
I20260709 12:20:14.550644 28365 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 15.
I20260709 12:20:14.551465 28441 raft_consensus.cc:1240] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:14.554448 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 103461us
I20260709 12:20:14.558032 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.550912 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fa1ea35643e43ea8ab0f71a59be9e29" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.549365 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:14.561127 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:14.550209  3298 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:14.542559  3371 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:14.538650  3105 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:14.566398 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.563625 28255 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:14.546954  3282 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:14.552405  3326 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.573197 28366 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:20:14.562553  3362 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:14.560948  3198 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.546411  3363 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:20:14.566443  3278 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:14.555891  3231 consensus_queue.cc:1059] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:20:14.543010  3374 consensus_queue.cc:1059] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:20:14.557672  3361 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:20:14.571899  3105 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.575910  3359 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:14.575955  3347 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:14.564309  3318 consensus_queue.cc:1059] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:20:14.565290  3245 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:20:14.577452  3362 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.580447  3282 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 28 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:14.585451  3278 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.570111  3298 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.589433  3105 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:14.573722  3371 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:14.627974 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 72858.2us
I20260709 12:20:14.600165  3362 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:14.603225  3278 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:14.643447 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:14.645318  3364 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:14.647152 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.647614 28365 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 11 index: 13. Preceding OpId from leader: term: 12 index: 14. (index mismatch)
I20260709 12:20:14.647747  3178 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:14.629200  3347 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:14.637751  3295 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.653393 28983 raft_consensus.cc:2393] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 21: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
W20260709 12:20:14.681084 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 110465us
I20260709 12:20:14.683100 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:14.683770 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:14.695031  3180 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:14.718072 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:14.719519 28366 raft_consensus.cc:1240] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 14. Current term is 15. Ops: []
W20260709 12:20:14.720446  3380 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:20:14.676208  3365 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:14.721010  3368 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:14.721114  3365 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.721971  3368 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:14.721906  3326 consensus_queue.cc:1059] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:20:14.726293  3180 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:14.726938  3180 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.729085  3180 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:14.730353  3375 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:20:14.736109 28288 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:20:14.680058  3282 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.696376 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:14.664515  3295 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.747221 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:14.748188 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:14.752345 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:14.754690 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:14.762710 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:14.763525 28286 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 23.
I20260709 12:20:14.768923 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:14.764487  3231 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:20:14.774374  3385 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:20:14.773876 28435 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:20:14.778414 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.777029 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.780908  3325 raft_consensus.cc:3055] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:14.781395  3325 raft_consensus.cc:740] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:14.782431  3325 consensus_queue.cc:260] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.783742  3325 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:14.752992  3388 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:14.790277  3231 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.790972  3231 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.793567 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.798110 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.798846 28439 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
I20260709 12:20:14.764807  3381 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:14.797389 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.803289 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:14.807576 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:14.806252 28363 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:20:14.808375 28360 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 12.
I20260709 12:20:14.810808 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:14.812628 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:14.787631  3309 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.815116  3101 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:14.820171  3278 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.809448  3175 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.795471  3363 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:20:14.825601 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:14.825693 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.826511 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.826514 28437 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:14.828948 28285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 1620) took 1254 ms. Trace:
I20260709 12:20:14.875573 28285 rpcz_store.cc:276] 0709 12:20:13.574318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.574479 (+   161us) service_pool.cc:224] Handling call
0709 12:20:14.828760 (+1254281us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54002418d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:14.828875 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1216219520}
I20260709 12:20:14.750319  3347 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.845309  3362 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.831875  3295 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:14.794239  3392 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.068s
I20260709 12:20:14.835807 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:14.836447  3308 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:14.838424 28434 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:20:14.839109 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:14.840251 28438 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:14.842067 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.843989 28954 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:14.900184 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.845075  3364 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:14.846016  3371 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.846235 28989 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:20:14.840159  3239 raft_consensus.cc:3055] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:14.874856 28439 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
W20260709 12:20:14.830330  3383 log.cc:927] Time spent T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:20:14.837638  3348 raft_consensus.cc:3055] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:14.881286 28292 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:14.882908 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 68568us
I20260709 12:20:14.921026 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:14.885046 28286 raft_consensus.cc:1240] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:14.865401  3170 raft_consensus.cc:3055] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:14.952893 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:14.955492  3295 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:14.956420  3295 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:14.918967  3239 raft_consensus.cc:740] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:14.921679  3286 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:14.958148 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:14.958367  3239 consensus_queue.cc:260] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:14.960932  3309 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:14.902446 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 58572.9us
I20260709 12:20:14.905656 28989 pending_rounds.cc:85] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 12
I20260709 12:20:14.874596  3298 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:14.875353  3175 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:14.909140  3177 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:14.914106  3365 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:14.851568  3101 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.969851 28989 pending_rounds.cc:107] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:20:15.007969 28989 raft_consensus.cc:2889] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:14.880556  3231 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.954492 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:14.971195 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:15.013891 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.905297  3364 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
W20260709 12:20:15.009244 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 54850.8us
I20260709 12:20:14.919332  3348 raft_consensus.cc:740] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:15.016398 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:15.022750  3175 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:14.951996  3170 raft_consensus.cc:740] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:20:15.026895 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 55808.2us
I20260709 12:20:14.981716  3242 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.006922  3177 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.030215  3381 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:15.015707  3101 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:15.026242  3178 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:15.073789  3278 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.095214 28441 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:15.031539  3365 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:20:15.093761  3399 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:20:15.078732  3178 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.065131  3177 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.028425 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:15.050091  3295 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
W20260709 12:20:15.167441  3394 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:20:15.035916  3242 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.163550  3178 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.164031  3365 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.113595  3359 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.077s
W20260709 12:20:15.183511  3105 consensus_meta.cc:287] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:20:15.199872  3231 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:15.200578  3231 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.203214  3231 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:15.192745 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 285966us
W20260709 12:20:15.193367 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 420674us
I20260709 12:20:15.185159  3278 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.204463  3278 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.206388  3362 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.206810  3278 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.122361  3348 consensus_queue.cc:260] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.209414  3348 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:15.212373  3101 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.122047  3364 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.179214 28291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 3664) took 1052 ms. Trace:
I20260709 12:20:15.125202  3170 consensus_queue.cc:260] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.227761  3170 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:15.230969  3362 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:15.231539  3362 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:15.231963  3362 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:15.214574  3364 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:15.217006 28291 rpcz_store.cc:276] 0709 12:20:14.126978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.160713 (+ 33735us) service_pool.cc:224] Handling call
0709 12:20:14.162861 (+  2148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:14.379566 (+216705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.178918 (+799352us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54002409d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:15.179138 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422835072}
W20260709 12:20:15.249588 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45174.2us
I20260709 12:20:15.250223 28290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 3665) took 1088 ms. Trace:
I20260709 12:20:15.250545 28330 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:15.250789 28290 rpcz_store.cc:276] 0709 12:20:14.127493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.161219 (+ 33726us) service_pool.cc:224] Handling call
0709 12:20:14.260414 (+ 99195us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b5800000098. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb762a22 00007f85c467b8d9 00007f85c37d5ee0 00007f85c389f0ab 00007f85c389f131 00007f85c3860fc0 00007f85c384b929 00007f85c384b3d2 00007f85bcdc2bd4
0709 12:20:14.260637 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:14.363575 (+102938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.215315 (+851740us) spinlock_profiling.cc:243] Waited 98.8 ms on lock 0x7b54003d9fd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:15.215530 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233083520}
I20260709 12:20:15.266613 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:15.267863 28437 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:15.277660 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.204061  3231 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:15.290953 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 270071us
W20260709 12:20:15.298072 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.302732  3231 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.303669  3381 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:15.195411  3365 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.305258  3231 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.305374  3386 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:20:15.302757 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 274406us
I20260709 12:20:15.308111 28290 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:20:15.218772  3105 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.288583  3236 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:20:15.289223  3124 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:20:15.334354 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:15.330490 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:15.336835 28437 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
W20260709 12:20:15.337813 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 502000us
I20260709 12:20:15.339634 28283 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:15.310705  3386 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:15.350890 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:15.351753 28365 raft_consensus.cc:2393] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:15.358230 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:15.363473 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:15.366077 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:15.366881 28365 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:15.371196 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:15.364168 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:15.375356 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:15.376112 28290 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
W20260709 12:20:15.371371 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 120926us
I20260709 12:20:15.349573  3386 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:15.384872 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:15.385703 28438 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:20:15.392813 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:15.358274  3409 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.394367 28990 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:15.396656 28285 raft_consensus.cc:1275] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:15.400451  3377 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.401881  3101 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.407444  3416 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:15.408561  3416 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:15.391656  3242 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:15.417155 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 59167us
I20260709 12:20:15.419231 28257 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:15.393477  3365 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:15.401089  3377 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.397702  3407 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:15.398375  3409 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.426390 28364 raft_consensus.cc:1275] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:20:15.421638 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 67116.3us
I20260709 12:20:15.428083  3401 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:15.431618 28365 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:20:15.435032 28257 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:15.439853 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:15.413486  3295 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.444927  3295 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.448179  3295 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.420655  3124 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:15.448936  3124 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.407331  3362 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.451375  3124 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.453799  3362 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.422009  3347 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:15.457573 28364 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:20:15.461207  3401 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:15.463912  3161 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:15.466178  3233 raft_consensus.cc:3055] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:15.466565  3233 raft_consensus.cc:740] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:15.458937 28330 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:15.469573 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:15.474238 28364 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:20:15.482707 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 118663us
I20260709 12:20:15.484625 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:15.488703 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:15.495744  3347 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:15.459824  3418 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.498751 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:20:15.505007 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.529269 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:15.532241 28285 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:15.440461  3242 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:20:15.498489 28364 raft_consensus.cc:1275] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:20:15.466107  3416 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.538072 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:15.538878 28438 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:15.531627 28441 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
W20260709 12:20:15.496241  3406 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:20:15.542071 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:15.496249  3347 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:15.551306 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 92717.1us
I20260709 12:20:15.440660  3407 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:15.554308 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 95472.6us
I20260709 12:20:15.526711  3105 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.563186 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:15.551398  3101 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:20:15.563674 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 103268us
I20260709 12:20:15.568668 28438 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:20:15.570179 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:15.467489  3233 consensus_queue.cc:260] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.556288  3347 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:15.545111  3418 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.553222  3177 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.596946 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:15.598294 28985 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:15.568131  3101 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.590025  3105 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:15.583516  3415 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.621449 28287 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:15.601933  3278 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.610487  3101 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:15.603722  3362 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:15.602567  3419 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:20:15.624027  3413 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.630915  3105 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 22 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:20:15.652236 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.657094 28436 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:15.667203  3242 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:15.686456  3242 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:15.686964  3342 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 1.610s	user 0.006s	sys 0.015s
I20260709 12:20:15.687300  3418 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.689059  3242 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.665814  3413 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.690749 28291 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:15.690992 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56626 (request call id 1372) took 2198 ms. Trace:
I20260709 12:20:15.691349 28358 rpcz_store.cc:276] 0709 12:20:13.492711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.499297 (+  6586us) service_pool.cc:224] Handling call
0709 12:20:13.536194 (+ 36897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:13.542418 (+  6224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:13.542452 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:13.542951 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:20:13.733711 (+190760us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:13.733779 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:13.733845 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:15.686266 (+1952421us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:20:15.690541 (+  4275us) raft_consensus.cc:1685] finished
0709 12:20:15.690580 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:15.690797 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5785,"raft.queue_time_us":6964,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"spinlock_wait_cycles":4504832,"thread_start_us":183731,"threads_started":1,"wal-append.queue_time_us":345894}
I20260709 12:20:15.692620  3413 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.658906  3409 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:15.645588  3377 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.658737  3412 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:15.695611 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:15.699689 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 978) took 1288 ms. Trace:
I20260709 12:20:15.700176 28989 rpcz_store.cc:276] 0709 12:20:14.411126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.411351 (+   225us) service_pool.cc:224] Handling call
0709 12:20:14.845847 (+434496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.008484 (+162637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.699449 (+690965us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54007eac50. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8868ad 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:15.699612 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":998425856}
I20260709 12:20:15.642256  3362 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.703437 28436 raft_consensus.cc:2468] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:15.704413  3362 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.639290  3415 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.707487  3415 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.708312  3400 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:15.709014  3400 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.711119  3233 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:15.711822  3400 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:15.715458 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.719854 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:15.632051  3278 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.726665  3278 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.729197  3177 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:15.731738  3428 log.cc:927] Time spent T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:20:15.733649 28287 raft_consensus.cc:1275] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:15.734189 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:15.739275  3407 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.648743  3431 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.729632 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:15.745427  3105 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:15.751425  3178 consensus_meta.cc:287] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.001s
I20260709 12:20:15.770982 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:15.773890 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 1630) took 1463 ms. Trace:
W20260709 12:20:15.773540 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 193526us
I20260709 12:20:15.774307 28289 rpcz_store.cc:276] 0709 12:20:14.310377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.347979 (+ 37602us) service_pool.cc:224] Handling call
0709 12:20:15.773652 (+1425673us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54003322d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:15.773817 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259866368}
W20260709 12:20:15.775604 28288 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39592 (request call id 1621) took 2187 ms (client timeout 1858 ms). Trace:
W20260709 12:20:15.777949 28288 rpcz_store.cc:269] 0709 12:20:13.583679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.583848 (+   169us) service_pool.cc:224] Handling call
0709 12:20:15.768842 (+2184994us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54000f2654. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:15.768971 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2644365056}
I20260709 12:20:15.781255 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:15.783151 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 218807us
I20260709 12:20:15.756646  3415 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:15.767643  3400 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:15.788682 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 220486us
I20260709 12:20:15.806622 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:15.797889  3412 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:15.817366 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.819046  3381 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:20:15.804378  3165 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.824240  3165 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.825388 28366 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:20:15.826088 28366 pending_rounds.cc:85] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 10
I20260709 12:20:15.826395 28366 pending_rounds.cc:107] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:20:15.826715 28366 raft_consensus.cc:2889] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:15.827669  3201 consensus_queue.cc:1048] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:15.828272 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:15.830515  3246 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:15.830922  3246 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:15.771576  3431 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.775453  3178 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.797654  3177 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:15.858124  3177 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.811784  3412 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.804567  3401 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.860834 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:15.861238  3401 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.862607  3412 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.794296  3415 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.848264  3101 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.865237  3415 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:15.866298  3101 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.804420  3371 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:15.867230  3371 consensus_queue.cc:1048] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:15.867962 28436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 4025) took 1033 ms. Trace:
I20260709 12:20:15.868347 28436 rpcz_store.cc:276] 0709 12:20:14.834885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.838816 (+  3931us) service_pool.cc:224] Handling call
0709 12:20:15.867758 (+1028942us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400416c54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:15.867864 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751300864}
I20260709 12:20:15.860522  3177 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.853763 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 3565) took 1158 ms. Trace:
I20260709 12:20:15.867892 28989 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:15.871515 28292 rpcz_store.cc:276] 0709 12:20:14.695039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.695164 (+   125us) service_pool.cc:224] Handling call
0709 12:20:14.695528 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:14.920288 (+224760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.853480 (+933192us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54003f9b54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:15.853665 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426631936}
I20260709 12:20:15.826548  3165 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:15.866762 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:15.881372 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:15.833346  3417 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.833847  3246 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:15.904186 28439 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
W20260709 12:20:15.904870 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 86191.9us
I20260709 12:20:15.905833 28292 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:15.905690 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:15.822297  3387 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.114s
I20260709 12:20:15.863521  3401 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:15.907500  3392 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:15.908475  3431 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:15.910562  3443 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:15.914250 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.799280  3400 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.879630  3442 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.879719  3177 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.813673  3368 consensus_queue.cc:1059] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:20:15.913872 28332 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:15.913022  3387 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:15.917626  3420 consensus_queue.cc:1048] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:20:15.929134  3364 consensus_queue.cc:1059] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:15.942060 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:15.946039 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:15.916184  3126 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:15.951311 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:15.954931 28982 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:15.953250 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:15.963532 28988 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:20:15.938429  3260 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:15.956947 28991 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:20:15.906155  3417 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:15.969090  3180 raft_consensus.cc:2804] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:15.949884  3387 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:15.967062  3126 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.988715  3183 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.965705  3260 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:15.949052  3347 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.951352  3177 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:15.940447  3443 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.011317  3313 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:16.001848  3126 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:15.970048  3395 consensus_queue.cc:1059] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.083s
W20260709 12:20:16.020607 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 106891us
I20260709 12:20:16.014935  3260 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.045277 28986 raft_consensus.cc:1240] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:15.946436  3442 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.047258  3177 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.048306  3442 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.050555  3400 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:16.053357  3386 consensus_meta.cc:287] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.661s	user 0.011s	sys 0.000s
I20260709 12:20:16.055296  3443 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:16.022930  3183 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.070003  3183 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.057978  3347 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.037472  3387 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.074393  3381 consensus_queue.cc:1059] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:16.067020 28283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 3570) took 1335 ms. Trace:
I20260709 12:20:16.067584  3395 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.077975 28283 rpcz_store.cc:276] 0709 12:20:14.731582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.734656 (+  3074us) service_pool.cc:224] Handling call
0709 12:20:14.734923 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.340098 (+605175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.066739 (+726641us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b5400164254. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:16.066943 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390244608}
I20260709 12:20:16.033890  3180 raft_consensus.cc:697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:20:16.119355 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 125198us
I20260709 12:20:16.120359 28330 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:16.123428 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:16.124040 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:16.124312 28435 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:16.115566  3381 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:16.126072 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:16.082006  3386 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.132197 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:16.132853 28441 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:16.116230  3395 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.128327  3458 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:16.139079 28358 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:16.138195  3295 raft_consensus.cc:3055] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:20:16.140153 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:16.129673  3381 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.140937  3395 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.148455 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:16.152182  3386 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:16.154387  3126 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.149369  3180 consensus_queue.cc:237] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.142730  3458 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.161774 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:16.162415 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:16.165990 28989 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:20:16.163216 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.143718  3295 raft_consensus.cc:740] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:16.166687 28363 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:16.156899  3126 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.167675 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:16.146873  3381 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.145823  3461 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:16.164640  3172 consensus_queue.cc:1059] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 12:20:16.186172 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.187232 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.188601 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:16.189225 28367 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:16.179864 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.180589 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:16.160163  3442 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:16.192073  3126 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:16.170428  3422 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:16.192576  3126 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:16.193331 28364 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:16.194298 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.192989  3422 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.165313  3443 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:16.195145 28364 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:20:16.208209 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.208608 28989 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 22.
I20260709 12:20:16.209775 28404 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:16.170939  3458 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.204365 28364 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.193090  3126 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.217239 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.191452  3461 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.202895  3371 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:16.222528 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.194840  3442 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.224081 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:16.202555  3291 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:16.230741 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.226671  3461 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:16.214322  3443 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.233567 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:16.235118 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 104962us
I20260709 12:20:16.237493 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.220153  3201 raft_consensus.cc:2804] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:16.243047 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.242111 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.243921 28363 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:16.230974  3442 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:16.234138  3291 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:16.246152 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:16.236256  3443 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:16.217979  3246 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.454448  3313 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:16.230734  3371 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.455175  3313 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.455523  3245 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:16.456211  3245 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.456940  3371 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.457460  3313 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:16.459947  3368 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:16.460597  3368 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.445715  3310 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.463477  3310 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.469379 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:16.470381 28989 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 25: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:16.486418 28433 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:16.478261  3245 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:16.511394 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:16.512228 28989 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 17.
I20260709 12:20:16.514091 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:16.527329  3420 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:16.531539  3420 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:16.532032  3420 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.500550  3310 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:16.509845  3368 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:16.580220  3467 log.cc:927] Time spent T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:20:16.585512 28367 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:16.598399  3246 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:16.272554  3347 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:16.272650  3463 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:20:16.247299  3201 raft_consensus.cc:697] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 22 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:16.616007  3347 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.616919 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.616570  3201 consensus_queue.cc:237] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:16.323334 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.323961  3422 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:16.247530  3260 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.360325 28359 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:20:16.621192 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.626684 28359 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:16.632959  3183 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.633697  3183 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.235742  3295 consensus_queue.cc:260] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 12.14, Last appended by leader: 14, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.636636  3183 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:16.622152  3163 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.638262 28363 raft_consensus.cc:1240] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:16.242111 28255 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:16.638613  3163 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.639076 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:20:16.641484 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.648748  3260 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.619129 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.618667  3347 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.641126  3163 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:16.608310  3461 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.243782 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:16.653638  3443 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.659931 28362 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:16.708320  3443 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:16.668344 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:16.687835  3472 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.709944  3472 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.711357  3458 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.713822  3458 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.714394  3163 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.714968  3163 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.717175  3163 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.717983  3260 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:16.719866  3461 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.701053  3347 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:16.672143  3126 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.728161  3126 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.735245 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:16.736107 28435 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:16.703444 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 459720us
I20260709 12:20:16.710534  3388 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:16.738554 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:16.653638  3442 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.720270  3291 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.765980 28437 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:16.790786 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:16.785413 28989 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:16.785835 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:16.787022 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:16.800635 28435 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
W20260709 12:20:16.802875 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 560831us
I20260709 12:20:16.804025 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:16.804651 28990 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:16.804855 28255 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:16.801864 28332 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:20:16.806020 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.806809 28436 raft_consensus.cc:3055] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:16.811690 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:16.845160 28436 raft_consensus.cc:740] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:16.846772 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:16.845899 28255 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:16.847563 28985 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:16.846024 28990 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:16.783035  3181 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:16.766570  3443 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:16.774528  3372 log.cc:927] Time spent T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 2.373s	user 0.014s	sys 0.029s
I20260709 12:20:16.764953  3458 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:16.855432 28255 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:16.862099 28985 raft_consensus.cc:1240] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:20:16.862591 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:16.772580  3291 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:16.863257 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:16.782895  3246 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:16.866890 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:16.867630 28990 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:16.854725  3183 raft_consensus.cc:2804] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:16.883540  3183 raft_consensus.cc:697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:16.884579  3183 consensus_queue.cc:237] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.848769  3422 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.870227 28985 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:16.865061 28436 consensus_queue.cc:260] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:16.888379 28436 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:16.767424  3420 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.861042  3181 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.868481  3458 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.874193  3442 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.861334  3464 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.877123 28286 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 3571) took 2145 ms. Trace:
I20260709 12:20:16.769619  3368 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.090s
I20260709 12:20:16.878655  3437 raft_consensus.cc:3055] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:16.879128  3470 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:16.879549  3367 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:16.918354  3126 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:16.918848  3126 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:16.919101  3367 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:16.919535  3367 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.919899  3367 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:16.919822  3126 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.880889  3469 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:16.854727  3347 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:16.881453 28361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 3773) took 2579 ms. Trace:
W20260709 12:20:16.928079  3452 log.cc:927] Time spent T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.938s	user 0.000s	sys 0.012s
I20260709 12:20:16.802228  3461 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:20:16.881968  3472 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:16.930500 28360 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 3709) took 1061 ms. Trace:
I20260709 12:20:16.878693 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:16.931882 28437 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:16.893806  3424 log.cc:927] Time spent T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 1.351s	user 0.014s	sys 0.023s
I20260709 12:20:16.928021 28361 rpcz_store.cc:276] 0709 12:20:14.302086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.308865 (+  6779us) service_pool.cc:224] Handling call
0709 12:20:14.309222 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:14.315813 (+  6591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:14.315846 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:14.318609 (+  2763us) log.cc:844] Serialized 26 byte log entry
0709 12:20:14.384612 (+ 66003us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:14.384656 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:14.384729 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:16.773888 (+2389159us) spinlock_profiling.cc:243] Waited 6.82 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:20:16.881097 (+107209us) raft_consensus.cc:1685] finished
0709 12:20:16.881121 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.881376 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6787,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":2836,"spinlock_wait_cycles":15696640,"thread_start_us":55622,"threads_started":1,"wal-append.queue_time_us":72199}
W20260709 12:20:16.908273  3453 log.cc:927] Time spent T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.917s	user 0.000s	sys 0.034s
W20260709 12:20:16.894132  3446 log.cc:927] Time spent T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.908s	user 0.003s	sys 0.025s
I20260709 12:20:16.934235 28440 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 3536) took 1572 ms. Trace:
I20260709 12:20:16.933915  3470 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.888450  3422 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.937165  3422 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:16.933229 28435 raft_consensus.cc:1240] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:16.912923  3437 raft_consensus.cc:740] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:16.897816 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:16.946449 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.900887 28436 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:20:16.955349 28985 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.911381 28286 rpcz_store.cc:276] 0709 12:20:14.735199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.771949 (+ 36750us) service_pool.cc:224] Handling call
0709 12:20:14.772262 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:14.943739 (+171477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.876823 (+1933084us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54003db3d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:16.877022 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258944256}
I20260709 12:20:16.916575  3417 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:16.947865 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.943717 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:16.956619 28283 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:16.947569 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 13 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:16.908609 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:16.934657 28440 rpcz_store.cc:276] 0709 12:20:15.361937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.362074 (+   137us) service_pool.cc:224] Handling call
0709 12:20:15.362436 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:15.362780 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.362805 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:15.391250 (+ 28445us) log.cc:844] Serialized 26 byte log entry
0709 12:20:15.442440 (+ 51190us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:15.458969 (+ 16529us) log.cc:844] Serialized 14 byte log entry
0709 12:20:15.486265 (+ 27296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.486350 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:16.882738 (+1396388us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:20:16.933914 (+ 51176us) raft_consensus.cc:1685] finished
0709 12:20:16.934025 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.934172 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33039,"prepare.queue_time_us":16528,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":44,"raft.queue_time_us":80037,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":19893,"spinlock_wait_cycles":185944832,"thread_start_us":28432,"threads_started":1,"wal-append.queue_time_us":37206}
I20260709 12:20:16.952644 28437 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:16.947395 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:16.955603 28436 pending_rounds.cc:85] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 10
I20260709 12:20:16.958909 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:16.959491 28441 raft_consensus.cc:2410] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:20:16.959805 28437 raft_consensus.cc:2393] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 22: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:16.944335 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:16.960521 28290 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:20:16.944334 28285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:16.962325 28285 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:20:16.949525 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 3784) took 2224 ms. Trace:
I20260709 12:20:16.963913 28366 rpcz_store.cc:276] 0709 12:20:14.724727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.724916 (+   189us) service_pool.cc:224] Handling call
0709 12:20:14.725431 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.827269 (+1101838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.949253 (+1121984us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400134a54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:16.949458 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2528530816}
I20260709 12:20:16.959415 28436 pending_rounds.cc:107] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:20:16.965574 28436 raft_consensus.cc:2889] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:16.956555 28359 raft_consensus.cc:2368] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19.
I20260709 12:20:16.958002 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:16.930883 28360 rpcz_store.cc:276] 0709 12:20:15.869459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.869622 (+   163us) service_pool.cc:224] Handling call
0709 12:20:15.869976 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:15.870353 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.870383 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:15.871067 (+   684us) log.cc:844] Serialized 52 byte log entry
0709 12:20:15.989306 (+118239us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:15.989694 (+   388us) log.cc:844] Serialized 14 byte log entry
0709 12:20:15.989898 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.989981 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:16.927469 (+937488us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:20:16.930150 (+  2681us) raft_consensus.cc:1685] finished
0709 12:20:16.930255 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.930428 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":621,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":13341,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":5668,"spinlock_wait_cycles":14363008,"thread_start_us":117787,"threads_started":1,"wal-append.queue_time_us":118169}
I20260709 12:20:16.968641 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.969026 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:16.969533 28367 raft_consensus.cc:2393] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:16.948558 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.971541 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.971578 28361 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 17: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:16.974603 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.976020 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:16.976648 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.976521 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:16.978777 28361 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:20:16.978979 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 116462us
I20260709 12:20:16.946146  3226 raft_consensus.cc:3055] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:16.980463 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:16.980758 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:16.982230 28366 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 21: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:16.981335 28255 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:17.023286 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:16.887630  3246 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.937820  3387 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:16.946897  3126 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:16.937716  3422 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:16.984668 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.000689 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:16.992899 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:16.968542  3181 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:17.031216 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 170486us
W20260709 12:20:16.951817 28954 connection.cc:256] RPC call timeout handler was delayed by -1.4484730859999218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
I20260709 12:20:16.951188  3420 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:16.981482  3389 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:20:16.931627  3458 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.040416 28440 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.039621 28292 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:20:16.938298  3195 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:17.043984 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.983435  3226 raft_consensus.cc:740] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:17.051805 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:17.026038  3363 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.045552 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:16.983234  3417 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:17.035331 28985 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:17.050935 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.050506 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.022921  3462 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:17.055684 28432 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:20:17.055653 28991 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:20:17.044787 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:17.057375  3226 consensus_queue.cc:260] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.038568  3472 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:20:17.047870  3422 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:16.948469  3437 consensus_queue.cc:260] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.088794  3470 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.097788 28363 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:20:17.106971 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:17.030531  3473 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:16.933284  3464 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:17.137907 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 93167.6us
I20260709 12:20:17.038612  3347 consensus_queue.cc:1059] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:20:17.145685 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.154734 28289 raft_consensus.cc:2393] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 24: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:17.156467 28440 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
I20260709 12:20:17.143692  3188 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:17.158057 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:17.107472  3417 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:17.177770 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:17.180672 28283 raft_consensus.cc:3060] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:17.155659  3226 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:17.185889 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.144651  3461 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:17.178721 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.188800 28289 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:20:17.191808 28330 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:17.100312  3363 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.174350  3405 raft_consensus.cc:3055] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:17.107793  3462 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.195242 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.228778 28289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.176791  3437 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:17.179183  3165 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:17.227095 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.102454  3195 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:17.229943 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 187963us
I20260709 12:20:17.230743 28289 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:17.182991  3347 consensus_queue.cc:1243] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:17.242920 28289 raft_consensus.cc:1240] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:20:17.244732 28289 raft_consensus.cc:1240] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:17.247568 28289 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 28 LEADER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 12:20:17.190353  3461 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.204419  3405 raft_consensus.cc:740] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:17.240131  3417 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.233074 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.250124 28985 raft_consensus.cc:1240] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 19. Current term is 21. Ops: []
I20260709 12:20:17.251438 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.196527  3273 raft_consensus.cc:2804] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:17.196206  3475 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:20:17.196206  3377 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:17.271389  3472 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.272303  3477 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:17.179066  3188 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.273034  3178 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:17.248821  3165 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.276377  3480 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:17.277343  3246 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:17.285998 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.339301 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.340170 28985 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:17.298615 28358 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 3711) took 1424 ms. Trace:
I20260709 12:20:17.341902 28358 rpcz_store.cc:276] 0709 12:20:15.874388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.875076 (+   688us) service_pool.cc:224] Handling call
0709 12:20:15.875343 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.142891 (+267548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.298011 (+1155120us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b5400135e54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:17.298510 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606098688}
I20260709 12:20:17.311890 28983 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
W20260709 12:20:17.324841 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.317204  3461 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:17.345088 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.348477 28283 raft_consensus.cc:2468] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 22.
I20260709 12:20:17.348836 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:17.334720 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:17.349318 28367 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:17.350332 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:17.350893 28434 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:17.351285 28289 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:20:17.352837 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.363180 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.364130 28985 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:20:17.367776 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.368057 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.367776 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.325754  3273 raft_consensus.cc:697] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:17.348007  3480 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.338804  3165 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:17.374233  3442 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:17.330098  3477 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.328387  3472 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.328784  3377 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.371407 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.377393 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:17.376189  3405 consensus_queue.cc:260] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.424821 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:17.433153 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.458616 28983 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:17.432093 28286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:17.460086 28286 raft_consensus.cc:3060] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:17.437911 28291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
W20260709 12:20:17.383795 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.373994  3188 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:17.389843  3195 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:17.390664 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:17.391140 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.394591 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:17.347625  3350 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.351239  3431 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:17.374910 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.371536  3473 raft_consensus.cc:697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:17.444262  3480 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:17.574142 28982 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:17.457890  3388 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.670s	user 0.004s	sys 0.015s
I20260709 12:20:17.577908  3388 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:17.556152 28983 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:17.410583  3442 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.421583  3273 consensus_queue.cc:237] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.580892  3442 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:17.582497 28283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:17.444358 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:17.581313 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:17.502867 28284 raft_consensus.cc:2393] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 20: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:17.439015 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.434212 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:17.490630 28990 consensus_meta.cc:287] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.620s	user 0.017s	sys 0.009s
I20260709 12:20:17.585058 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.482759 28991 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:17.442188 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:17.586572 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.574453 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.583003 28434 raft_consensus.cc:2368] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19.
I20260709 12:20:17.451790 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.587184 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.588650 28288 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:17.587383 28987 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
I20260709 12:20:17.589847 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.589946 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.590608 28288 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:17.583556 28283 raft_consensus.cc:3060] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:17.420856 28440 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:17.591616 28288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8" is_pre_election: true
I20260709 12:20:17.594137 28438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 4047) took 1800 ms. Trace:
I20260709 12:20:17.585587 28990 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:20:17.590535 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:17.597421 28983 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.596591 28438 rpcz_store.cc:276] 0709 12:20:15.793987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.794194 (+   207us) service_pool.cc:224] Handling call
0709 12:20:15.819894 (+ 25700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.876998 (+ 57104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.877037 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:15.877083 (+    46us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:15.877118 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.593900 (+1716782us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b5400085f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:17.593953 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:17.594076 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130719488}
I20260709 12:20:17.600713 28290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
I20260709 12:20:17.590451 28987 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 16.
I20260709 12:20:17.603184 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.604004 28987 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:17.605093 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:20:17.610934 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:17.613577 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 222997us
I20260709 12:20:17.614949 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.615911 28991 raft_consensus.cc:2393] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 23: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:17.616662 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:17.453995  3367 consensus_meta.cc:287] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.534s	user 0.013s	sys 0.007s
W20260709 12:20:17.619894 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 183144us
I20260709 12:20:17.425362  3422 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:17.624869 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.436242  3488 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:17.641779 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.434144  3462 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:17.622103 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:17.626328 28982 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:20:17.452711 28433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 4057) took 1419 ms. Trace:
I20260709 12:20:17.442512  3374 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:17.652953 28291 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:17.651698 28433 rpcz_store.cc:276] 0709 12:20:16.033062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.033243 (+   181us) service_pool.cc:224] Handling call
0709 12:20:16.033512 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.486967 (+453455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.452477 (+965510us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400085a54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:17.452645 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041025792}
I20260709 12:20:17.627660 28991 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 28 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:20:17.613142  3313 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:17.622961  3262 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:17.629792  3324 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:17.630419  3486 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:461: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 12:20:17.504647  3366 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.175s
I20260709 12:20:17.456103  3318 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.182s
I20260709 12:20:17.638229  3473 consensus_queue.cc:237] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.638104 28288 raft_consensus.cc:1240] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
W20260709 12:20:17.611331  3451 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 1.624s	user 0.015s	sys 0.001s
I20260709 12:20:17.641598 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.446121  3477 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.645989  3484 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.643651 28983 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
I20260709 12:20:17.607740  3350 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.676159  3350 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:17.676983 28988 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:17.658968  3374 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.650220  3488 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.678254  3374 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:17.679989  3488 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:17.680581  3488 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.681015  3488 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.640012 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:17.642655 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.426193  3363 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.647802 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.683530 28436 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
W20260709 12:20:17.682808 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 54704.3us
I20260709 12:20:17.623090 28284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a3de34b6f4cf4606a61944db6e2739c8"
W20260709 12:20:17.642663 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.649281 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.438033  3483 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:17.442744  3464 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.654383 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.686548 28433 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:17.687069 28284 raft_consensus.cc:3060] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:17.409503  3475 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:17.641600 28283 raft_consensus.cc:2468] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:20:17.665393  3480 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:17.670867 27116 tablet_server.cc:179] TabletServer@127.26.123.1:0 shutting down...
I20260709 12:20:17.673199  3484 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.673620  3377 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.644603  3367 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.699751  3367 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.448750  3472 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.687922 28433 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:17.701447  3488 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:17.703078  3126 consensus_meta.cc:287] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.661s	user 0.017s	sys 0.006s
I20260709 12:20:17.691452 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.702636  3461 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.704350 28436 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 23: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:17.615909  3431 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.689849  3181 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:17.704839 28358 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 21: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:17.704938  3181 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.705406  3181 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:17.706318  3431 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:17.656162  3262 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:17.697038  3484 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:17.703970 28441 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:17.708060  3262 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:17.709218 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.693954  3475 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.705314 28287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59792 (request call id 3585) took 1830 ms. Trace:
I20260709 12:20:17.717808 28286 raft_consensus.cc:2468] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:20:17.717808 28290 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:17.716073 28287 rpcz_store.cc:276] 0709 12:20:15.874563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.874676 (+   113us) service_pool.cc:224] Handling call
0709 12:20:15.874972 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:15.875202 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.875227 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:15.875721 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 12:20:15.987889 (+112168us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:15.988168 (+   279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.988297 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:17.581129 (+1592832us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:20:17.705017 (+123888us) raft_consensus.cc:1685] finished
0709 12:20:17.705042 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:17.705246 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29162880,"thread_start_us":79855,"threads_started":1,"wal-append.queue_time_us":98956}
I20260709 12:20:17.663806  3313 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.723186 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:17.737448 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:17.663697  3324 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:17.763422 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 107110us
I20260709 12:20:17.720098  3483 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.666402  3486 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:17.722676  3461 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:17.773149 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.767831 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:17.784860 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:17.742654  3361 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:17.718710  3363 consensus_queue.cc:1059] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:20:17.742385  3126 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.763327  3495 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:17.792518 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:17.790729 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.788914 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:17.791755 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.793175 28367 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:17.766753  3388 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:17.794965 28360 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:20:17.794982 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.766269  3483 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:17.793123 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 358326us
I20260709 12:20:17.802862 28954 leader_election.cc:400] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19.
I20260709 12:20:17.803258 28954 leader_election.cc:403] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:20:17.792814  3226 consensus_meta.cc:287] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.600s	user 0.020s	sys 0.003s
I20260709 12:20:17.786290  3367 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:17.808687 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:17.806799 28956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.816951 28989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46950 (request call id 1418) took 1050 ms. Trace:
I20260709 12:20:17.817473 28989 rpcz_store.cc:276] 0709 12:20:16.766555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.766735 (+   180us) service_pool.cc:224] Handling call
0709 12:20:16.767138 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.793179 (+ 26041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.816633 (+1023454us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b54004c2554. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:17.816866 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41524608}
I20260709 12:20:17.799186  3488 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:17.810745  3431 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19.
I20260709 12:20:17.782727  3188 consensus_queue.cc:1059] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:20:17.831319  3496 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:17.837795 28284 raft_consensus.cc:2468] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:17.800348  3408 raft_consensus.cc:3055] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:17.839299  3486 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:17.840637 28367 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:20:17.842151 28986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46934 (request call id 996) took 1782 ms. Trace:
I20260709 12:20:17.883960 28986 rpcz_store.cc:276] 0709 12:20:16.060196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.060389 (+   193us) service_pool.cc:224] Handling call
0709 12:20:16.060851 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:16.061326 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:16.061361 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:16.063105 (+  1744us) log.cc:844] Serialized 26 byte log entry
0709 12:20:17.449666 (+1386561us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b18000346c0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:20:17.682532 (+232866us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:17.682573 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.682645 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:17.841757 (+159112us) raft_consensus.cc:1685] finished
0709 12:20:17.841783 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:17.842065 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1258,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":124,"spinlock_wait_cycles":30185856,"thread_start_us":232630,"threads_started":1,"wal-append.queue_time_us":1719271}
I20260709 12:20:17.843711  3313 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:17.851373  3324 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.853825 28364 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:17.798935  3495 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.797636  3126 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.857637  3475 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.860718 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:17.862870 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.820631  3497 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:17.866128 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:17.866990 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:17.869474 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.874004  3165 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.886911 28439 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33102 (request call id 4042) took 2323 ms. Trace:
I20260709 12:20:17.928387 28362 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:17.928859 28439 rpcz_store.cc:276] 0709 12:20:15.560012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.561444 (+  1432us) service_pool.cc:224] Handling call
0709 12:20:15.561811 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:15.904736 (+342925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.883574 (+1978838us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b540026bbd4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:17.883790 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786743168}
W20260709 12:20:17.926690 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 118170us
W20260709 12:20:17.932684 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.937826  3422 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:20:17.939301  3495 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:17.940505 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 79883.2us
I20260709 12:20:17.943779 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.947753  3126 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:17.882840  3408 raft_consensus.cc:740] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:17.949343  3408 consensus_queue.cc:260] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.952248  3181 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.932008 28954 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:17.962806 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:17.927981 28361 raft_consensus.cc:2410] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
W20260709 12:20:17.927340 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.969646 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:17.970688 28359 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:20:17.974334 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.978005 28364 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 28 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 20. (term mismatch)
I20260709 12:20:17.986720 28364 pending_rounds.cc:85] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Aborting all ops after (but not including) 19
I20260709 12:20:17.987063 28364 pending_rounds.cc:107] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Aborting uncommitted NO_OP operation due to leader change: 27.20
I20260709 12:20:17.987447 28364 raft_consensus.cc:2889] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:17.988183 28364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 3793) took 1808 ms. Trace:
I20260709 12:20:17.988581 28364 rpcz_store.cc:276] 0709 12:20:16.180248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.192812 (+ 12564us) service_pool.cc:224] Handling call
0709 12:20:16.193200 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:17.853441 (+1660241us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b580085dcd0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7e30aa 00007f85be865447 00007f85be87dfe7 00007f85be87f7de 00007f85be889b49 00007f85be88a416 00007f85be886d8f 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4
0709 12:20:17.987942 (+134501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.988128 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2344946560}
I20260709 12:20:17.971745  3495 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:17.991110 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 195298us
I20260709 12:20:17.992348 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:17.972437  3408 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:17.940793  3313 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:17.998719 28257 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:17.972779  3404 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
W20260709 12:20:18.003517 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 209687us
I20260709 12:20:18.002751 28257 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:17.995635 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:17.995004  3163 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:17.987669  3318 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
W20260709 12:20:18.007604 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.019748 28439 raft_consensus.cc:1240] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:18.006516 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:18.022814  3165 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.014637 28433 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 23: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:18.016216 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.005775 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:18.004828  3470 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:18.007242  3475 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:18.028856 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.030265 28433 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:17.999997  3374 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:17.977152 28954 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:18.033128 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:18.037150 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.037321 28255 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:18.029840  3475 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.039086 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:18.038482  3367 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:18.040082  3456 log.cc:927] Time spent T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 2.050s	user 0.014s	sys 0.000s
I20260709 12:20:18.032745  3470 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:18.034421  3181 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.035773  3361 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:18.050607 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.041250  3490 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:18.040249  3245 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:18.041477  3475 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:18.043092  3126 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:18.056957 28984 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:18.059705 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 65736.8us
I20260709 12:20:18.049192  3367 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.063231 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.041244  3262 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.074229  3374 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.075443 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:18.076287 28365 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55946 (request call id 3710) took 2203 ms. Trace:
I20260709 12:20:18.051182  3324 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.013s
I20260709 12:20:18.093101  3392 raft_consensus.cc:3055] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:18.058636  3490 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:18.068696 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.098783 28986 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 28 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 12:20:18.065073  3367 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:18.056787  3361 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:18.139598 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.109206 28432 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:18.111443 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.140421 28988 raft_consensus.cc:2393] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 23: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:18.140803 28989 raft_consensus.cc:2393] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 13: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:18.112445 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.141429 28991 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:18.113430 28364 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 22: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:18.113207 28365 rpcz_store.cc:276] 0709 12:20:15.873162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.873345 (+   183us) service_pool.cc:224] Handling call
0709 12:20:15.873682 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:15.882088 (+  8406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.882123 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:15.882706 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:20:15.988165 (+105459us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:15.988227 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.988304 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:17.992978 (+2004674us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:20:18.075962 (+ 82984us) raft_consensus.cc:1685] finished
0709 12:20:18.075982 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.076217 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8583,"raft.queue_time_us":129,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":7470,"spinlock_wait_cycles":11681664,"thread_start_us":36426,"threads_started":1,"wal-append.queue_time_us":71226}
I20260709 12:20:18.059212  3126 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.119565 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.122108 28987 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:18.094205 28366 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:20:18.128219 28330 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:18.064966 28954 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:18.151882 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 87002.2us
I20260709 12:20:18.152639 28954 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:18.132161 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.133057  3378 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:18.138219  3239 raft_consensus.cc:3060] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:18.146683 28406 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.158880 28954 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:18.159091 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.118001  3496 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:18.062381  3483 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.161630 28438 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:20:18.122614  3392 raft_consensus.cc:740] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:18.131995  3461 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.146042  3361 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.178304  3405 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.165350 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.165140  3470 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.174015  3262 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.174373  3295 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:18.174539  3490 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.170352 28330 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:18.168020 28984 raft_consensus.cc:2468] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:20:18.180763 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:18.206799 28364 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:18.158349  3126 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:18.161839  3318 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:18.158028  3389 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:18.187022 28435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 3553) took 1283 ms. Trace:
I20260709 12:20:18.170526  3417 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:20:18.170769  3496 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.164412  3422 consensus_queue.cc:1048] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:20:18.167071  3432 raft_consensus.cc:2804] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:18.164463  3378 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.167673  3195 raft_consensus.cc:2764] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:20:18.191943 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.168996  3464 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:20:18.162467  3374 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:20:18.238610  3499 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:18.202204 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.217386 28991 raft_consensus.cc:2468] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:18.215072 28330 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.219105 28364 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:18.238981 28435 rpcz_store.cc:276] 0709 12:20:16.903181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.903342 (+   161us) service_pool.cc:224] Handling call
0709 12:20:16.903766 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.942309 (+ 38543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.186743 (+1244434us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b54002db4d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:18.186951 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67255808}
I20260709 12:20:18.205595  3165 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:18.253872 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:18.261920 28956 leader_election.cc:400] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19.
W20260709 12:20:18.279575 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64577.9us
I20260709 12:20:18.183261  3461 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.238734  3226 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:18.284092 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 144500us
I20260709 12:20:18.285080 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:18.280562 28956 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:18.287140 28956 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:18.290176 28257 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:18.239598  3426 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
W20260709 12:20:18.305650 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.281256  3359 raft_consensus.cc:2764] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 12:20:18.246011  3389 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.306538  3195 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:18.260488  3495 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:20:18.289283  3170 raft_consensus.cc:2764] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:20:18.286851  3486 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:18.314473 28257 leader_election.cc:400] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:20:18.319423 28257 leader_election.cc:403] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:20:18.309388 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.294828  3497 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:18.321959 28983 raft_consensus.cc:1275] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:18.323115 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:18.289232  3226 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.290014  3262 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:20:18.312089  3163 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:18.331336 28255 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:18.351217  3389 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.353390  3513 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:18.354602  3378 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.354842  3486 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:18.329720  3432 raft_consensus.cc:697] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:18.320173  3195 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:18.358297 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 77460us
W20260709 12:20:18.345455 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 186639us
W20260709 12:20:18.360424 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.358773 28404 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:18.329365  3324 raft_consensus.cc:3060] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 21
W20260709 12:20:18.367439 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 114971us
W20260709 12:20:18.378923 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47673.3us
I20260709 12:20:18.396147 28441 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33110 (request call id 3559) took 1465 ms. Trace:
I20260709 12:20:18.384155  3350 raft_consensus.cc:3055] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:18.391691 28954 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.396785 28441 rpcz_store.cc:276] 0709 12:20:16.915120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.000264 (+ 85144us) service_pool.cc:224] Handling call
0709 12:20:18.380173 (+1379909us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54002e40d4. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:18.380295 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1526652160}
W20260709 12:20:18.401686 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 113241us
I20260709 12:20:18.401899 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.407172 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.402308 28257 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:18.425676 28289 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 3689) took 1038 ms. Trace:
I20260709 12:20:18.426201 28289 rpcz_store.cc:276] 0709 12:20:17.386986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.387546 (+   560us) service_pool.cc:224] Handling call
0709 12:20:17.388724 (+  1178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:17.456867 (+ 68143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:17.456904 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:17.456956 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:17.457001 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.425389 (+968388us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b540006fc54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:18.425447 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.425605 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156037376}
I20260709 12:20:18.378257  3483 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.448946 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.378475  3318 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:18.392069  3480 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:18.465479  3480 raft_consensus.cc:697] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:20:18.466259  3505 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:20:18.466612  3480 consensus_queue.cc:237] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:18.467803  3508 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:20:18.451757 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 49998.3us
I20260709 12:20:18.470324 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:18.471658 28363 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 3808) took 1131 ms. Trace:
I20260709 12:20:18.472074 28363 rpcz_store.cc:276] 0709 12:20:17.339837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.340424 (+   587us) service_pool.cc:224] Handling call
0709 12:20:17.340755 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:17.341154 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:17.341185 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:17.341686 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:20:17.953090 (+611404us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b1800354f40. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85c4662ab1 00007f85c466f281 00007f85bb77ce07 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85be42120b 00007f85be426d23 00007f85be4270d1 00007f85be842a0f
0709 12:20:18.160664 (+207574us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:18.160705 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.160771 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:18.471289 (+310518us) raft_consensus.cc:1685] finished
0709 12:20:18.471315 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.471582 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":811,"raft.queue_time_us":7539,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":16039,"spinlock_wait_cycles":4177792,"thread_start_us":207257,"threads_started":1,"wal-append.queue_time_us":902841}
W20260709 12:20:18.474591  3511 log.cc:927] Time spent T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:20:18.357362  3347 consensus_queue.cc:1059] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:20:18.476044  3347 raft_consensus.cc:3055] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:18.476423  3347 raft_consensus.cc:740] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:18.406390  3170 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:18.479807  3170 consensus_queue.cc:1059] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:18.394476  3195 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:18.462180  3504 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:20:18.462899  3513 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:18.463104  3507 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 12:20:18.482390 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.491142 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:20:18.492362 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43537.4us
I20260709 12:20:18.519816 28441 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 24.
I20260709 12:20:18.495749  3404 consensus_queue.cc:1059] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 12:20:18.507526 28434 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 13.
I20260709 12:20:18.398727  3388 raft_consensus.cc:2804] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:18.400489  3350 raft_consensus.cc:740] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:18.481688  3170 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:18.456812  3163 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:20:18.523756  3273 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.017s	sys 0.014s
I20260709 12:20:18.551198 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.521238 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:18.374434  3475 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.495893  3422 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:18.492610  3426 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:18.503333  3461 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.489245  3226 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.523874  3363 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.492838  3367 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.477311  3347 consensus_queue.cc:260] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.543680  3366 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:18.554833  3163 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:18.555188  3273 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.018s	sys 0.014s
I20260709 12:20:18.555157  3388 raft_consensus.cc:697] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:18.548426  3512 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:18.550098  3170 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:18.580804  3163 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:18.583539  3350 consensus_queue.cc:260] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.518724  3488 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:18.547183  3515 raft_consensus.cc:493] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:18.572633 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:20:18.580860 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 172954us
I20260709 12:20:18.582175  3395 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:18.584196 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 113969us
I20260709 12:20:18.534150  3126 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.601445 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:18.607226 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.568986  3520 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:18.607244  3515 raft_consensus.cc:515] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.614027  3386 raft_consensus.cc:2764] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:20:18.601464  3488 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.489334  3432 consensus_queue.cc:237] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.616715  3324 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
W20260709 12:20:18.617342 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44784.6us
I20260709 12:20:18.639691 28330 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.618986 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.406514  3517 consensus_queue.cc:1059] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.203s
I20260709 12:20:18.527251  3513 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.593323  3170 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:18.647222  3388 consensus_queue.cc:237] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.650105  3426 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:18.650714 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.651507  3426 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:18.630244  3520 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.624387  3475 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.653147  3324 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:18.653673  3324 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.626463  3527 raft_consensus.cc:2764] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:20:18.655656  3324 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.629474 28292 raft_consensus.cc:1275] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:20:18.599507  3347 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:18.658910  3126 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.659533 28292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59778 (request call id 3697) took 1264 ms. Trace:
I20260709 12:20:18.659899 28292 rpcz_store.cc:276] 0709 12:20:17.395514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.636114 (+240600us) service_pool.cc:224] Handling call
0709 12:20:17.636525 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:18.658287 (+1021762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.659296 (+  1009us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54000f2b54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:18.659467 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220534144}
I20260709 12:20:18.660791  3469 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.661283  3469 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.664894  3469 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.640973  3367 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.609822  3512 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.581075  3366 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:18.665910 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47038.5us
I20260709 12:20:18.666741 28954 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.667689  3512 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.642865  3361 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.668339  3527 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:18.647118  3513 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.668900  3528 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:18.669194 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.669597  3361 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:18.621625  3195 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.670409  3520 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:18.671674  3195 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.620209  3165 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:18.672600 28439 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:20:18.674787 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.675168 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:18.644320  3515 leader_election.cc:290] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.644618  3517 raft_consensus.cc:3055] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:18.676029  3517 raft_consensus.cc:740] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:18.570523  3363 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.676824  3517 consensus_queue.cc:260] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.667798  3366 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.675740 28363 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:18.667991 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:18.680830  3363 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.681260 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:18.681768  3195 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:18.685235 28954 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:18.687912  3195 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:18.688316  3195 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.688654  3195 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:18.692495 28365 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:18.697829  3170 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.699098 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:18.703946 28364 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 12.
I20260709 12:20:18.699770 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.700016 28363 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:18.700821  3195 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.705117 28991 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 13.
I20260709 12:20:18.702870 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:18.706425  3366 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:18.706990  3366 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:18.707970  3366 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.708498  3195 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:18.710789 28255 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:18.711732  3347 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:18.711833  3170 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:18.698383  3496 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.652861  3488 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.715624  3520 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.719589  3488 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.720238  3488 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.720561 28185 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 326) took 6296 ms. Trace:
I20260709 12:20:18.711621 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:18.722453  3361 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:18.724396  3361 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.724741  3361 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:18.721300 28185 rpcz_store.cc:276] 0709 12:20:12.423935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.429849 (+  5914us) service_pool.cc:224] Handling call
0709 12:20:18.720461 (+6290612us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37158,"reported_tablets":6}
I20260709 12:20:18.733538 28436 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:20:18.729717  3501 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.007s
I20260709 12:20:18.736972 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:18.740356 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.741070 28991 raft_consensus.cc:2468] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:20:18.741770 28432 raft_consensus.cc:1240] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:18.748490 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.753875 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.756760 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.757553 28991 raft_consensus.cc:2393] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 21: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:18.762933 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.764955 28991 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.765071 28990 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:18.767573 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.768229 28433 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:20:18.722640 28404 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:20:18.770164 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 47628.6us
I20260709 12:20:18.771713 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.756199  3359 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:18.777436 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.778832 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.781642 28989 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 24: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:18.781440 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.782416 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.782891 28984 raft_consensus.cc:2368] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:20:18.774859  3126 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:18.787065 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.787761 28989 raft_consensus.cc:2376] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17. Re-sending same reply.
I20260709 12:20:18.781435  3359 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.789955  3359 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:18.793349 28332 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:20:18.794714  3525 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:20:18.767148  3431 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:20:18.796483  3520 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.797135  3431 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:18.797544  3431 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:18.797672  3417 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:18.798321  3417 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.798691  3417 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:18.798542  3431 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.799933  3431 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:18.803263  3488 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.803791  3517 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:18.801039  3361 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.807333 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:18.810284 28363 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 23.
I20260709 12:20:18.810940 28361 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:18.803968  3126 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:18.807646 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:18.761051  3163 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.807925 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:18.821460 28365 raft_consensus.cc:2393] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 20: Already voted for candidate a3de34b6f4cf4606a61944db6e2739c8 in this term.
I20260709 12:20:18.811964 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.812271 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:18.822741 28990 raft_consensus.cc:2376] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22. Re-sending same reply.
I20260709 12:20:18.823220 28362 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:20:18.807432 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.824434 28989 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 19: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:18.826256 28991 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20.
I20260709 12:20:18.826575 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.827239 28985 raft_consensus.cc:2376] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 20. Re-sending same reply.
I20260709 12:20:18.827996 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:18.828363 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:18.829556  3308 raft_consensus.cc:2804] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:18.834235 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.836493 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:18.820816  3488 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:18.834946 28989 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 23.
I20260709 12:20:18.838791 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:18.839351 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:18.839406 28439 raft_consensus.cc:3060] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:18.839939 28365 raft_consensus.cc:3060] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:18.841632 28433 raft_consensus.cc:1240] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:18.843271 28255 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:18.834832  3308 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:18.853503 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:18.855665 28954 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 26 similar messages]
I20260709 12:20:18.857167 28255 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:18.858603 28365 raft_consensus.cc:2468] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:18.861424 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:18.858772 28255 leader_election.cc:400] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Vote denied by peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985) with higher term. Message: Invalid argument: T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for earlier term 20. Current term is 21.
I20260709 12:20:18.839599  3480 raft_consensus.cc:3060] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:18.895339 28439 raft_consensus.cc:2468] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:20:18.841116  3488 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.833978  3181 consensus_queue.cc:1048] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:20:18.889503 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:18.899523 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:18.854786  3347 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:18.859671  3367 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:20:18.834540  3417 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.860009  3359 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.904057  3163 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.929927  3359 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.932543  3359 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.934002 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:18.934994 28439 raft_consensus.cc:2410] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for term 26 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:20:18.854529  3308 raft_consensus.cc:3060] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:18.902226 28362 raft_consensus.cc:2368] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 21. Current term is 22.
I20260709 12:20:18.947104  3361 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.947890  3126 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:18.957162 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.906392  3496 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:18.860417  3473 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:18.891577  3388 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.860656  3501 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:20:18.964322 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:20:18.964517 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 51316.9us
I20260709 12:20:18.966003 28989 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:18.974718 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 14 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:18.975526 28362 raft_consensus.cc:3060] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:20:18.967100  3388 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.981081  3388 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.982609 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:18.982944 28989 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:20:18.982930 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:18.984535  3417 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.985576 28404 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:18.986104 28991 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:18.986478  3488 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:18.987257 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:18.987967  3181 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:18.988372  3181 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.988726  3181 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.964816  3308 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.991899  3308 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:18.986554  3388 raft_consensus.cc:2804] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:18.994465  3388 raft_consensus.cc:697] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:18.995657  3388 consensus_queue.cc:237] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 12.14, Last appended by leader: 14, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:18.994632  3392 consensus_queue.cc:260] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:18.999295  3392 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:19.021287  3480 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:19.024159 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 345601us
I20260709 12:20:19.023579 28362 raft_consensus.cc:2468] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
W20260709 12:20:19.034705 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 179064us
I20260709 12:20:19.042596 28330 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
W20260709 12:20:19.037616 28954 leader_election.cc:336] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:19.045681 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:19.021291  3496 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:19.056831 28983 raft_consensus.cc:1240] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:19.057482 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.060812 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee4696ba90a447fa07c4d09407217f6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:19.064905 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:19.065959 28987 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
I20260709 12:20:19.069237 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:19.072023 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:19.073092 28984 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 28.
I20260709 12:20:19.073335 28991 raft_consensus.cc:1240] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:19.051472  3386 raft_consensus.cc:2764] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:20:19.080013 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:19.084543 28991 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:19.093204 28330 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:19.097188 28954 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:19.098991 28954 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:19.104199  3195 raft_consensus.cc:2804] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:19.106304 28330 leader_election.cc:304] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:19.077203  3170 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:19.115967 28954 leader_election.cc:400] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 21. Current term is 22.
I20260709 12:20:19.116921 28954 leader_election.cc:403] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:20:19.121982  3464 raft_consensus.cc:3055] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:19.122655  3464 raft_consensus.cc:740] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:19.123445 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:19.123711  3464 consensus_queue.cc:260] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.124459  3165 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:19.127122 28434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33102 (request call id 4090) took 1098 ms. Trace:
I20260709 12:20:19.110924  3437 raft_consensus.cc:2804] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:19.133858  3195 raft_consensus.cc:697] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:19.095578  3318 consensus_queue.cc:1059] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:20:19.146054  3512 consensus_queue.cc:1059] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:19.149601  3181 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.152618 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.166766 28985 raft_consensus.cc:1275] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:20:19.168766  3165 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:19.169723 28434 rpcz_store.cc:276] 0709 12:20:18.028397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.028608 (+   211us) service_pool.cc:224] Handling call
0709 12:20:19.126874 (+1098266us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b5400184f54. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be88b2fa 00007f85c384cb17 00007f85bcdc76d4 00007f85bcdc7662 00007f85bcdc7591 00007f85bcdc750c 00007f85bcdc67c2 00007f85bcbc248c 00007f85bcbc18c6 00007f85bcbc126f
0709 12:20:19.127032 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099244288}
I20260709 12:20:19.104816  3359 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:19.179347 28362 raft_consensus.cc:2393] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 26: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
W20260709 12:20:19.168031 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 63469.6us
I20260709 12:20:19.171129  3431 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:19.176998  3513 raft_consensus.cc:2749] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:19.168726  3426 consensus_queue.cc:1059] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 12:20:19.182732  3437 raft_consensus.cc:697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:19.185184  3165 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:19.194177  3437 consensus_queue.cc:237] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.202418 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.196135  3195 consensus_queue.cc:237] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.204360 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:19.205051 28361 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:19.210201 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:19.210697 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.210215 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.213410  3165 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.215786  3350 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:19.216709  3165 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 29 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:19.209550 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:19.224061 28432 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:20:19.229220 28361 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 21.
I20260709 12:20:19.234673 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.246416 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:19.247469 28363 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:20:19.238598 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:19.248647 28989 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:19.254491 28954 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:19.239250 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:19.239109 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:19.239032 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:19.239492 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:19.240526 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:19.223150 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:19.264958 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:19.265655 28440 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:19.267750 28438 raft_consensus.cc:2410] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:20:19.257081 28985 raft_consensus.cc:1275] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:19.264174 28367 raft_consensus.cc:2393] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 19: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:19.268497 28436 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:19.272554 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:19.274446 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:19.275223 28986 raft_consensus.cc:2376] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17. Re-sending same reply.
I20260709 12:20:19.264966 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:19.281549 28363 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:19.282836 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:19.285136 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:19.286278 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:19.286612 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:19.288455 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:19.289253 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.191172  3395 consensus_queue.cc:1048] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:20:19.261917 28432 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 24.
I20260709 12:20:19.291455 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:19.294829 28985 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:19.245170 28991 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 26.
I20260709 12:20:19.295615 28986 raft_consensus.cc:1275] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:20:19.297281 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:19.298029 28986 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:20:19.296581 28330 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:19.255923  3496 raft_consensus.cc:493] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:19.266672  3477 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.311666 28440 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:19.307808 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:19.308681 28989 raft_consensus.cc:2468] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 29.
I20260709 12:20:19.316167 28991 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:20:19.317492 28330 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:19.314416 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:19.318678 28984 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:19.292623  3473 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:19.281239  3520 consensus_queue.cc:1048] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:20:19.327028 28985 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 24.
I20260709 12:20:19.300676  3359 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:19.291906  3163 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:20:19.286377  3363 consensus_queue.cc:1048] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:20:19.311327  3273 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:19.310310  3374 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:19.317394  3367 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:19.325893  3461 raft_consensus.cc:2804] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:20:19.322235  3347 consensus_queue.cc:1048] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:20:19.319110  3496 raft_consensus.cc:515] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.322497  3477 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:19.374782  3470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.000s	sys 0.044s
W20260709 12:20:19.384027  3470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.000s	sys 0.045s
I20260709 12:20:19.359064  3273 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 26 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:19.346496  3480 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:20:19.392298 28984 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:19.372845  3461 raft_consensus.cc:697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 29 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:19.408833  3404 raft_consensus.cc:3055] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:19.409492  3404 raft_consensus.cc:740] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:19.410470  3404 consensus_queue.cc:260] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.421432 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:19.422246 28440 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:19.422780  3163 raft_consensus.cc:493] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:19.423346  3163 raft_consensus.cc:515] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:19.423879  3540 log.cc:927] Time spent T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:20:19.414812 28332 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.428814  3181 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:19.425604 28366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55954 (request call id 3824) took 1233 ms. Trace:
I20260709 12:20:19.432984 28440 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 26.
I20260709 12:20:19.433766 28366 rpcz_store.cc:276] 0709 12:20:18.191861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.192048 (+   187us) service_pool.cc:224] Handling call
0709 12:20:18.192428 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:18.548849 (+356421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.548889 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:18.549412 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 12:20:18.846203 (+296791us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:20:18.846242 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.425228 (+578986us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400126454. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be887e01 00007f85be88683e 00007f85c384b419 00007f85bcdc2bd4 00007f85bcdc2b62 00007f85bcdc2a91 00007f85bcdc2a0c 00007f85bcdc1cc2 00007f85bcbc248c 00007f85bcbc18c6
0709 12:20:19.425278 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:19.425324 (+    46us) raft_consensus.cc:1685] finished
0709 12:20:19.425337 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:19.425541 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101078,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":57734,"spinlock_wait_cycles":259626368,"thread_start_us":539477,"threads_started":2,"wal-append.queue_time_us":341090}
I20260709 12:20:19.433040 28332 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:19.438778  3163 leader_election.cc:290] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:19.422399  3404 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:19.443706  3518 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.844s	user 0.005s	sys 0.003s
W20260709 12:20:19.448623  3523 log.cc:927] Time spent T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.832s	user 0.000s	sys 0.007s
W20260709 12:20:19.450372  3542 log.cc:927] Time spent T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:20:19.457374  3170 raft_consensus.cc:3055] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:19.457935  3170 raft_consensus.cc:740] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:19.458909  3170 consensus_queue.cc:260] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.459981  3170 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:19.463017  3526 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.463596  3526 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.471361  3464 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:19.475010  3262 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:19.475673  3262 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.482115  3262 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:19.507944 28954 leader_election.cc:336] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:19.509166 28954 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:19.509943  3521 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:19.510619  3521 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.514415  3521 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:19.516878 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:19.491952  3496 leader_election.cc:290] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:19.493645  3477 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:19.494426  3461 consensus_queue.cc:237] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.495919  3347 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:19.509618 28954 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:19.530047 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 97078.6us
I20260709 12:20:19.531281 28441 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 18: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:19.536394 28434 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:19.536952 28437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33110 (request call id 3571) took 1353 ms. Trace:
I20260709 12:20:19.545028 28366 raft_consensus.cc:1240] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Rejecting Update request from peer f526369c0d0743b192be6e205540a059 for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:19.545163 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 70362us
I20260709 12:20:19.543488 28437 rpcz_store.cc:276] 0709 12:20:18.183088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.251727 (+ 68639us) service_pool.cc:224] Handling call
0709 12:20:18.269013 (+ 17286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:18.269420 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.269454 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:18.269968 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:20:18.544008 (+274040us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:18.544051 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.544120 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:19.443080 (+898960us) spinlock_profiling.cc:243] Waited 46.2 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85be42286e 00007f85be421bb6 00007f85be4217d8 00007f85be42b221 00007f85be42b1d9 00007f85be42b169
0709 12:20:19.517980 (+ 74900us) raft_consensus.cc:1685] finished
0709 12:20:19.518004 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:19.536884 (+ 18880us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":826,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":7205,"spinlock_wait_cycles":106230528,"thread_start_us":260347,"threads_started":1,"wal-append.queue_time_us":298750}
W20260709 12:20:19.551093 28954 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:19.551772 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:19.553880 28189 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 333) took 6578 ms. Trace:
I20260709 12:20:19.554476 28189 rpcz_store.cc:276] 0709 12:20:12.975844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:12.976067 (+   223us) service_pool.cc:224] Handling call
0709 12:20:19.553792 (+6577725us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1128461,"reported_tablets":8}
I20260709 12:20:19.559566 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:19.562165 28366 raft_consensus.cc:1240] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:19.532838  3526 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:19.564117 28954 leader_election.cc:336] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:19.565763 28361 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:19.572299 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:19.576924 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819ca6350ca84ff49e7254638d8755e7" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 13 candidate_status { last_received { term: 12 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:19.579705 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:19.542579  3431 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 for earlier term 21. Current term is 22.
I20260709 12:20:19.545622  3513 raft_consensus.cc:2804] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:19.599925 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.600910 28365 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:19.602756 28361 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 12:20:19.603410 28181 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:19.566197  3318 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:19.553071  3359 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:19.611760  3388 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:19.613955  3388 consensus_queue.cc:1059] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:19.615063  3388 raft_consensus.cc:3055] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:19.615447  3388 raft_consensus.cc:740] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:19.566733  3273 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.616469  3388 consensus_queue.cc:260] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.605306 28181 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 14 to 15, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 15 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:19.647951 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 78070.7us
I20260709 12:20:19.625422 28181 catalog_manager.cc:5697] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:19.652812 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 88726.9us
W20260709 12:20:19.653796 28954 leader_election.cc:336] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:19.655674 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 338655us
I20260709 12:20:19.562879  3477 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.664620 28361 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:20:19.654294 28954 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:19.670135 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.672904 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.675364 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.677526 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.658011 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:19.684863 28984 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
W20260709 12:20:19.686522 28954 consensus_peers.cc:597] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:19.687004 28984 pending_rounds.cc:85] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Aborting all ops after (but not including) 12
I20260709 12:20:19.548712  3347 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.695395 28255 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: 
I20260709 12:20:19.683205  3496 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:19.687633 28954 leader_election.cc:336] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:19.690783  3188 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:19.689670 28984 pending_rounds.cc:107] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:20:19.646353  3388 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:19.705914 28984 raft_consensus.cc:2889] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:19.657625  3359 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:19.712460 28954 consensus_peers.cc:597] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:19.716120 28954 leader_election.cc:336] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:19.717231 28954 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:19.714146 28255 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:19.717722 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:19.693845  3195 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:19.651609  3363 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:20:19.723294 28255 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:19.688661  3513 raft_consensus.cc:697] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:19.737054  3501 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.739567  3431 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:19.742128 28955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.708511  3470 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:19.709407  3188 raft_consensus.cc:493] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:19.755316  3347 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:19.721982  3374 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:20:19.756282  3389 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.756860  3389 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.750474  3188 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:19.758991  3389 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:19.759858  3513 consensus_queue.cc:237] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.692934  3366 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:20:19.690254  3477 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.733080  3473 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:19.768126 28984 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:19.768847 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:19.717602  3496 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.748208  3426 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:19.798768 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 469195us
I20260709 12:20:19.800294 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:19.800298 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.809505 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:19.815228 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.752103  3501 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.777935  3470 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:19.773156  3163 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:19.782316  3473 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:20:19.836337  3350 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.004s
I20260709 12:20:19.778565  3262 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.070s
I20260709 12:20:19.869153 28987 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:19.780282  3347 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.072s
W20260709 12:20:19.919965 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 267030us
W20260709 12:20:19.781350 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 97478.1us
I20260709 12:20:19.796384  3426 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.933882 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:19.934603 28359 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:19.869639  3163 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:19.960928  3473 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:19.963999  3552 log.cc:927] Time spent T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:20:19.966039  3426 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:19.976316 28332 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:19.980970  3308 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:19.967217  3520 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:20:19.968669  3366 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.984935  3366 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:19.987129  3366 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:19.989151  3170 consensus_meta.cc:287] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.001s
I20260709 12:20:19.990185 28985 raft_consensus.cc:1275] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:20:19.991288  3543 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:20:19.984010  3470 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:19.996994 28991 raft_consensus.cc:1275] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 12 index: 14. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
I20260709 12:20:19.954854  3557 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:20.011341 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.013245  3501 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:19.979622  3488 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:20.017372  3555 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.063s	user 0.000s	sys 0.003s
I20260709 12:20:20.035233  3477 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.035966  3488 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.038300  3488 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.026887  3496 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.000162  3195 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.016916  3557 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.065943  3557 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:20.091223 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 290591us
W20260709 12:20:20.093883 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 293483us
I20260709 12:20:20.097261 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:20.100320 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:20:20.104799 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 128434us
I20260709 12:20:20.114148 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.113963 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.116343 28360 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 29 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 28. Current term is 29. Ops: []
I20260709 12:20:20.117812 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.118913 28358 raft_consensus.cc:2410] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:20:20.127046 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:20.129961 28257 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:19.999986 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:20:20.132418 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 132497us
W20260709 12:20:20.139237 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.139550 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:20.148882 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:20.172098 28330 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:20.174625 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.175840 28986 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 14.
I20260709 12:20:20.178596  3163 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.048907  3188 raft_consensus.cc:515] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:20.049582  3195 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.193025  3195 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:20.195158  3126 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:20.195776  3126 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.196511  3262 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:20.184777 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.196277  3126 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:20.207979 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 378736us
I20260709 12:20:20.210541  3262 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:20.211774  3262 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:20.215184  3477 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:20.216487  3477 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:20.195037  3318 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.218537  3318 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.223119  3480 raft_consensus.cc:2749] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:20.223644  3350 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:20.224539  3350 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.225090  3395 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.226122  3367 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.224860  3308 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:20.193065  3188 leader_election.cc:290] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:20.279651 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.274618 28365 raft_consensus.cc:1275] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:20:20.338752 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 180301us
W20260709 12:20:20.344672 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:20.293916  3377 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.004s	sys 0.013s
I20260709 12:20:20.227126  3367 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:20.359123 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 136125us
I20260709 12:20:20.353842  3318 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.292333  3395 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:20.369442  3377 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.006s	sys 0.013s
I20260709 12:20:20.382874 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:20.389509 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.389847  3526 consensus_queue.cc:1048] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:20:20.393862  3461 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:20.394443  3461 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:20.394899  3461 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:20.398109 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 116046us
I20260709 12:20:20.398488 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.399569 28990 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 17.
I20260709 12:20:20.402201 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.405457  3262 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.405293  3350 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.369158  3188 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.436750  3188 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.420604  3367 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.393540  3562 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.271792  3477 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.105s
I20260709 12:20:20.379125  3389 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:20:20.457468  3188 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.458487  3562 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:20.462253  3395 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:20.424911 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 285604us
W20260709 12:20:20.433616 28954 leader_election.cc:336] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:20.434989 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:20.435405 28441 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:20:20.484834 28954 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:20.483618  3562 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.495224 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8850fe0201a48528da0da1cdac707f0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:20.485769  3188 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:20.501760 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:20.507227 28365 raft_consensus.cc:3060] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:20.498967  3562 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.509405  3562 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.510630  3188 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:20.512280  3562 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.513059  3188 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:20.514598 28990 raft_consensus.cc:1275] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:20.515877 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:20.517814  3585 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:20.518344  3585 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.518694  3585 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:20.522881 28988 raft_consensus.cc:1275] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:20:20.527024  3367 consensus_queue.cc:1059] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:20:20.528034  3367 consensus_queue.cc:1243] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer 6634fa3fbd594726a5ec3c1686900284 log is divergent from this leader: its last log entry 29.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
W20260709 12:20:20.542752 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 48009us
I20260709 12:20:20.545094  3367 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:20.549865  3591 consensus_queue.cc:1059] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:20.574119 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:20.575004 28434 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:20.595150 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:20.598544 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.605599 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:20.610898 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.614210 28360 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:20.608927 28988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.619779 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.620528 28440 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:20:20.619961 28988 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 24.
I20260709 12:20:20.576128  3562 consensus_queue.cc:1048] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:20:20.616648 28437 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:20.625483 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:20.620786 28363 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 23.
I20260709 12:20:20.629256 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:20.630926  3562 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:20.631433  3562 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.631824  3562 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:20.632757  3404 raft_consensus.cc:3055] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:20.633239  3404 raft_consensus.cc:740] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:20.633808 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.634480  3404 consensus_queue.cc:260] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:20.635485 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 221605us
I20260709 12:20:20.635948  3359 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:20.637200 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:20.612675  3389 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:20.637925  3477 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.638569  3389 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:20.638710  3477 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:20.631083  3590 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:20:20.641211  3389 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:20.641791 28984 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:20:20.642782  3561 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:20.643285  3561 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.643678  3561 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:20.635316  3464 consensus_queue.cc:1048] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.056s
I20260709 12:20:20.658623  3392 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:20:20.659608 28434 raft_consensus.cc:1275] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:20.664587  3461 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.669970  3513 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:20.710331  3513 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.710856  3513 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:20.712212  3461 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:20.627383  3576 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.081s	user 0.001s	sys 0.001s
W20260709 12:20:20.675927 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 60843.4us
I20260709 12:20:20.684491 28433 raft_consensus.cc:1240] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Rejecting Update request from peer 6634fa3fbd594726a5ec3c1686900284 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:20:20.715513  3585 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.717032  3558 consensus_queue.cc:1048] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:20.641167  3477 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.718140  3350 consensus_queue.cc:1048] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:20.693622  3347 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.697701  3404 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:20.719780  3347 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.694325  3367 consensus_queue.cc:1048] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:20.699848 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.700982 28360 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:20.702473 28366 raft_consensus.cc:1217] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 29 LEADER]: Deduplicated request from leader. Original: 25.19->[28.20-28.20]   Dedup: 28.20->[]
I20260709 12:20:20.637341  3126 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:20.717952 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 71137us
I20260709 12:20:20.724059 28365 raft_consensus.cc:2468] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 21.
I20260709 12:20:20.726433 28954 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:20.725980 28366 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 29 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
W20260709 12:20:20.726716 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 120514us
I20260709 12:20:20.710446  3559 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:20.751353 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64101.6us
I20260709 12:20:20.728317  3585 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.771878 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:20.776364  3588 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.769932  3590 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:20.777163  3588 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.778700 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:20.789897 28440 raft_consensus.cc:2368] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 21. Current term is 22.
W20260709 12:20:20.794734 28330 leader_election.cc:336] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.654s (SENT)
I20260709 12:20:20.800964  3350 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:20.801134 28330 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:20.802064 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
W20260709 12:20:20.800302 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.754s (SENT)
I20260709 12:20:20.803534 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:20.800330  3165 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:20.804334  3431 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:20:20.805114 28404 leader_election.cc:336] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.851s (SENT)
I20260709 12:20:20.805011  3431 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:20.807089 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:20.808264 28404 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.897s (SENT)
W20260709 12:20:20.809543 28404 leader_election.cc:336] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.910s (SENT)
I20260709 12:20:20.811316 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:20.811941  3587 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:20.806694  3350 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.812956 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:20.817550  3350 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.819478 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:20.823199 28434 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
I20260709 12:20:20.809427  3431 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.763824  3559 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:20.822815 28404 leader_election.cc:304] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:20.825353  3126 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.830147  3585 raft_consensus.cc:2804] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:20.827399  3591 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:20.832223  3374 raft_consensus.cc:3055] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:20:20.832683  3374 raft_consensus.cc:740] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:20:20.832854 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44437.4us
I20260709 12:20:20.833845  3374 consensus_queue.cc:260] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.834964  3374 raft_consensus.cc:3060] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:20.835445  3588 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:20.825785  3389 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:20.836265  3389 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:20.836709  3389 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:20.837847  3583 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.838515  3583 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:20.841725  3600 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:20:20.845885  3548 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:20.850136  3592 log.cc:927] Time spent T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.176s	user 0.006s	sys 0.000s
I20260709 12:20:20.852598  3126 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.853325  3126 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.854103  3585 raft_consensus.cc:697] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:20.855269  3585 consensus_queue.cc:237] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:20.855791  3126 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.856894 28987 raft_consensus.cc:1217] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Deduplicated request from leader. Original: 25.19->[28.20-28.20]   Dedup: 28.20->[]
I20260709 12:20:20.857410 28987 raft_consensus.cc:1240] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:20:20.781055  3590 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:20.858376 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 99467.5us
I20260709 12:20:20.859992 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:20.860154  3590 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.860628  3548 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.861304  3548 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.862162 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:20.863862  3548 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.864643  3521 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:20.865069  3521 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:20.865427  3521 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:20.861589  3347 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:20.861081  3513 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.887997 28332 leader_election.cc:400] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 21. Current term is 22.
I20260709 12:20:20.890269 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.892654 28332 leader_election.cc:403] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:20:20.893152 28986 raft_consensus.cc:2410] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:20:20.897315 28983 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:20.840888  3583 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:20.906787 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 78396.4us
I20260709 12:20:20.918066  3417 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:20.918442  3588 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.918902  3586 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:20.919071  3588 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:20.905444 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.934924 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:20.935727 28434 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 15.
I20260709 12:20:20.905433  3359 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:20.945562 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:20.946408 28358 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:20:20.947566  3563 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:20.948108  3563 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.948251  3588 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:20.949563 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:20.950964 28434 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:20:20.951292  3559 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:20.953114 28954 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:20.953655 28954 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:20.966239 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:20.966621 28954 leader_election.cc:336] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:20.967193 28954 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:20.973512 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.974581 28984 raft_consensus.cc:2410] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 for term 16 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 12:20:20.970820  3431 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:20.977027  3163 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:20.979717  3562 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:20.981935 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:20.948489  3563 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:20.985030 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:20.942961  3347 raft_consensus.cc:493] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.987491 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:20.962075  3588 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:20.994843 28954 leader_election.cc:336] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:20.999236 28404 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.066s (SENT)
I20260709 12:20:21.007449 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:20.983517  3559 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.008440 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:20.997272  3464 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:21.018621  3464 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:21.019101  3464 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:21.020283  3587 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:21.070262 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:21.071044 28256 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.031020  3588 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:21.079582 28954 leader_election.cc:336] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:21.080760 28954 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:21.081136 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:20.994692 28984 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:20:21.088183 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 89009.6us
I20260709 12:20:21.056419  3548 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:21.092080 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:20:21.089218 28954 leader_election.cc:336] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:21.101586 28434 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.098023  3470 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:21.105254  3605 log.cc:927] Time spent T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:20:21.050674  3559 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.040813  3561 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.087932  3588 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:21.088344  3566 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.121454 28434 raft_consensus.cc:2468] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:21.122470  3295 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:21.132879  3521 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.135824  3470 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.083447  3562 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:21.074924  3347 raft_consensus.cc:515] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.143900 28332 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:21.153277 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:21.157653  3566 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.159670  3583 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:21.160230  3566 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:21.167690 28956 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:21.167698  3295 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.170264  3586 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:21.170725  3586 raft_consensus.cc:493] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.171080  3586 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:21.171700  3295 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:21.154299  3548 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.176350  3548 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:21.177475  3611 log.cc:927] Time spent T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:20:21.178056 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:21.178918  3513 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:21.179390  3561 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:21.180758  3347 leader_election.cc:290] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:21.184078 28333 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.154210  3165 raft_consensus.cc:2804] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:21.190634 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:21.195084 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 361228us
I20260709 12:20:21.203958 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:21.187541  3126 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:21.207334  3613 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:20:21.212623  3477 raft_consensus.cc:493] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.214485 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:21.221192  3612 log.cc:927] Time spent T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.225s	user 0.000s	sys 0.002s
I20260709 12:20:21.221427  3590 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.222052  3590 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.222860 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fb143bfa1d44653b638d40442c7e955" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:21.223520 28989 raft_consensus.cc:3060] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:21.227061 28987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:21.227842 28987 raft_consensus.cc:3060] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:20:21.209271  3126 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:21.215756  3359 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.236559  3359 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.237845 28985 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:21.239007  3590 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:21.214233  3165 raft_consensus.cc:697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:21.211522  3561 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:21.210944  3295 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:21.240782 28404 leader_election.cc:336] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.190s (SENT)
W20260709 12:20:21.250252 28330 leader_election.cc:336] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.652s (SENT)
I20260709 12:20:21.253051 28989 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 22.
I20260709 12:20:21.256284 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:21.257859 28404 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:21.261592  3165 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.270107  3295 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.241215  3126 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:21.259181 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.219655  3477 raft_consensus.cc:515] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.274371  3561 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:21.275982 28330 leader_election.cc:336] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 1.817s (SENT)
I20260709 12:20:21.278731 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:21.279626  3513 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:20:21.281525  3616 log.cc:927] Time spent T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:20:21.287518 28364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.293599 28987 raft_consensus.cc:2468] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:20:21.260571 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.318998 28986 raft_consensus.cc:2393] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 20: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:21.320035 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.321635 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.317685 28366 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 18.
I20260709 12:20:21.325093 28330 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:21.316597 28364 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:21.326615 28363 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:21.315158  3585 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.334672  3295 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:21.333879  3359 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:21.338260  3558 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:21.347219  3431 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.293776  3586 raft_consensus.cc:515] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:21.406260 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 130349us
I20260709 12:20:21.414002 28983 raft_consensus.cc:2393] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 19: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:21.417559 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:21.418478 28991 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:21.431311  3586 leader_election.cc:290] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:21.434068 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 203840us
I20260709 12:20:21.434751  3477 leader_election.cc:290] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:21.435109 28404 leader_election.cc:336] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Timed out: RequestConsensusVote RPC to 127.26.123.1:42113 timed out after 2.488s (SENT)
I20260709 12:20:21.412724  3431 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.445899  3431 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:21.435616 28404 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:20:21.468096 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 51142.6us
I20260709 12:20:21.362979  3585 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.422557  3557 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:21.479956  3559 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:21.484489  3295 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:21.522264 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 87200.6us
I20260709 12:20:21.514635  3188 raft_consensus.cc:2749] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:21.530223 28991 raft_consensus.cc:2468] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 25.
W20260709 12:20:21.525662  3563 consensus_meta.cc:287] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.001s
I20260709 12:20:21.532217  3563 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:21.537563  3374 consensus_meta.cc:287] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.702s	user 0.016s	sys 0.000s
I20260709 12:20:21.538466  3561 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:21.542676  3163 consensus_meta.cc:287] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.000s
W20260709 12:20:21.544708  3464 consensus_meta.cc:287] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.525s	user 0.017s	sys 0.008s
W20260709 12:20:21.545701  3417 consensus_meta.cc:287] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.001s
W20260709 12:20:21.551138  3389 consensus_meta.cc:287] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: Time spent flushing consensus metadata: real 0.714s	user 0.004s	sys 0.015s
I20260709 12:20:21.546200  3417 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721) with higher term. Message: Invalid argument: T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 for earlier term 21. Current term is 22.
I20260709 12:20:21.525849 28181 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43788 (request call id 319) took 4474 ms. Trace:
I20260709 12:20:21.555984 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:21.562811 28985 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:21.566071 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.566848 28367 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:21.567619  3374 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.568260 28954 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:21.562357 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:21.570007  3586 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:21.570351  3586 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.570604  3188 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:21.571086  3188 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:21.571493  3188 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:21.574375  3585 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:21.570673  3586 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.568270  3374 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.570664 28404 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:21.593704 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 26 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:21.600502 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.517724  3295 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.606338 28367 raft_consensus.cc:1275] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:21.610177 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.606030 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.560508  3389 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.586805  3585 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:21.555228  3464 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.620222  3295 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:21.621454  3521 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:21.634732  3620 log.cc:927] Time spent T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: Append to log took a long time: real 0.116s	user 0.002s	sys 0.001s
I20260709 12:20:21.635237 28434 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:21.640455  3618 log.cc:927] Time spent T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:20:21.634874  3389 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:21.642970  3470 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:21.644748  3619 log.cc:927] Time spent T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:20:21.631556 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986f172991d84af28d4f5cbe8c517e6e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.645572 28434 raft_consensus.cc:2468] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:21.612039  3374 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:21.654974 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85a68e4142444e5cb52829e957c1abf7" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.656275 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.660274 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:21.615231  3585 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:21.671510 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:21.677489 28330 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:21.684528 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cb896260dae4cf6bb09ba7728c603f3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.685520 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5bac9d516d84373a10b8812e66d31ed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.689263 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.684886 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.700356 28367 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
I20260709 12:20:21.671044  3585 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:21.707422 28360 raft_consensus.cc:2468] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 22.
I20260709 12:20:21.698940 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:21.724635 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:21.726898  3477 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:21.693323  3470 consensus_queue.cc:1059] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:20:21.706892  3163 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:36572 (request call id 10) took 20368 ms. Trace:
I20260709 12:20:21.736361 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.752604  3170 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:21.752887  3470 raft_consensus.cc:3055] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:21.753281  3470 raft_consensus.cc:740] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
W20260709 12:20:21.758785 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 143127us
I20260709 12:20:21.733630  3563 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:21.764567 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea7beef1d4c49159d75e5f61eaaeb5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:20:21.746615  3627 log.cc:927] Time spent T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:20:21.765398 28440 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:21.761644 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 96616.5us
I20260709 12:20:21.765828 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.749652  3163 rpcz_store.cc:276] 0709 12:20:01.302365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:01.302505 (+   140us) service_pool.cc:224] Handling call
0709 12:20:01.539201 (+236696us) spinlock_profiling.cc:243] Waited 66.2 ms on lock 0x7b18000513c0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be8c273d 00007f85be88df3f 00007f85be88d6b3 00007f85be8859fa 00007f85c384d4c9 00007f85bcdc9c54 00007f85bcdc9be2 00007f85bcdc9b11 00007f85bcdc9a8c 00007f85bcdc8d42
0709 12:20:01.731112 (+191911us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b580089a2d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7e30aa 00007f85be865447 00007f85be88270a 00007f85be88376a 00007f85be88291b 00007f85be88e03c 00007f85be88d6b3 00007f85be8859fa 00007f85c384d4c9 00007f85bcdc9c54
0709 12:20:01.876024 (+144912us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b58004f07d0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be7e30aa 00007f85be865447 00007f85be88270a 00007f85be88376a 00007f85be88291b 00007f85be88e03c 00007f85be88d6b3 00007f85be8859fa 00007f85c384d4c9 00007f85bcdc9c54
0709 12:20:01.922204 (+ 46180us) log.cc:844] Serialized 494 byte log entry
0709 12:20:21.661043 (+19738839us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41,"raft.queue_time_us":3387,"raft.run_cpu_time_us":1317,"raft.run_wall_time_us":1310,"spinlock_wait_cycles":663695232,"thread_start_us":3257364,"threads_started":2,"wal-append.queue_time_us":557772,"wal-append.run_cpu_time_us":3747,"wal-append.run_wall_time_us":1315892}
I20260709 12:20:21.788139 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:21.788949 28991 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 25: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:21.747964  3347 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:21.797583 28404 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:21.753490  3170 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.795202  3126 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.805563  3624 raft_consensus.cc:2804] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:21.806246  3624 raft_consensus.cc:697] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 22 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:21.807539  3126 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:21.807256  3624 consensus_queue.cc:237] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.814160  3464 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:21.816591 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:21.817369 28982 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:21.556860 28181 rpcz_store.cc:276] 0709 12:20:16.949423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.950265 (+   842us) service_pool.cc:224] Handling call
0709 12:20:21.423757 (+4473492us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:19.841288 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:19.978963 (+137675us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:19.981809 (+  2846us) write_op.cc:435] Acquired schema lock
0709 12:20:19.997927 (+ 16118us) tablet.cc:662] Decoding operations
0709 12:20:20.014658 (+ 16731us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:20.014719 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:20.014746 (+    27us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:20:20.015077 (+   331us) tablet.cc:701] Row locks acquired
0709 12:20:20.015097 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:20:20.015260 (+   163us) write_op.cc:270] Start()
0709 12:20:20.015379 (+   119us) write_op.cc:276] Timestamp: P: 1783599620015208 usec, L: 0
0709 12:20:20.015400 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:20:20.016149 (+   749us) log.cc:844] Serialized 1298 byte log entry
0709 12:20:20.444803 (+428654us) op_driver.cc:464] REPLICATION: finished
0709 12:20:20.663060 (+218257us) write_op.cc:301] APPLY: starting
0709 12:20:20.663189 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:20.700710 (+ 37521us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:20.700852 (+   142us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:20.703160 (+  2308us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:20.703515 (+   355us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 12:20:20.703583 (+    68us) write_op.cc:312] APPLY: finished
0709 12:20:20.703973 (+   390us) log.cc:844] Serialized 41 byte log entry
0709 12:20:20.704237 (+   264us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:20.704435 (+   198us) write_op.cc:454] Released schema lock
0709 12:20:20.704523 (+    88us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:20.705318 (+   795us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:20.705350 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1183580,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":217822,"apply.run_cpu_time_us":8180,"apply.run_wall_time_us":42222,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16661,"num_ops":3,"prepare.queue_time_us":119882,"prepare.run_cpu_time_us":9396,"prepare.run_wall_time_us":220017,"raft.queue_time_us":380438,"raft.run_cpu_time_us":26397,"raft.run_wall_time_us":175456,"replication_time_us":429031,"thread_start_us":632226,"threads_started":4,"wal-append.queue_time_us":43577,"wal-append.run_cpu_time_us":19365,"wal-append.run_wall_time_us":1649812}]]}
I20260709 12:20:21.800313 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:21.816498 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6d72f1fb474adfafdd159090dc63b5" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.829797 28363 raft_consensus.cc:2393] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 25: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:21.828823 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:21.830539 28433 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:21.841068 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54211b8ac4fe44c59e9f0338d9960177" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.832741  3521 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:20:21.844973 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 47468.2us
I20260709 12:20:21.848680 28404 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:21.849831  3557 raft_consensus.cc:2804] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:21.850446  3557 raft_consensus.cc:697] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:21.851454  3557 consensus_queue.cc:237] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.857082 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.857365 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:21.858228 28363 raft_consensus.cc:2410] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:20:21.863973 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:21.874198 28435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23a4c78997e24df7bce95fcf6d70c95c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.878103 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.878935 28441 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 18.
I20260709 12:20:21.880611  3548 raft_consensus.cc:493] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:21.881297  3548 raft_consensus.cc:515] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.883486  3548 leader_election.cc:290] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:21.885848 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 208406us
I20260709 12:20:21.888219 28363 raft_consensus.cc:2368] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
W20260709 12:20:21.890358 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 108140us
I20260709 12:20:21.891634 28982 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
I20260709 12:20:21.893074 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:21.894618 28255 leader_election.cc:304] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:21.899564 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:21.900629 28363 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 16: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:21.906288  3548 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:21.907318  3548 raft_consensus.cc:2749] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:21.907845 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d8e709ec114a04bfd86247e24dd63c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.908680 28434 raft_consensus.cc:2393] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 21: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:21.914362 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72871f9ecbeb417fa5bd39d4714f0a90" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.914768 28433 raft_consensus.cc:2468] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 23.
I20260709 12:20:21.915534 28435 raft_consensus.cc:1240] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:21.845772  3521 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.918968 28255 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:21.924180 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:21.924386 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b6d26dd064626ab3b950e5a6c272f" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:20:21.925020 28330 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:21.925793 28404 consensus_peers.cc:597] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:21.926740 27116 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 28317)
I20260709 12:20:21.932026  3586 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.934113  3548 raft_consensus.cc:3060] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:21.934470  3586 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:21.868569  3470 consensus_queue.cc:260] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:21.938710  3470 raft_consensus.cc:3060] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:21.921656  3521 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:21.946966 28440 raft_consensus.cc:2468] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:21.930099 28255 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:21.886386  3188 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:21.885706  3585 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:21.951707 28404 leader_election.cc:336] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:21.952138 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448f610d25494d2c875c5042ec162f71" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.952293 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:21.953128 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d6b97dd9e9466d9581f6088e681a11" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.955637 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f380821c5264bfc8a5f0bf36bc52fd4" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:21.959748 28434 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:20:21.964479 28437 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:21.960352 28434 pending_rounds.cc:85] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Aborting all ops after (but not including) 11
I20260709 12:20:21.968034 28440 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:21.970341 28439 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:21.966454 28434 pending_rounds.cc:107] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:20:21.974292 28434 raft_consensus.cc:2889] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:21.971951  3557 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:21.986052 28438 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:21.990190  3188 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:21.995023  3585 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.013881  3635 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:22.014719  3635 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Becoming Leader. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:22.015833  3635 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.017889 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916dfa76a5254f63bfc746dbe9c5fde0" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.033932 28436 raft_consensus.cc:2393] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:22.016604 28956 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:22.035475  3588 raft_consensus.cc:2804] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:22.036283  3588 raft_consensus.cc:697] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 23 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:20:22.023602 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 92951.5us
W20260709 12:20:22.037442 28404 consensus_peers.cc:597] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:22.037389  3588 consensus_queue.cc:237] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:22.038334 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:22.038980 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:22.039618 28404 consensus_peers.cc:597] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:22.043476 28956 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:22.043999 28404 consensus_peers.cc:597] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:22.054288  3431 raft_consensus.cc:2804] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:22.062363 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:22.066265 28440 raft_consensus.cc:2393] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 16: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
I20260709 12:20:22.062222 28189 catalog_manager.cc:5697] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 17 to 18, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 18 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:22.071852 28189 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:22.080154 28189 catalog_manager.cc:5697] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 20 to 21, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 21 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:22.044719 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:22.092229 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47569.1us
I20260709 12:20:22.105357 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255e0140f2a64433a6c2dd64fe24e6b3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.124794 28330 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:22.131595 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:22.136101 28189 catalog_manager.cc:5697] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 11 to 14, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 14 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.140060 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:22.140870 28440 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.142266 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:22.143141 28360 raft_consensus.cc:3060] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.113194  3513 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.138375 28332 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:22.169171  3513 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.183473  3513 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.184048  3513 raft_consensus.cc:2804] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:22.184517  3513 raft_consensus.cc:697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
W20260709 12:20:22.189932 28404 leader_election.cc:336] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:22.193396 28360 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
W20260709 12:20:22.216153 28257 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.219503 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:22.189993 28189 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 13 to 15, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 15 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:20:22.221647 28189 catalog_manager.cc:5697] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 17 to 21, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 21 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:22.223542 28189 catalog_manager.cc:5697] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 18 to 21, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 21 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:22.162657  3374 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:22.228839 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 51681.8us
I20260709 12:20:22.225196 28189 catalog_manager.cc:5697] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:22.217301  3417 raft_consensus.cc:2764] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:20:22.226152  3563 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:22.242647 28189 catalog_manager.cc:5697] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
W20260709 12:20:22.240857 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 126265us
I20260709 12:20:22.248658 28440 raft_consensus.cc:2468] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:20:22.229068  3634 raft_consensus.cc:493] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:20:22.250617 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 78860.6us
I20260709 12:20:22.230567  3163 raft_consensus.cc:493] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:22.250476 28189 catalog_manager.cc:5697] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:22.234355  3557 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:22.235599  3470 raft_consensus.cc:493] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:22.185449  3513 consensus_queue.cc:237] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:22.254876 28404 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:22.260984 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 97648.2us
I20260709 12:20:22.235433  3431 raft_consensus.cc:697] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:20:22.259992 28954 leader_election.cc:336] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:22.233978  3374 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:22.248706 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 114499us
W20260709 12:20:22.310808 28255 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 57406.3us
I20260709 12:20:22.285550  3520 raft_consensus.cc:2804] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:22.317859 28189 catalog_manager.cc:5697] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:22.310405 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:22.262012  3563 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:22.264815  3470 raft_consensus.cc:515] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.282956  3163 raft_consensus.cc:515] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:22.316922  3477 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.262511  3634 raft_consensus.cc:515] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:22.362380 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 70895.8us
W20260709 12:20:22.365047 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 113667us
W20260709 12:20:22.366945 28404 leader_election.cc:336] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:22.367446 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:20:22.369364 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 68849.9us
I20260709 12:20:22.369266 28189 catalog_manager.cc:5697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:22.339792  3643 log.cc:927] Time spent T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:20:22.334780  3520 raft_consensus.cc:697] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:22.394142 28189 catalog_manager.cc:5697] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 16 to 17, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 17 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:20:22.380945  3585 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:22.400734 28189 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:22.343951  3521 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.408913  3163 leader_election.cc:290] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.410445  3557 consensus_queue.cc:1048] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:22.412314  3637 raft_consensus.cc:493] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:22.378515  3477 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:22.412943  3637 raft_consensus.cc:515] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:22.381395  3431 consensus_queue.cc:237] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.440774 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:22.435197 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.435942 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.433530 28360 raft_consensus.cc:1240] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:22.422221 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:22.418286 28189 catalog_manager.cc:5697] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 21 to 22, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 22 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:22.435632 28362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.446872 28990 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:22.452515 28954 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:22.453147 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.453418  3651 raft_consensus.cc:2804] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:22.453843 28433 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:22.454175  3651 raft_consensus.cc:697] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 21 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:22.455375  3651 consensus_queue.cc:237] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.413681  3374 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.439726  3655 raft_consensus.cc:493] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:22.463243  3563 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.425267  3477 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.415144  3637 leader_election.cc:290] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.423630  3634 leader_election.cc:290] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.416568  3470 leader_election.cc:290] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.416453  3548 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:22.428292  3170 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.466444 28330 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:22.468173  3659 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:22.468840 27116 tablet_replica.cc:333] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:22.468811  3659 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.470073  3520 consensus_queue.cc:237] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.473418 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:22.467428 28189 catalog_manager.cc:5697] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 20 to 22, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 22 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:22.425426  3521 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:22.471570  3659 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.463296  3655 raft_consensus.cc:515] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.467664  3464 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:22.478304  3464 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:22.478727  3464 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:22.485251 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 85662.3us
I20260709 12:20:22.482698 28189 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 15 to 19, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 19 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.481837  3655 leader_election.cc:290] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.488773 27116 raft_consensus.cc:2243] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:20:22.482193  3347 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.490518  3417 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:22.490374  3347 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.491086  3417 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.505400  3655 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:22.506174  3417 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.506363 28404 leader_election.cc:400] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:20:22.506847 28404 leader_election.cc:403] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:20:22.507402 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:22.508074 28982 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:22.508344  3559 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20.
I20260709 12:20:22.510010 28330 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:22.511415 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:22.512887 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:22.513644 28363 raft_consensus.cc:2368] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 20. Current term is 21.
I20260709 12:20:22.528362 28367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.528601 28365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.528865 28358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.530117 28366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.532433 27116 raft_consensus.cc:2272] T 39dfaebe2a90498f928f7cd6cc499fac P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:22.536203 27116 tablet_replica.cc:333] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:22.537319 27116 raft_consensus.cc:2243] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:22.541463 28990 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:22.543449  3657 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:20:22.545706 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 423954us
I20260709 12:20:22.547269 27116 raft_consensus.cc:2272] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:22.554378 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.559994 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.565243 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:22.567485 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.563977 28189 catalog_manager.cc:5697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 16 to 18, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 18 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:20:22.576828  3431 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.577288  3657 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.578307  3431 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.578459  3657 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.580978  3657 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.577612  3662 raft_consensus.cc:493] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:22.583299 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:22.586208 28984 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 23: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:22.570814 27116 tablet_replica.cc:333] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:22.564424  3417 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:22.592517 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.593329 28359 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 22.
I20260709 12:20:22.590752  3417 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:22.603325  3417 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:22.607156  3648 log.cc:927] Time spent T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.196s	user 0.002s	sys 0.002s
I20260709 12:20:22.506131  3655 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.617527  3347 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.618702 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:22.619376 28359 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:22.619469 28432 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:22.622219 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956fb13fa38243fd862789f3046aba97" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.629495 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8; no voters: f526369c0d0743b192be6e205540a059
I20260709 12:20:22.633461  3657 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:22.634028  3657 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:22.634438  3657 raft_consensus.cc:3060] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:22.635205 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.636658  3635 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.637259  3635 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.639729  3635 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.641983 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.646900 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f7ed53101a4cc3b4e19149d5a4a1a8" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:22.658087 27116 raft_consensus.cc:2243] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:22.664229 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d68008d2b3fd4f7bb72bf69f612b2998" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:22.662855 28359 raft_consensus.cc:2468] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 23.
I20260709 12:20:22.661406 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:22.666962 28954 leader_election.cc:336] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:22.668082 28954 leader_election.cc:336] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:22.671244 28954 leader_election.cc:336] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:22.675840 28434 raft_consensus.cc:1275] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:20:22.661648  3646 log.cc:927] Time spent T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: Append to log took a long time: real 0.252s	user 0.010s	sys 0.010s
I20260709 12:20:22.678126 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:22.679217 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.675426 28954 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:22.681521 28432 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:22.691293 27116 raft_consensus.cc:2272] T d8850fe0201a48528da0da1cdac707f0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:22.692039 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "705e15df427147839fee08c92bc268dc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.693410  3431 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.693585 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "014b394597e7430489bd093d8616dcad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.696074  3655 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.662636  3662 raft_consensus.cc:515] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:22.677600  3654 log.cc:927] Time spent T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: Append to log took a long time: real 0.264s	user 0.009s	sys 0.000s
I20260709 12:20:22.709468 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2edb513f7343d2a2463b94d992b7a3" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.703985 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.706791 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.711117 28436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c99f01ab7374acfa37f7669b8fcdeb2" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.715658 28432 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3de34b6f4cf4606a61944db6e2739c8 in term 19.
W20260709 12:20:22.715688 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 46540.1us
W20260709 12:20:22.722124 28954 leader_election.cc:336] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:22.722513 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "582bd3fd6d214e92aefb0baa58c76aed" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.724707 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0d1d6f8a6a04f6ea4e6f31327e92009" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:22.723578  3431 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:22.729410  3662 leader_election.cc:290] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.680642  3513 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:20:22.732460  3638 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:22.724973 28174 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bce32c3c49414e228fe6d912d95aeaac with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:20:22.735663 28174 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 73605.4us
I20260709 12:20:22.740643 28361 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:22.744043 28984 raft_consensus.cc:1275] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:22.745687 28434 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 24.
I20260709 12:20:22.747517 28432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "610cd41ac66c4074be8e2aa7b22d5e6c" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:20:22.748091 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.739158  3417 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.752163 28404 leader_election.cc:400] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 20. Current term is 21.
I20260709 12:20:22.752696 28404 leader_election.cc:403] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 12:20:22.744603 28954 leader_election.cc:336] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:22.754024 27116 tablet_replica.cc:333] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:22.761204 28434 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer a3de34b6f4cf4606a61944db6e2739c8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:22.766780 28255 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:22.768167 28255 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:22.765175  3557 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701) with higher term. Message: Invalid argument: T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b for earlier term 20. Current term is 21.
I20260709 12:20:22.773592 28956 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:22.777377  3417 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.780958  3651 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:22.782780  3635 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:20:22.785323  3662 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:22.789148 28954 leader_election.cc:304] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:22.792088  3464 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.793015  3657 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.803967  3663 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:22.805466  3657 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.806159 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:22.806978 28359 raft_consensus.cc:2393] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a3de34b6f4cf4606a61944db6e2739c8 in current term 23: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:22.781518  3651 raft_consensus.cc:493] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:22.807634  3651 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:22.707099  3664 consensus_queue.cc:1048] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.081s
I20260709 12:20:22.812157 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:22.794595  3464 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.819294 27116 raft_consensus.cc:2243] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:22.837594 27116 raft_consensus.cc:2272] T 97c4af01d02243b2b0727baa5bbacf44 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:22.836273 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:20:22.825205 28330 leader_election.cc:336] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:22.838606 28434 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.830442  3635 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.829229  3669 consensus_queue.cc:1048] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:20:22.843812 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 54782.8us
I20260709 12:20:22.846696  3635 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.838630 28330 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:22.846462  3664 raft_consensus.cc:2749] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:22.860001 28954 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:22.867867 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.869665 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:22.870290  3625 raft_consensus.cc:2804] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:22.874203  3464 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:22.875917  3625 raft_consensus.cc:697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 23 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:22.875341 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:22.877048  3625 consensus_queue.cc:237] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.888732 28984 raft_consensus.cc:1275] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 12:20:22.890146 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 64981.7us
I20260709 12:20:22.894778 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:22.900345  3635 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:22.907006 28330 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:22.911342 28330 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:22.912585 28330 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:22.913620 28330 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:22.906183 28434 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:22.932595 28257 leader_election.cc:304] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:22.934712 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "114fa146124747358c997ac1008e1b93" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:22.935390 28985 raft_consensus.cc:1240] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:22.934190 28332 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:22.939882  3657 raft_consensus.cc:2749] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:22.940855  3432 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:22.941438  3432 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:22.942361  3432 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:22.946465 28954 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:22.946887 28954 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:22.898577  3638 consensus_queue.cc:1048] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
W20260709 12:20:22.949931 28954 consensus_peers.cc:597] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:22.944605 28255 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:22.952797 28330 consensus_peers.cc:597] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:22.958324  3662 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:22.959090  3563 consensus_queue.cc:1059] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:22.930114  3651 raft_consensus.cc:515] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.961373  3563 consensus_queue.cc:1243] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer f526369c0d0743b192be6e205540a059 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:22.962013  3563 raft_consensus.cc:3055] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:22.962360  3563 raft_consensus.cc:740] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:22.963231  3563 consensus_queue.cc:260] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.964498  3563 raft_consensus.cc:3060] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:22.957706 28255 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:22.966727  3662 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:22.978264  3662 raft_consensus.cc:493] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.978849  3662 raft_consensus.cc:515] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:22.978869  3651 leader_election.cc:290] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:22.966257 28982 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:22.983928  3662 leader_election.cc:290] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:22.989638 27116 tablet_replica.cc:333] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:22.990734 27116 raft_consensus.cc:2243] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:20:22.991525 27116 pending_rounds.cc:70] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Trying to abort 1 pending ops.
I20260709 12:20:22.991779 27116 pending_rounds.cc:77] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8: Aborting op as it isn't in flight: id { term: 24 index: 17 } timestamp: 7305624043442900992 op_type: NO_OP noop_request { }
I20260709 12:20:22.992170 27116 raft_consensus.cc:2889] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:20:22.992504 27116 raft_consensus.cc:2272] T 5235a8de06744285b8c3fe8f9f857223 P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:22.997364 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5235a8de06744285b8c3fe8f9f857223" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:22.998142 28433 raft_consensus.cc:3060] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:22.955927 28330 consensus_peers.cc:597] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:22.999699 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 43841.6us
I20260709 12:20:22.950734  3664 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:23.000356 28330 consensus_peers.cc:597] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.010300 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:23.010979 28330 consensus_peers.cc:597] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.014724 28954 consensus_peers.cc:597] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.018843 28330 consensus_peers.cc:597] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.032990 28434 raft_consensus.cc:1275] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:23.040243 28434 raft_consensus.cc:1275] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:23.051403 28360 raft_consensus.cc:1240] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Rejecting Update request from peer 9aea2391841b4888b4e195e5a1f4623b for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:23.040081 28330 consensus_peers.cc:597] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.063472 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 136672us
I20260709 12:20:23.069666 28434 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:23.045868  3625 consensus_queue.cc:1048] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:20:23.054719  3188 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:23.073803 28257 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.068710  3651 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:23.050750  3558 consensus_queue.cc:1048] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:20:23.066859  3669 consensus_queue.cc:1059] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:20:23.097349  3651 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:23.109388 28360 raft_consensus.cc:1275] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:23.105698 27116 tablet_replica.cc:333] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.088790  3431 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:20:23.127532  3651 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:23.107549  3374 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.136648  3588 consensus_queue.cc:1048] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:23.098004  3188 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:23.139627  3188 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:23.141328 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 76905.1us
I20260709 12:20:23.142388 28257 leader_election.cc:304] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.143870 28257 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.144501  3374 raft_consensus.cc:2749] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.154875  3638 raft_consensus.cc:3055] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:20:23.158133 28954 leader_election.cc:336] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.169929 27116 raft_consensus.cc:2243] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:23.067083 28330 consensus_peers.cc:597] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.190802 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 124482us
W20260709 12:20:23.200588 28330 consensus_peers.cc:597] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.203112 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:23.208503 28330 leader_election.cc:336] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.209568 27116 raft_consensus.cc:2272] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:23.190164  3638 raft_consensus.cc:740] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:23.209986 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:23.217564 28433 raft_consensus.cc:2468] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 25.
W20260709 12:20:23.226119 28954 leader_election.cc:336] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:23.236007 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.237162 28956 leader_election.cc:304] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:23.242563 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 98774.5us
I20260709 12:20:23.247571 28257 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.211817  3563 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:23.248813 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 165976us
W20260709 12:20:23.250986 28330 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.249361  3464 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.248634  3431 raft_consensus.cc:2804] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:20:23.263063 28404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.262995 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:23.268083 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:23.263569 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:23.277716 28985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:23.282311 28985 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.275695  3374 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:23.287565 28983 raft_consensus.cc:1275] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:20:23.289209 28255 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.280594  3638 consensus_queue.cc:260] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:23.276701  3625 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:23.301399 28985 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:23.309499 27116 tablet_replica.cc:333] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.315546 28257 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.316694  3374 raft_consensus.cc:3060] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.326232  3374 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.327569 28257 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.328454 27116 raft_consensus.cc:2243] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:23.335328 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 65928.4us
I20260709 12:20:23.368741 28330 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.393029  3558 raft_consensus.cc:2749] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:23.393790 27116 raft_consensus.cc:2272] T d68008d2b3fd4f7bb72bf69f612b2998 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:23.395864  3432 consensus_queue.cc:1048] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:23.397174 27116 tablet_replica.cc:333] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.376868  3638 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:23.391297  3431 raft_consensus.cc:697] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 25 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:20:23.383571 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 112302us
W20260709 12:20:23.387557 28954 consensus_peers.cc:597] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:23.404109 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 76653.6us
I20260709 12:20:23.405023 28257 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.406680 27116 raft_consensus.cc:2243] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:23.406785 28257 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
W20260709 12:20:23.409475 28258 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.418253  3657 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:23.419167  3657 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:23.411073  3662 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.423023 28434 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:23.429827  3464 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:23.423274 27116 raft_consensus.cc:2272] T b5bac9d516d84373a10b8812e66d31ed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:23.436923 28257 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.399819  3431 consensus_queue.cc:237] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:23.446512 28257 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.442112  3374 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:23.453457  3656 consensus_queue.cc:1059] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:23.453755 28257 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.458961  3563 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:23.463495 28257 leader_election.cc:304] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.465261  3635 raft_consensus.cc:3060] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:23.471500 27116 tablet_replica.cc:333] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.474479  3563 raft_consensus.cc:2749] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:23.479563 28257 leader_election.cc:304] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.481240  3374 raft_consensus.cc:2749] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:23.487588 28257 leader_election.cc:304] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.491308  3347 raft_consensus.cc:3055] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:23.509972  3347 raft_consensus.cc:740] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a3de34b6f4cf4606a61944db6e2739c8, State: Running, Role: LEADER
I20260709 12:20:23.511133  3347 consensus_queue.cc:260] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:23.514206  3347 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:23.521522  3563 raft_consensus.cc:2749] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:23.538375 28257 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 50893.9us
I20260709 12:20:23.539631 28257 leader_election.cc:304] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.543646 28257 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.550743 28359 raft_consensus.cc:1275] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:23.533648  3635 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:23.542711  3374 raft_consensus.cc:2749] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:20:23.561836 28257 raft_consensus.cc:2686] T 6b2edb513f7343d2a2463b94d992b7a3 P a3de34b6f4cf4606a61944db6e2739c8: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:20:23.578444 28257 leader_election.cc:304] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.579582  3563 raft_consensus.cc:2749] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:23.579859 28257 leader_election.cc:304] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.580664 27116 raft_consensus.cc:2243] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:23.580812  3635 raft_consensus.cc:2749] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.581264 28257 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
W20260709 12:20:23.581981 28257 raft_consensus.cc:2686] T c0d1d6f8a6a04f6ea4e6f31327e92009 P a3de34b6f4cf4606a61944db6e2739c8: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:20:23.586587  3697 consensus_queue.cc:1048] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
W20260709 12:20:23.595583 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.596346 28954 consensus_peers.cc:597] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:23.596786 28257 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:23.598239  3563 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.601722 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c8442db4dc44c619fd67179aa212063" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:23.600067  3563 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:20:23.612843 28984 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:23.618438  3520 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:23.620400 28439 raft_consensus.cc:1240] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:23.623339 28440 raft_consensus.cc:1240] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Rejecting Update request from peer a3de34b6f4cf4606a61944db6e2739c8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:23.623610 28439 raft_consensus.cc:2955] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:23.620715 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d225f747aa45ff89df6c0ba308faad" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
W20260709 12:20:23.635488 28404 leader_election.cc:336] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.641064 27116 raft_consensus.cc:2272] T 916dfa76a5254f63bfc746dbe9c5fde0 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:23.678395 27116 tablet_replica.cc:333] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.679354 27116 raft_consensus.cc:2243] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:23.680171 27116 raft_consensus.cc:2272] T 448f610d25494d2c875c5042ec162f71 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:23.705677 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 70219.5us
W20260709 12:20:23.707516 28404 consensus_peers.cc:597] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.709512 27116 tablet_replica.cc:333] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.710911 27116 raft_consensus.cc:2243] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:23.714401 27116 pending_rounds.cc:70] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Trying to abort 1 pending ops.
I20260709 12:20:23.714730 27116 pending_rounds.cc:77] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8: Aborting op as it isn't in flight: id { term: 18 index: 13 } timestamp: 7305624020815441920 op_type: NO_OP noop_request { }
I20260709 12:20:23.715500 27116 raft_consensus.cc:2889] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:20:23.716180 27116 raft_consensus.cc:2272] T 705e15df427147839fee08c92bc268dc P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:23.718468  3347 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.721988  3347 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:23.724396  3347 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:23.724689 27116 tablet_replica.cc:333] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.725729 27116 raft_consensus.cc:2243] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:23.726984 27116 raft_consensus.cc:2272] T 775a5161e47944f5ad46f1616a48b2fe P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:23.727133 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:23.729780 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4036b67205e41c597049a10a71e9467" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:23.731492 28255 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:23.732774 28404 consensus_peers.cc:597] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.733165  3347 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:23.742669 28404 leader_election.cc:336] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:23.743759 28404 leader_election.cc:336] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.744171 28404 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.753901  3585 raft_consensus.cc:2749] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:23.754949 28181 catalog_manager.cc:5697] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 17 to 19, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 19 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:23.757836 28330 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:23.767925 28404 leader_election.cc:336] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:23.774788 28404 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:23.775336 28330 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:23.776453 28330 consensus_peers.cc:597] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.777053 28330 consensus_peers.cc:597] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.777510 28181 catalog_manager.cc:5697] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 18 to 19, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1). New cstate: current_term: 19 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:20:23.777649 28330 consensus_peers.cc:597] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.779135 28330 consensus_peers.cc:597] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.779778 28330 consensus_peers.cc:597] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.775302 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.781369 28181 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 22 to 26, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 26 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:20:23.783026 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.783712  3638 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:23.784005 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:23.784970 28404 leader_election.cc:336] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.783218 28181 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 17 to 19, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 19 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
W20260709 12:20:23.781699 28330 consensus_peers.cc:597] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.787364 28330 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:23.788332 28330 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:20:23.789206 28330 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.777993 27116 tablet_replica.cc:333] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.790407 27116 raft_consensus.cc:2243] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:23.791057 27116 raft_consensus.cc:2272] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:23.794574 28330 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.795413 27116 tablet_replica.cc:333] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
W20260709 12:20:23.795627 28330 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.796061 28330 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:23.796103 28404 leader_election.cc:336] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.798888  3558 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.799523 28404 leader_election.cc:304] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.800313 27116 raft_consensus.cc:2243] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:23.800411  3638 raft_consensus.cc:2749] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.801116 27116 raft_consensus.cc:2272] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P a3de34b6f4cf4606a61944db6e2739c8 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:23.802629 28404 leader_election.cc:336] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.808576 28257 leader_election.cc:304] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.809577  3347 raft_consensus.cc:2749] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:23.785081 28189 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43818 (request call id 334) took 2111 ms. Trace:
I20260709 12:20:23.811556 28181 catalog_manager.cc:5697] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 19 to 21, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 21 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:23.816241 28181 catalog_manager.cc:5697] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 25 to 29, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 29 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:23.826447 28181 catalog_manager.cc:5697] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 14 to 16, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 16 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:23.810966 28189 rpcz_store.cc:276] 0709 12:20:21.674106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:21.674556 (+   450us) service_pool.cc:224] Handling call
0709 12:20:23.784983 (+2110427us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:22.952062 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:22.978384 (+ 26322us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:22.978440 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:20:22.978459 (+    19us) tablet.cc:662] Decoding operations
0709 12:20:22.979943 (+  1484us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:22.980073 (+   130us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:22.980098 (+    25us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:20:22.981472 (+  1374us) tablet.cc:701] Row locks acquired
0709 12:20:22.981522 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:20:22.981727 (+   205us) write_op.cc:270] Start()
0709 12:20:23.015688 (+ 33961us) write_op.cc:276] Timestamp: P: 1783599622981657 usec, L: 0
0709 12:20:23.019107 (+  3419us) op_driver.cc:348] REPLICATION: starting
0709 12:20:23.118518 (+ 99411us) log.cc:844] Serialized 7026 byte log entry
0709 12:20:23.320822 (+202304us) op_driver.cc:464] REPLICATION: finished
0709 12:20:23.478824 (+158002us) write_op.cc:301] APPLY: starting
0709 12:20:23.478957 (+   133us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:23.612953 (+133996us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:23.613063 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:23.628335 (+ 15272us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:23.703974 (+ 75639us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 12:20:23.704895 (+   921us) write_op.cc:312] APPLY: finished
0709 12:20:23.730097 (+ 25202us) log.cc:844] Serialized 164 byte log entry
0709 12:20:23.730297 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:23.730495 (+   198us) write_op.cc:454] Released schema lock
0709 12:20:23.730574 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:23.731050 (+   476us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:23.731081 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":157683,"apply.run_cpu_time_us":26524,"apply.run_wall_time_us":252210,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79331,"num_ops":18,"prepare.queue_time_us":140229,"prepare.run_cpu_time_us":31772,"prepare.run_wall_time_us":342665,"raft.queue_time_us":24375,"raft.run_cpu_time_us":24248,"raft.run_wall_time_us":108009,"replication_time_us":301550,"spinlock_wait_cycles":52608,"thread_start_us":329354,"threads_started":4,"wal-append.queue_time_us":172828}]]}
I20260709 12:20:23.811190 27116 tablet_replica.cc:333] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.833463 27116 raft_consensus.cc:2243] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:23.834246 27116 raft_consensus.cc:2272] T e77b6d26dd064626ab3b950e5a6c272f P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:23.837368 27116 tablet_replica.cc:333] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.838581 27116 raft_consensus.cc:2243] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:23.798836 28330 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.839408 28330 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:23.812551 28257 leader_election.cc:304] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:23.843598 27116 pending_rounds.cc:70] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Trying to abort 1 pending ops.
W20260709 12:20:23.844312 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 45514.6us
W20260709 12:20:23.844924 28330 consensus_peers.cc:597] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.845757  3558 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:20:23.846290 28257 raft_consensus.cc:2686] T 61d225f747aa45ff89df6c0ba308faad P a3de34b6f4cf4606a61944db6e2739c8: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:20:23.844208 27116 pending_rounds.cc:77] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8: Aborting op as it isn't in flight: id { term: 16 index: 10 } timestamp: 7305624020926631936 op_type: NO_OP noop_request { }
I20260709 12:20:23.847340 28181 catalog_manager.cc:5697] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 13 to 15, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 15 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:23.848546 27116 raft_consensus.cc:2889] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:20:23.849274 27116 raft_consensus.cc:2272] T 986f172991d84af28d4f5cbe8c517e6e P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:23.851035 28181 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 13 to 14, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 14 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:23.852706 27116 tablet_replica.cc:333] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.852888 28181 catalog_manager.cc:5697] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:23.862092 27116 raft_consensus.cc:2243] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:23.863004 27116 raft_consensus.cc:2272] T 54211b8ac4fe44c59e9f0338d9960177 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:23.870980  3585 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:23.875494 28404 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 118 similar messages]
I20260709 12:20:23.879004  3585 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:23.845314 28330 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.880237 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:20:23.880330 28404 leader_election.cc:336] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.881371 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:23.882782 28404 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.883255 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:23.884785  3638 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:23.885848  3585 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:23.885872  3558 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.887776  3347 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.888345  3347 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:23.891909 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:23.895207 28404 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.895700 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:23.897816  3638 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:23.901355 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.901170 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ac8850682043e5bf8caf7fc64fae05" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:23.904033 28404 consensus_peers.cc:597] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.904866 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.905381 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:23.899430  3347 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:23.906821 28255 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:23.910287 28404 leader_election.cc:336] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:23.910925 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:23.914561 28330 consensus_peers.cc:597] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.902861  3638 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:23.915261 28330 consensus_peers.cc:597] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:23.916441 28404 consensus_peers.cc:597] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.916533  3624 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.911703  3585 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:23.910458  3563 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:23.925257 27116 tablet_replica.cc:333] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:23.927872 27116 raft_consensus.cc:2243] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:23.931367 27116 raft_consensus.cc:2272] T 610cd41ac66c4074be8e2aa7b22d5e6c P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:23.941229 28954 consensus_peers.cc:597] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:23.962978 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:24.017604  3651 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:24.018260  3651 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:24.020627  3651 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:24.023634 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:24.026723 28434 raft_consensus.cc:1275] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:24.027889 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:24.028697 28439 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:24.034046 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:24.034780  3655 consensus_queue.cc:1048] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:24.051426  3651 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:24.047230  3464 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:24.075130  3464 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:24.077647  3464 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:24.055570  3651 raft_consensus.cc:493] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:24.083318  3651 raft_consensus.cc:3060] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:24.089174 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:24.090235 28439 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:24.092033 27116 tablet_replica.cc:333] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.093180 27116 raft_consensus.cc:2243] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.106782 27116 raft_consensus.cc:2272] T b0d8e709ec114a04bfd86247e24dd63c P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.112038 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:20:24.120216 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:24.120779 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:24.133303  3188 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:24.126387  3464 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:24.164189 28330 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:24.154572  3464 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:24.175283 27116 tablet_replica.cc:333] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.176378 27116 raft_consensus.cc:2243] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.177008 27116 pending_rounds.cc:70] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Trying to abort 1 pending ops.
I20260709 12:20:24.177258 27116 pending_rounds.cc:77] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8: Aborting op as it isn't in flight: id { term: 18 index: 14 } timestamp: 7305624044288221184 op_type: NO_OP noop_request { }
I20260709 12:20:24.177649 27116 raft_consensus.cc:2889] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:20:24.178323 27116 raft_consensus.cc:2272] T bea7beef1d4c49159d75e5f61eaaeb5a P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.179106  3464 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:24.203676 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:24.209028 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 44394.1us
W20260709 12:20:24.209862 28404 leader_election.cc:336] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:24.210312 28404 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:24.210983 27116 tablet_replica.cc:333] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.212049 27116 raft_consensus.cc:2243] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:24.212241  3559 raft_consensus.cc:2749] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:24.224653 27116 raft_consensus.cc:2272] T 9a388b90c9b140649cada36c135d7080 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.234007 27116 tablet_replica.cc:333] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.235059 27116 raft_consensus.cc:2243] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.236150 27116 raft_consensus.cc:2272] T 6c8442db4dc44c619fd67179aa212063 P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.239229 27116 tablet_replica.cc:333] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.240094 27116 raft_consensus.cc:2243] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.240657 27116 raft_consensus.cc:2272] T b4036b67205e41c597049a10a71e9467 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.243688  3464 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:24.226229  3651 raft_consensus.cc:515] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:24.246173  3464 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:24.247265 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97c4af01d02243b2b0727baa5bbacf44" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:24.247944 28434 raft_consensus.cc:3060] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:24.248862 27116 tablet_replica.cc:333] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.248870 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:24.249990 28359 raft_consensus.cc:2393] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:24.250151 27116 raft_consensus.cc:2243] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.250881 27116 raft_consensus.cc:2272] T ab6d72f1fb474adfafdd159090dc63b5 P a3de34b6f4cf4606a61944db6e2739c8 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.253145 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d63db98cb72412194c214fa082eee94" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:24.255201 27116 tablet_replica.cc:333] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.256182 27116 raft_consensus.cc:2243] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.256824 27116 raft_consensus.cc:2272] T 255e0140f2a64433a6c2dd64fe24e6b3 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.257200 28434 raft_consensus.cc:2468] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 22.
I20260709 12:20:24.260834 27116 tablet_replica.cc:333] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.261865 27116 raft_consensus.cc:2243] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:24.263038 27116 raft_consensus.cc:2272] T d6fd84f874b14098badd4d77b5703109 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.264000 28439 raft_consensus.cc:2468] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:24.264561  3651 leader_election.cc:290] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:24.265933 27116 tablet_replica.cc:333] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.265949 28956 leader_election.cc:304] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:24.267432 27116 raft_consensus.cc:2243] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.267645  3655 raft_consensus.cc:2804] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:24.268038 27116 raft_consensus.cc:2272] T 9cb896260dae4cf6bb09ba7728c603f3 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.268607  3655 raft_consensus.cc:697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:24.270865 28332 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:24.271137 27116 tablet_replica.cc:333] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.269785  3655 consensus_queue.cc:237] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:24.271796  3464 raft_consensus.cc:2804] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:24.272368  3464 raft_consensus.cc:697] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:24.273279  3464 consensus_queue.cc:237] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:24.274528 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 48199us
I20260709 12:20:24.282493 27116 raft_consensus.cc:2243] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.284615 27116 raft_consensus.cc:2272] T 7f380821c5264bfc8a5f0bf36bc52fd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:24.292346 28330 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:24.322398 27116 tablet_replica.cc:333] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.323525 27116 raft_consensus.cc:2243] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.346510 27116 raft_consensus.cc:2272] T e2ac8850682043e5bf8caf7fc64fae05 P a3de34b6f4cf4606a61944db6e2739c8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.352481 27116 tablet_replica.cc:333] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.353425 27116 raft_consensus.cc:2243] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.354074 27116 raft_consensus.cc:2272] T 23a4c78997e24df7bce95fcf6d70c95c P a3de34b6f4cf4606a61944db6e2739c8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.380903 27116 tablet_replica.cc:333] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.401997 27116 raft_consensus.cc:2243] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:20:24.403417 27116 raft_consensus.cc:2272] T 4ac0d46a7b244f2494d81e6667cfa1bf P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.414057 28189 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:24.415884 28189 catalog_manager.cc:5697] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:24.432040 27116 tablet_replica.cc:333] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.433120 27116 raft_consensus.cc:2243] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.450726 28181 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46508 (request call id 199) took 2614 ms. Trace:
I20260709 12:20:24.451972 28181 rpcz_store.cc:276] 0709 12:20:21.839471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:21.845198 (+  5727us) service_pool.cc:224] Handling call
0709 12:20:24.450649 (+2605451us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:23.941933 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:23.942534 (+   601us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:23.942584 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:20:23.942604 (+    20us) tablet.cc:662] Decoding operations
0709 12:20:23.976827 (+ 34223us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:23.986290 (+  9463us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:23.987868 (+  1578us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:20:24.078404 (+ 90536us) tablet.cc:701] Row locks acquired
0709 12:20:24.078428 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:20:24.078607 (+   179us) write_op.cc:270] Start()
0709 12:20:24.078745 (+   138us) write_op.cc:276] Timestamp: P: 1783599624078547 usec, L: 0
0709 12:20:24.078769 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:20:24.079498 (+   729us) log.cc:844] Serialized 3954 byte log entry
0709 12:20:24.195030 (+115532us) op_driver.cc:464] REPLICATION: finished
0709 12:20:24.195927 (+   897us) write_op.cc:301] APPLY: starting
0709 12:20:24.196018 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:24.228657 (+ 32639us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:24.228753 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:24.307302 (+ 78549us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:24.307540 (+   238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 12:20:24.307587 (+    47us) write_op.cc:312] APPLY: finished
0709 12:20:24.307899 (+   312us) log.cc:844] Serialized 97 byte log entry
0709 12:20:24.308107 (+   208us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:24.308289 (+   182us) write_op.cc:454] Released schema lock
0709 12:20:24.308338 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:24.308804 (+   466us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:24.308836 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2580,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":607,"apply.run_cpu_time_us":14517,"apply.run_wall_time_us":112835,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41780,"num_ops":10,"prepare.queue_time_us":59846,"prepare.run_cpu_time_us":5546,"prepare.run_wall_time_us":150875,"replication_time_us":101131,"spinlock_wait_cycles":2432,"thread_start_us":27124,"threads_started":1}]]}
I20260709 12:20:24.466459 28189 catalog_manager.cc:5697] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 22 to 23, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 23 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:24.469383 28189 catalog_manager.cc:5697] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 17 to 21, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 21 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:20:24.471464 28189 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index 14 to 15, term changed from 20 to 23, VOTER 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) evicted. New cstate: current_term: 23 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:24.433701 27116 raft_consensus.cc:2272] T 65569448c1eb448799671a8f4ba82cd4 P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.484838 28189 catalog_manager.cc:5697] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 16 to 19, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 19 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:24.492985 28189 catalog_manager.cc:5697] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:20:24.494647 28189 catalog_manager.cc:5697] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 21 to 25, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 25 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } }
I20260709 12:20:24.496120 28189 catalog_manager.cc:5697] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 19 to 21, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 21 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:24.497572 28189 catalog_manager.cc:5697] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 18 to 22, leader changed from f526369c0d0743b192be6e205540a059 (127.26.123.4) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 22 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:24.505170 28189 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:24.518245 27116 tablet_replica.cc:333] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.519457 27116 raft_consensus.cc:2243] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:24.520838 27116 raft_consensus.cc:2272] T 1fc9bc911cb145b0a259ee6481eefdf6 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.537463  3717 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:24.538105  3717 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:24.542050  3717 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:24.544389 27116 tablet_replica.cc:333] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.544715 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:24.545349 27116 raft_consensus.cc:2243] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:20:24.547854 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1410767867a5470e81621556e44c6560" candidate_uuid: "a3de34b6f4cf4606a61944db6e2739c8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:24.549854 28255 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3de34b6f4cf4606a61944db6e2739c8; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:24.551924 27116 raft_consensus.cc:2272] T bce32c3c49414e228fe6d912d95aeaac P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.553334  3717 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:24.556790 27116 tablet_replica.cc:333] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.557797 27116 raft_consensus.cc:2243] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.558620 27116 raft_consensus.cc:2272] T 014b394597e7430489bd093d8616dcad P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.563670 27116 tablet_replica.cc:333] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.564754 27116 raft_consensus.cc:2243] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.565455 27116 raft_consensus.cc:2272] T 1410767867a5470e81621556e44c6560 P a3de34b6f4cf4606a61944db6e2739c8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.568406 27116 tablet_replica.cc:333] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.569265 27116 raft_consensus.cc:2243] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.570186 27116 raft_consensus.cc:2272] T 23d6b97dd9e9466d9581f6088e681a11 P a3de34b6f4cf4606a61944db6e2739c8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.577580 28185 catalog_manager.cc:5697] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 19 to 21, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 21 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:24.584607 28348 tablet_service.cc:1558] Processing DeleteTablet for tablet bce32c3c49414e228fe6d912d95aeaac with delete_type TABLET_DATA_TOMBSTONED (TS 6634fa3fbd594726a5ec3c1686900284 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:39084
I20260709 12:20:24.588313 27116 tablet_replica.cc:333] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.589300 27116 raft_consensus.cc:2243] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.589948 27116 raft_consensus.cc:2272] T 72871f9ecbeb417fa5bd39d4714f0a90 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.592902 27116 tablet_replica.cc:333] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.593766 27116 raft_consensus.cc:2243] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:24.594897 27116 raft_consensus.cc:2272] T 777967d6330c4654813298b12226a560 P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.598493 27116 tablet_replica.cc:333] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.599453 27116 raft_consensus.cc:2243] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.600567 27116 raft_consensus.cc:2272] T 9c99f01ab7374acfa37f7669b8fcdeb2 P a3de34b6f4cf4606a61944db6e2739c8 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.602001 28185 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "a3de34b6f4cf4606a61944db6e2739c8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:24.600324 28189 catalog_manager.cc:5697] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 22 to 25, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 25 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:24.607563 27116 tablet_replica.cc:333] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.612638 27116 raft_consensus.cc:2243] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.613404 27116 raft_consensus.cc:2272] T 582bd3fd6d214e92aefb0baa58c76aed P a3de34b6f4cf4606a61944db6e2739c8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.620692  3721 tablet_replica.cc:333] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:24.622162  3721 raft_consensus.cc:2243] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.623291  3721 raft_consensus.cc:2272] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.626993  3721 ts_tablet_manager.cc:1916] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:24.628376 27116 tablet_replica.cc:333] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.630057 27116 raft_consensus.cc:2243] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.630726 27116 raft_consensus.cc:2272] T 85a68e4142444e5cb52829e957c1abf7 P a3de34b6f4cf4606a61944db6e2739c8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.640218 27116 tablet_replica.cc:333] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.641024 27116 raft_consensus.cc:2243] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:24.642150 27116 raft_consensus.cc:2272] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P a3de34b6f4cf4606a61944db6e2739c8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.647773 27116 tablet_replica.cc:333] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.650905 27116 raft_consensus.cc:2243] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.651587 27116 raft_consensus.cc:2272] T 21eeb096d8a4423bbc24e8935e0539fc P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.658414 28185 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43804 (request call id 327) took 2720 ms. Trace:
I20260709 12:20:24.656057 27116 tablet_replica.cc:333] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.663174 27116 raft_consensus.cc:2243] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:24.662612 28185 rpcz_store.cc:276] 0709 12:20:21.938054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:21.938788 (+   734us) service_pool.cc:224] Handling call
0709 12:20:24.658346 (+2719558us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:24.616742 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:24.617264 (+   522us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:24.617300 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:20:24.617318 (+    18us) tablet.cc:662] Decoding operations
0709 12:20:24.617783 (+   465us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:24.617840 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:24.617866 (+    26us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:20:24.618179 (+   313us) tablet.cc:701] Row locks acquired
0709 12:20:24.618196 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:20:24.618343 (+   147us) write_op.cc:270] Start()
0709 12:20:24.618454 (+   111us) write_op.cc:276] Timestamp: P: 1783599624618296 usec, L: 0
0709 12:20:24.618481 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:20:24.619146 (+   665us) log.cc:844] Serialized 913 byte log entry
0709 12:20:24.625484 (+  6338us) op_driver.cc:464] REPLICATION: finished
0709 12:20:24.628746 (+  3262us) write_op.cc:301] APPLY: starting
0709 12:20:24.628835 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:24.630344 (+  1509us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:24.630428 (+    84us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:24.631757 (+  1329us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:24.631930 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 12:20:24.631973 (+    43us) write_op.cc:312] APPLY: finished
0709 12:20:24.632170 (+   197us) log.cc:844] Serialized 33 byte log entry
0709 12:20:24.632342 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:24.632431 (+    89us) write_op.cc:454] Released schema lock
0709 12:20:24.632473 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:24.632860 (+   387us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:24.632891 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":3029,"apply.run_cpu_time_us":4057,"apply.run_wall_time_us":4072,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8356,"num_ops":2,"prepare.queue_time_us":135,"prepare.run_cpu_time_us":2684,"prepare.run_wall_time_us":2683,"replication_time_us":6847}]]}
I20260709 12:20:24.664361 27116 raft_consensus.cc:2272] T c474098832a04762b58521bdc2a92b5a P a3de34b6f4cf4606a61944db6e2739c8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.686820 28189 catalog_manager.cc:5697] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 19 to 20, leader changed from 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 20 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:24.690546 27116 tablet_replica.cc:333] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.690243 28189 catalog_manager.cc:5697] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } }
I20260709 12:20:24.691417 27116 raft_consensus.cc:2243] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.692016 27116 raft_consensus.cc:2272] T cdc4169a890e44e3ad14a98135e7071e P a3de34b6f4cf4606a61944db6e2739c8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.693334 28189 catalog_manager.cc:5697] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 20 to 23, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 23 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:24.706382 27116 tablet_replica.cc:333] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8: stopping tablet replica
I20260709 12:20:24.707355 27116 raft_consensus.cc:2243] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:24.708324 27116 raft_consensus.cc:2272] T 114fa146124747358c997ac1008e1b93 P a3de34b6f4cf4606a61944db6e2739c8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:24.756175  3721 ts_tablet_manager.cc:1929] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.14
I20260709 12:20:24.756814  3721 log.cc:1199] T bce32c3c49414e228fe6d912d95aeaac P 6634fa3fbd594726a5ec3c1686900284: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/wals/bce32c3c49414e228fe6d912d95aeaac
I20260709 12:20:24.759469 28174 catalog_manager.cc:5028] TS 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): tablet bce32c3c49414e228fe6d912d95aeaac (table test-workload [id=c0383ca16d6f44178ce495b61104f58e]) successfully deleted
W20260709 12:20:24.761322 28330 consensus_peers.cc:597] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:24.763175 28185 catalog_manager.cc:5697] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:24.803316 28359 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:20:24.807653  3651 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:24.881453 28439 raft_consensus.cc:1275] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:20:24.888224  3655 consensus_queue.cc:1048] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:24.889307 28439 raft_consensus.cc:1275] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:24.890974  3720 consensus_queue.cc:1048] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:25.062219  3727 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.062898  3727 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.066804  3727 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.067667 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:25.068403 28439 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 18.
I20260709 12:20:25.069684 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:25.071079  3727 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:25.071477  3727 raft_consensus.cc:493] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.071842  3727 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:25.076289 28954 leader_election.cc:336] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:25.083976  3727 raft_consensus.cc:515] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.091212 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775a5161e47944f5ad46f1616a48b2fe" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:25.091812 28439 raft_consensus.cc:3060] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:25.096360 28954 leader_election.cc:336] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:25.109874  3727 leader_election.cc:290] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.112357 28439 raft_consensus.cc:2468] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:25.115411 28956 leader_election.cc:304] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:25.116654  3727 raft_consensus.cc:2804] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:25.117687  3727 raft_consensus.cc:697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:25.118613  3727 consensus_queue.cc:237] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.161484 28189 catalog_manager.cc:5697] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 18 to 19, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 19 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:25.273139  3720 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.274315  3720 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:25.276849  3720 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.279222 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:25.280179 28439 raft_consensus.cc:2393] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 19: Already voted for candidate 9aea2391841b4888b4e195e5a1f4623b in this term.
W20260709 12:20:25.311029 28330 leader_election.cc:336] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:25.311636 28330 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:25.312559  3720 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:25.323927  3720 raft_consensus.cc:2749] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:25.351850  3557 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.352494  3557 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.354871  3557 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:25.366341 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:25.370668 28404 leader_election.cc:336] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:25.371208 28404 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:25.372037  3557 raft_consensus.cc:2749] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:25.436610 28330 consensus_peers.cc:597] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:25.442059 28404 consensus_peers.cc:597] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:25.464924  3716 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.465660  3716 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:25.473906  3716 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.477192 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:25.487635  3651 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.488353  3651 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:25.493325 28330 leader_election.cc:336] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:25.493933 28330 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:25.494753  3720 raft_consensus.cc:2749] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:25.496541 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:25.500317 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:25.501807  3651 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.503232 28439 raft_consensus.cc:1275] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:25.505105  3651 consensus_queue.cc:1048] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:20:25.511785 28954 consensus_peers.cc:597] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:25.516508 28954 consensus_peers.cc:597] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:25.517180 28954 consensus_peers.cc:597] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:25.517686 28954 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:25.518899  3557 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.519583  3557 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:25.522176  3557 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:25.524094  3716 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.526808  3716 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:25.526043 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:25.527849 28360 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 19.
I20260709 12:20:25.518165 28954 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:25.529263  3655 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:25.534648 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:25.535334 28433 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 16.
I20260709 12:20:25.536916 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:25.537811  3720 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:25.538242  3720 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.538592  3720 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:25.537858 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:25.545969  3738 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:25.546506  3738 raft_consensus.cc:493] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.546854  3738 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:25.546048 28404 leader_election.cc:336] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:25.558619  3716 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:25.602059 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 54239.1us
I20260709 12:20:25.559690  3651 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.595085  3727 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.639580  3651 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.632373  3744 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.649777  3727 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.696262  3651 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:25.695209  3720 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:25.702425  3720 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.702566  3738 raft_consensus.cc:515] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:25.703277  3742 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.703887  3742 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.704936  3738 leader_election.cc:290] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:25.698249  3744 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.706180  3742 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:25.707592  3727 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:25.708014  3744 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:25.711591 28404 consensus_peers.cc:597] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:25.712227 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:25.709939  3720 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.725651 27116 tablet_server.cc:196] TabletServer@127.26.123.1:0 shutdown complete.
W20260709 12:20:25.734373 28404 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:25.740906 28404 leader_election.cc:336] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:25.727890  3720 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:25.756172 28360 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65569448c1eb448799671a8f4ba82cd4" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:25.757007 28360 raft_consensus.cc:3060] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:25.760509  3747 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.761224  3747 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.763644  3747 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:25.766039  3720 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:25.773337 28404 leader_election.cc:336] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:25.774165 28404 consensus_peers.cc:597] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:25.774675 28404 consensus_peers.cc:597] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:25.787915  3750 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.788645  3750 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.791266  3750 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.792824  3727 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:25.786443 28332 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 96845.3us
I20260709 12:20:25.793417  3727 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.793241 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:25.795612 28434 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:25.796067  3727 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.798266 28434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:25.799014 28434 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:20:25.797757 28956 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:25.800458 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:25.800794  3651 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:25.801250  3651 raft_consensus.cc:493] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.801656  3651 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:25.803252  3727 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:25.804630 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dfaebe2a90498f928f7cd6cc499fac" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:25.805270 28439 raft_consensus.cc:3060] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:25.806286 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 123857us
I20260709 12:20:25.811200 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:25.880712 28359 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 19.
I20260709 12:20:25.808916 28360 raft_consensus.cc:2468] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 20.
I20260709 12:20:25.807615 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:25.886436 28438 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 18.
I20260709 12:20:25.842823 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
W20260709 12:20:25.842669 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:25.890178 28404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 47594.6us
I20260709 12:20:25.895247 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:25.895927 28990 raft_consensus.cc:2468] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:25.803728  3727 raft_consensus.cc:493] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.898247  3727 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:25.899936  3651 raft_consensus.cc:515] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:25.813431 28954 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:25.904294 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:42113, user_credentials={real_user=slave}} blocked reactor thread for 90914.3us
W20260709 12:20:25.904784 28954 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:25.926184  3557 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.927057 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:25.926856  3557 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:25.928207  3753 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:25.928699  3753 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.929116  3753 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:25.933928 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:25.936956 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:25.871836 28439 raft_consensus.cc:2468] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
I20260709 12:20:25.940739 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:25.939801 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:25.941228 28404 leader_election.cc:304] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:25.942150 28332 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:25.941763 28990 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 23.
I20260709 12:20:25.942119 28361 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
I20260709 12:20:25.946264 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:25.947110 28983 raft_consensus.cc:2393] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aea2391841b4888b4e195e5a1f4623b in current term 20: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:25.949828 28363 consensus_queue.cc:237] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:20:25.952234 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
I20260709 12:20:25.955269  3651 leader_election.cc:290] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.957075  3638 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:25.957614  3638 raft_consensus.cc:493] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.958046  3638 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:25.954442 28404 consensus_peers.cc:597] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:25.954161  3746 raft_consensus.cc:2804] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:25.963702  3746 raft_consensus.cc:697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:25.964000 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:25.964670  3746 consensus_queue.cc:237] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:25.952075  3754 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:25.967101 28404 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:25.951066  3747 raft_consensus.cc:2804] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:25.968094  3745 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:25.968513  3745 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.968873  3745 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:25.969241  3748 raft_consensus.cc:3060] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:25.970325  3739 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.970949  3739 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.973395  3739 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:25.967224  3754 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:25.980707  3727 raft_consensus.cc:515] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:25.981374  3754 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:25.983057  3727 leader_election.cc:290] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:25.974450  3651 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:25.984094 28404 leader_election.cc:336] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:25.937821  3557 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:25.991058 28956 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:33721, user_credentials={real_user=slave}} blocked reactor thread for 52981.8us
I20260709 12:20:25.995962  3750 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
W20260709 12:20:26.005867 28954 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:44701, user_credentials={real_user=slave}} blocked reactor thread for 69010.8us
W20260709 12:20:26.007355 28330 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:25.978878  3753 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:26.010435 28954 leader_election.cc:336] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.014042 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6fd84f874b14098badd4d77b5703109" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:20:26.014428 28954 leader_election.cc:336] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.014712 28439 raft_consensus.cc:3060] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.013770 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c474098832a04762b58521bdc2a92b5a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:25.978951  3745 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:26.015322 28954 consensus_peers.cc:597] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.023447 28439 raft_consensus.cc:2468] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
I20260709 12:20:26.024791 28956 leader_election.cc:304] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:26.030543 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:26.012115  3750 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:26.036230  3750 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:26.040206 28983 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 17.
W20260709 12:20:26.041577 28404 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.022600  3747 raft_consensus.cc:697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:26.044760  3745 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:26.050678  3727 raft_consensus.cc:2804] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:26.051389  3727 raft_consensus.cc:697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 22 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:26.052446  3727 consensus_queue.cc:237] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:26.045027  3747 consensus_queue.cc:237] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:26.062556  3753 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:26.053212 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284" is_pre_election: true
I20260709 12:20:26.067579 28359 raft_consensus.cc:2468] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 17.
I20260709 12:20:26.043319 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:26.073351 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fc9bc911cb145b0a259ee6481eefdf6" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:26.074075 28438 raft_consensus.cc:3060] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:26.074416 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bce32c3c49414e228fe6d912d95aeaac" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:26.072320 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:26.075173 28984 raft_consensus.cc:3060] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:26.086808  3750 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:26.087381  3750 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:26.087776  3750 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:26.079800  3749 raft_consensus.cc:2804] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:26.089021  3749 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:26.089453  3749 raft_consensus.cc:3060] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:26.091965 28954 leader_election.cc:336] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:26.092852 28954 leader_election.cc:336] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.098443  3753 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:26.099632  3753 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:26.100734  3748 raft_consensus.cc:2749] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:26.102777 28189 catalog_manager.cc:5697] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 15 to 17, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 17 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:26.104961  3754 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:26.107321 28954 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.106931  3749 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:26.109438  3749 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:26.116204  3638 raft_consensus.cc:515] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:26.121471 28440 raft_consensus.cc:2468] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 20.
I20260709 12:20:26.122186 28438 raft_consensus.cc:2468] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 19.
I20260709 12:20:26.123427 28332 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: 
W20260709 12:20:26.126960 28330 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:39985, user_credentials={real_user=slave}} blocked reactor thread for 46958.5us
I20260709 12:20:26.130730 28956 leader_election.cc:304] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:26.133111 28984 raft_consensus.cc:2468] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 24.
I20260709 12:20:26.139451 28404 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:26.138394 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:26.143333 28330 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.128547  3720 raft_consensus.cc:2804] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:20:26.144117 28330 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:26.144815 28330 consensus_peers.cc:597] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.145171 28330 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:26.145661 28330 consensus_peers.cc:597] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:26.137109  3753 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:26.143576  3558 raft_consensus.cc:2804] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:26.147462  3558 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:26.147954  3558 raft_consensus.cc:3060] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:26.148636  3638 leader_election.cc:290] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:26.149559 28185 catalog_manager.cc:5697] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 21 to 22, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 22 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:26.140116  3758 raft_consensus.cc:2804] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:26.174399  3758 raft_consensus.cc:697] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:26.175469  3758 consensus_queue.cc:237] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:26.176913  3747 raft_consensus.cc:2804] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:26.157027  3720 raft_consensus.cc:697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:26.177597  3747 raft_consensus.cc:697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:26.178303  3720 consensus_queue.cc:237] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:26.178535  3747 consensus_queue.cc:237] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:26.157958 28181 catalog_manager.cc:5697] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 18 to 20, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 20 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:26.182863 28359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac0d46a7b244f2494d81e6667cfa1bf" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:26.183508 28359 raft_consensus.cc:3060] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:26.158833  3558 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:26.161753  3754 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:26.162055 28954 consensus_peers.cc:597] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:26.191689 28330 consensus_peers.cc:597] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:26.198958 28361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:26.199846 28361 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 20: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:26.202632 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:26.203519 28983 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 20.
I20260709 12:20:26.213209  3750 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:26.215250 28438 raft_consensus.cc:1275] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:20:26.219885 28404 leader_election.cc:336] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:26.220692 28404 consensus_peers.cc:597] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:26.222551 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a388b90c9b140649cada36c135d7080" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:26.223403 28991 raft_consensus.cc:2393] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 20: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:26.219897 28359 raft_consensus.cc:2468] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
I20260709 12:20:26.229848  3752 consensus_queue.cc:1048] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:26.236799  3750 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:26.238088  3558 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:26.233495 28404 leader_election.cc:304] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b; no voters: a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:26.240835 28954 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:26.246963 28954 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.247484 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:26.242451 28363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6634fa3fbd594726a5ec3c1686900284"
I20260709 12:20:26.249146 28363 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f526369c0d0743b192be6e205540a059 in current term 18: Already voted for candidate 6634fa3fbd594726a5ec3c1686900284 in this term.
I20260709 12:20:26.249416 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:26.252765 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:26.253672  3750 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:20:26.254400 28330 consensus_peers.cc:597] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85a68e4142444e5cb52829e957c1abf7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.259593 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6bd3eccb6c4f4c8fedbc76fa829a96" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:26.260778 28991 raft_consensus.cc:2393] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6634fa3fbd594726a5ec3c1686900284 in current term 18: Already voted for candidate f526369c0d0743b192be6e205540a059 in this term.
I20260709 12:20:26.253799  3746 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:26.249208  3651 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:26.256467  3761 raft_consensus.cc:2804] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:26.263726  3746 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a3de34b6f4cf4606a61944db6e2739c8)
I20260709 12:20:26.282537  3761 raft_consensus.cc:697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:26.283664  3761 consensus_queue.cc:237] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:26.275995  3746 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:26.291535 28330 consensus_peers.cc:597] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.292413 28330 consensus_peers.cc:597] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.292861 28330 consensus_peers.cc:597] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.293243 28330 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:26.294082 28330 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.294528 28330 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
W20260709 12:20:26.295662 28330 consensus_peers.cc:597] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.296154 28330 consensus_peers.cc:597] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.296779 28330 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:26.297688 28330 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.299516  3753 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:20:26.313694 28330 consensus_peers.cc:597] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.314338 28330 consensus_peers.cc:597] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.314677 28330 consensus_peers.cc:597] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:26.314921 28330 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.315227 28330 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:26.316140  3749 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:20:26.317453 28330 consensus_peers.cc:597] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.324884 28404 consensus_peers.cc:597] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:26.332583  3746 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:26.334029 28330 consensus_peers.cc:597] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.334684 28330 consensus_peers.cc:597] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:26.337000  3746 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Requested vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:26.341094 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "777967d6330c4654813298b12226a560" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:26.341933 28983 raft_consensus.cc:3060] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:26.346062  3742 raft_consensus.cc:2955] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:20:26.347927 28440 raft_consensus.cc:2955] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:20:26.353307 28330 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:26.353871 28983 raft_consensus.cc:2468] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6634fa3fbd594726a5ec3c1686900284 in term 21.
I20260709 12:20:26.355679 28330 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059; no voters: a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:26.357178  3746 raft_consensus.cc:2804] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:26.357692  3746 raft_consensus.cc:697] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Becoming Leader. State: Replica: 6634fa3fbd594726a5ec3c1686900284, State: Running, Role: LEADER
I20260709 12:20:26.358560  3746 consensus_queue.cc:237] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:26.394448 28189 catalog_manager.cc:5697] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 18 to 19, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 19 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:26.401571 28185 catalog_manager.cc:5697] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 19 to 20, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 20 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:26.408761 28363 consensus_queue.cc:237] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:20:26.417363 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.420051 28330 consensus_peers.cc:597] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.435084 28181 catalog_manager.cc:5697] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 23 to 24, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 24 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:26.437049 28181 catalog_manager.cc:5697] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 15 to 16, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:26.438674 28181 catalog_manager.cc:5697] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 13, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 18 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:20:26.454071 28954 consensus_peers.cc:597] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:26.466723 28359 raft_consensus.cc:1275] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:20:26.471832  3761 consensus_queue.cc:1048] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:20:26.487095 28330 consensus_peers.cc:597] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:26.490358 28181 catalog_manager.cc:5697] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: term changed from 20 to 21, leader changed from a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1) to 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2). New cstate: current_term: 21 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:26.500958 28440 raft_consensus.cc:1275] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:26.505476  3753 consensus_queue.cc:1048] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:20:26.520087 28404 consensus_peers.cc:597] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.545480 28404 consensus_peers.cc:597] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:26.560587 28330 consensus_peers.cc:597] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0d8e709ec114a04bfd86247e24dd63c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.566087  3720 raft_consensus.cc:2955] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Committing config change with OpId 21.12: config changed from index -1 to 12, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
I20260709 12:20:26.572849 28440 raft_consensus.cc:1275] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:20:26.574448  3742 consensus_queue.cc:1048] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:26.579373 28438 raft_consensus.cc:2955] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Committing config change with OpId 21.12: config changed from index -1 to 12, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:20:26.605796 28330 consensus_peers.cc:597] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.614673 28438 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:20:26.620131 28330 consensus_peers.cc:597] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:26.623718 28954 consensus_peers.cc:597] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.619508 28189 catalog_manager.cc:5697] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 reported cstate change: config changed from index -1 to 12, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 21 leader_uuid: "6634fa3fbd594726a5ec3c1686900284" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:26.644399 28991 raft_consensus.cc:1275] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:20:26.647933 28439 raft_consensus.cc:1275] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:26.650767  3742 consensus_queue.cc:1048] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:26.651297 28439 raft_consensus.cc:1275] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:20:26.653194  3750 consensus_queue.cc:1048] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:26.662089  3745 consensus_queue.cc:1048] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:20:26.667824 28404 consensus_peers.cc:597] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:26.683210 28330 consensus_peers.cc:597] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:26.683710  3762 log.cc:927] Time spent T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: Append to log took a long time: real 0.051s	user 0.000s	sys 0.016s
I20260709 12:20:26.703510 28359 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:26.719637 28359 raft_consensus.cc:1275] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:20:26.732199 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:26.755438  3761 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:26.763734  3787 ts_tablet_manager.cc:933] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:26.768435  3761 consensus_queue.cc:1048] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:20:26.774143 28954 consensus_peers.cc:597] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:26.775153 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21eeb096d8a4423bbc24e8935e0539fc. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.801450  3787 tablet_copy_client.cc:323] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:20:26.807435 28440 raft_consensus.cc:1275] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:20:26.814440 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.814461  3727 consensus_queue.cc:1048] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:20:26.825896 28376 tablet_copy_service.cc:140] P 6634fa3fbd594726a5ec3c1686900284: Received BeginTabletCopySession request for tablet 85a68e4142444e5cb52829e957c1abf7 from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:56626)
I20260709 12:20:26.826638 28376 tablet_copy_service.cc:161] P 6634fa3fbd594726a5ec3c1686900284: Beginning new tablet copy session on tablet 85a68e4142444e5cb52829e957c1abf7 from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:56626: session id = f526369c0d0743b192be6e205540a059-85a68e4142444e5cb52829e957c1abf7
W20260709 12:20:26.856621 28404 consensus_peers.cc:597] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.860035 28376 tablet_copy_source_session.cc:215] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:26.865932  3787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a68e4142444e5cb52829e957c1abf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:26.925827 27116 tablet_server.cc:179] TabletServer@127.26.123.2:0 shutting down...
I20260709 12:20:26.937173  3761 raft_consensus.cc:2955] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } }
W20260709 12:20:26.940330 28404 consensus_peers.cc:597] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:26.942090 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.973255 28440 consensus_queue.cc:237] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:20:26.978346 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:26.982744 28330 consensus_peers.cc:597] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:26.991801  3787 tablet_copy_client.cc:806] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:26.998543 28983 raft_consensus.cc:1275] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Refusing update from remote peer 6634fa3fbd594726a5ec3c1686900284: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:20:26.999297 28185 catalog_manager.cc:5697] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: config changed from index -1 to 13, NON_VOTER f526369c0d0743b192be6e205540a059 (127.26.123.4) added. New cstate: current_term: 19 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:27.000938  3787 tablet_copy_client.cc:670] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:27.001940  3749 consensus_queue.cc:1048] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:20:27.002640 28404 consensus_peers.cc:597] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:27.019929 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:27.036940 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer f526369c0d0743b192be6e205540a059 (127.26.123.4:39985): Couldn't send request to peer f526369c0d0743b192be6e205540a059. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 986f172991d84af28d4f5cbe8c517e6e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:27.039417 27116 tablet_copy_service.cc:362] P 6634fa3fbd594726a5ec3c1686900284: Destroying tablet copy session f526369c0d0743b192be6e205540a059-85a68e4142444e5cb52829e957c1abf7 due to service shutdown
W20260709 12:20:27.042127 28954 consensus_peers.cc:597] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:27.044211 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:27.048344 27116 tablet_copy_service.cc:434] P 6634fa3fbd594726a5ec3c1686900284: ending tablet copy session f526369c0d0743b192be6e205540a059-85a68e4142444e5cb52829e957c1abf7 on tablet 85a68e4142444e5cb52829e957c1abf7 with peer f526369c0d0743b192be6e205540a059
W20260709 12:20:27.076668 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:27.100611 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:27.105090 28404 consensus_peers.cc:597] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:27.135854 28404 consensus_peers.cc:597] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:27.182837  3794 ts_tablet_manager.cc:933] T b0d8e709ec114a04bfd86247e24dd63c P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:27.215153 28954 consensus_peers.cc:597] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:27.224627 28954 consensus_peers.cc:597] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:27.235075  3797 ts_tablet_manager.cc:933] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:27.238976 28404 consensus_peers.cc:597] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:27.266525  3794 tablet_copy_client.cc:323] T b0d8e709ec114a04bfd86247e24dd63c P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.2:44701
I20260709 12:20:27.280092  3797 tablet_copy_client.cc:323] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:20:27.283568 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 21eeb096d8a4423bbc24e8935e0539fc from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:20:27.284212 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 21eeb096d8a4423bbc24e8935e0539fc from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-21eeb096d8a4423bbc24e8935e0539fc
W20260709 12:20:27.292601 28404 consensus_peers.cc:597] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:27.302253 28404 consensus_peers.cc:597] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:27.305833 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:27.306977 28451 tablet_copy_source_session.cc:215] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:27.308120 27116 tablet_replica.cc:333] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.309629 27116 raft_consensus.cc:2243] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.311178 27116 raft_consensus.cc:2272] T 39dfaebe2a90498f928f7cd6cc499fac P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.314999  3797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21eeb096d8a4423bbc24e8935e0539fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:27.333345 27116 tablet_replica.cc:333] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.334460 27116 raft_consensus.cc:2243] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.335723 27116 raft_consensus.cc:2272] T 6b2edb513f7343d2a2463b94d992b7a3 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:27.374174 28404 consensus_peers.cc:597] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:27.406605 27116 tablet_replica.cc:333] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.451288 27116 raft_consensus.cc:2243] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 17 LEADER]: Raft consensus shutting down.
W20260709 12:20:27.455289 28404 consensus_peers.cc:597] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:27.464371 27116 raft_consensus.cc:2272] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.506973 27116 tablet_replica.cc:333] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.508026 27116 raft_consensus.cc:2243] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.508709 27116 raft_consensus.cc:2272] T d8850fe0201a48528da0da1cdac707f0 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.521536  3797 tablet_copy_client.cc:806] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:27.522261  3797 tablet_copy_client.cc:670] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:27.527837 27116 tablet_replica.cc:333] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.528929 27116 raft_consensus.cc:2243] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.531126 27116 raft_consensus.cc:2272] T 97c4af01d02243b2b0727baa5bbacf44 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.542426 27116 tablet_replica.cc:333] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.543484 27116 raft_consensus.cc:2243] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.544799 27116 raft_consensus.cc:2272] T 8ab1586b52be49808b90fcc88bdf2d60 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.546062  3797 tablet_copy_client.cc:538] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:20:27.553297 28404 consensus_peers.cc:597] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:27.568791 27116 tablet_replica.cc:333] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.573207 27116 raft_consensus.cc:2243] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.574491 27116 raft_consensus.cc:2272] T d68008d2b3fd4f7bb72bf69f612b2998 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.572079  3797 tablet_bootstrap.cc:492] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:20:27.613874 27116 tablet_replica.cc:333] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.614841 27116 raft_consensus.cc:2243] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.619544  3800 ts_tablet_manager.cc:933] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:27.622933 27116 raft_consensus.cc:2272] T 189fdd6055df4826a42af9c225596e34 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.625857 27116 tablet_replica.cc:333] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.626830 27116 raft_consensus.cc:2243] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.628010 27116 raft_consensus.cc:2272] T 916dfa76a5254f63bfc746dbe9c5fde0 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.649139  3800 tablet_copy_client.cc:323] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: tablet copy: Beginning tablet copy session from remote peer at address 127.26.123.3:33721
I20260709 12:20:27.651013 28451 tablet_copy_service.cc:140] P 9aea2391841b4888b4e195e5a1f4623b: Received BeginTabletCopySession request for tablet 986f172991d84af28d4f5cbe8c517e6e from peer f526369c0d0743b192be6e205540a059 ({username='slave'} at 127.0.0.1:53280)
I20260709 12:20:27.651557 28451 tablet_copy_service.cc:161] P 9aea2391841b4888b4e195e5a1f4623b: Beginning new tablet copy session on tablet 986f172991d84af28d4f5cbe8c517e6e from peer f526369c0d0743b192be6e205540a059 at {username='slave'} at 127.0.0.1:53280: session id = f526369c0d0743b192be6e205540a059-986f172991d84af28d4f5cbe8c517e6e
I20260709 12:20:27.659346 27116 tablet_replica.cc:333] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.660470 27116 raft_consensus.cc:2243] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.660938 28451 tablet_copy_source_session.cc:215] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:27.661697 27116 raft_consensus.cc:2272] T 66010729a96e49ebb751b59d4c7f5f3a P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.664887 27116 tablet_replica.cc:333] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.670085 27116 raft_consensus.cc:2243] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.677557  3800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986f172991d84af28d4f5cbe8c517e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:27.677937 27116 raft_consensus.cc:2272] T b5bac9d516d84373a10b8812e66d31ed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.680954 27116 tablet_replica.cc:333] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.681963 27116 raft_consensus.cc:2243] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.682641 27116 raft_consensus.cc:2272] T 819ca6350ca84ff49e7254638d8755e7 P 6634fa3fbd594726a5ec3c1686900284 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.707984 27116 tablet_replica.cc:333] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.708875 27116 raft_consensus.cc:2243] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.717424 27116 raft_consensus.cc:2272] T 705e15df427147839fee08c92bc268dc P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.728673 27116 tablet_replica.cc:333] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.729585 27116 raft_consensus.cc:2243] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.734014 27116 raft_consensus.cc:2272] T 8d63db98cb72412194c214fa082eee94 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.740276 27116 tablet_replica.cc:333] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.741238 27116 raft_consensus.cc:2243] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.749146 27116 raft_consensus.cc:2272] T 9a388b90c9b140649cada36c135d7080 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.751484  3800 tablet_copy_client.cc:806] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:27.752144  3800 tablet_copy_client.cc:670] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:27.764022 27116 tablet_replica.cc:333] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.765982 27116 raft_consensus.cc:2243] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.766779 27116 raft_consensus.cc:2272] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.769672 27116 tablet_replica.cc:333] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.770596 27116 raft_consensus.cc:2243] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.772508 27116 raft_consensus.cc:2272] T 85a68e4142444e5cb52829e957c1abf7 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.774263  3800 tablet_copy_client.cc:538] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:27.775436 27116 tablet_replica.cc:333] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.776420 27116 raft_consensus.cc:2243] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.777072 27116 raft_consensus.cc:2272] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.780031 27116 tablet_replica.cc:333] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.781064 27116 raft_consensus.cc:2243] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.782269 27116 raft_consensus.cc:2272] T 9cb896260dae4cf6bb09ba7728c603f3 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.789458  3797 tablet_bootstrap.cc:492] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=3 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:27.791359  3797 tablet_bootstrap.cc:492] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:20:27.798084  3797 ts_tablet_manager.cc:1403] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.226s	user 0.044s	sys 0.034s
I20260709 12:20:27.802157 27116 tablet_replica.cc:333] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.803331 27116 raft_consensus.cc:2243] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.804629 27116 raft_consensus.cc:2272] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.801472  3797 raft_consensus.cc:359] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:20:27.807160  3797 raft_consensus.cc:740] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:20:27.808027  3797 consensus_queue.cc:260] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:20:27.810613  3797 ts_tablet_manager.cc:1434] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:20:27.811928 27116 tablet_replica.cc:333] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.812939 27116 raft_consensus.cc:2243] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.813675 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-21eeb096d8a4423bbc24e8935e0539fc received from {username='slave'} at 127.0.0.1:53280
I20260709 12:20:27.813881 27116 raft_consensus.cc:2272] T 5fb143bfa1d44653b638d40442c7e955 P 6634fa3fbd594726a5ec3c1686900284 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.814219 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-21eeb096d8a4423bbc24e8935e0539fc on tablet 21eeb096d8a4423bbc24e8935e0539fc with peer f526369c0d0743b192be6e205540a059
I20260709 12:20:27.817021 27116 tablet_replica.cc:333] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.818051 27116 raft_consensus.cc:2243] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.819563 27116 raft_consensus.cc:2272] T e2ac8850682043e5bf8caf7fc64fae05 P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.828209 27116 tablet_replica.cc:333] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.829241 27116 raft_consensus.cc:2243] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.830606 27116 raft_consensus.cc:2272] T 1410767867a5470e81621556e44c6560 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.835991 27116 tablet_replica.cc:333] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.836977 27116 raft_consensus.cc:2243] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.837709 27116 raft_consensus.cc:2272] T 014b394597e7430489bd093d8616dcad P 6634fa3fbd594726a5ec3c1686900284 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.840373  3800 tablet_bootstrap.cc:492] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: Bootstrap starting.
I20260709 12:20:27.840811 27116 tablet_replica.cc:333] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.841879 27116 raft_consensus.cc:2243] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.843222 27116 raft_consensus.cc:2272] T 9cdd1e39ef1841c382e814af04ea5bc9 P 6634fa3fbd594726a5ec3c1686900284 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.846755 27116 tablet_replica.cc:333] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.847771 27116 raft_consensus.cc:2243] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.849157 27116 raft_consensus.cc:2272] T 23a4c78997e24df7bce95fcf6d70c95c P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.852459 27116 tablet_replica.cc:333] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.853652 27116 raft_consensus.cc:2243] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.857645 27116 raft_consensus.cc:2272] T c59ee067797b40d396f97683aa7eca81 P 6634fa3fbd594726a5ec3c1686900284 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.860870 27116 tablet_replica.cc:333] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.861860 27116 raft_consensus.cc:2243] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.863283 27116 raft_consensus.cc:2272] T b4036b67205e41c597049a10a71e9467 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.866169 27116 tablet_replica.cc:333] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.867033 27116 raft_consensus.cc:2243] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.868280 27116 raft_consensus.cc:2272] T ab6d72f1fb474adfafdd159090dc63b5 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.871263 27116 tablet_replica.cc:333] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.872176 27116 raft_consensus.cc:2243] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.873203 27116 raft_consensus.cc:2272] T 956fb13fa38243fd862789f3046aba97 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.877434 27116 tablet_replica.cc:333] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.878455 27116 raft_consensus.cc:2243] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.879722 27116 raft_consensus.cc:2272] T 54211b8ac4fe44c59e9f0338d9960177 P 6634fa3fbd594726a5ec3c1686900284 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.882956 27116 tablet_replica.cc:333] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.885098 27116 raft_consensus.cc:2243] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.886439 27116 raft_consensus.cc:2272] T 610cd41ac66c4074be8e2aa7b22d5e6c P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.889714 27116 tablet_replica.cc:333] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.890820 27116 raft_consensus.cc:2243] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.892338 27116 raft_consensus.cc:2272] T b0d8e709ec114a04bfd86247e24dd63c P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:27.894414 28954 consensus_peers.cc:597] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:27.895525 27116 tablet_replica.cc:333] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.900178 27116 raft_consensus.cc:2243] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.901578 27116 raft_consensus.cc:2272] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P 6634fa3fbd594726a5ec3c1686900284 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.904764 27116 tablet_replica.cc:333] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.906083 27116 raft_consensus.cc:2243] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.906992 27116 raft_consensus.cc:2272] T 986f172991d84af28d4f5cbe8c517e6e P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.927829 27116 tablet_replica.cc:333] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.928821 27116 raft_consensus.cc:2243] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.929724 27116 raft_consensus.cc:2272] T 4ac0d46a7b244f2494d81e6667cfa1bf P 6634fa3fbd594726a5ec3c1686900284 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.936718 27116 tablet_replica.cc:333] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.937811 27116 raft_consensus.cc:2243] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.939330 27116 raft_consensus.cc:2272] T 65569448c1eb448799671a8f4ba82cd4 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:27.940074 28954 consensus_peers.cc:597] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:27.946287 27116 tablet_replica.cc:333] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.947373 27116 raft_consensus.cc:2243] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.948778 27116 raft_consensus.cc:2272] T 1fc9bc911cb145b0a259ee6481eefdf6 P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.950646  3727 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.951272  3727 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:27.953188 27116 tablet_replica.cc:333] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.960026  3795 raft_consensus.cc:493] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:27.960767  3795 raft_consensus.cc:515] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:27.963138  3795 leader_election.cc:290] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:27.967365 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc4169a890e44e3ad14a98135e7071e" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:27.953614  3727 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:27.971310 28954 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:27.973702 28330 leader_election.cc:336] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:27.971685 27116 raft_consensus.cc:2243] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:27.980410 28330 leader_election.cc:304] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: a3de34b6f4cf4606a61944db6e2739c8, f526369c0d0743b192be6e205540a059
I20260709 12:20:27.981240 27116 raft_consensus.cc:2272] T 3ee4696ba90a447fa07c4d09407217f6 P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:27.981935  3795 raft_consensus.cc:2749] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:27.984860  3795 raft_consensus.cc:493] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:27.985389 27116 tablet_replica.cc:333] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:27.985442  3795 raft_consensus.cc:515] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:27.987707 28954 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:27.988332 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:27.991143  3754 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:27.991304 28433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:27.989629  3795 leader_election.cc:290] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:27.991753 27116 raft_consensus.cc:2243] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:27.991869 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3e7c85ff90423aa9f9d234d992379b" candidate_uuid: "6634fa3fbd594726a5ec3c1686900284" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:27.998646 27116 raft_consensus.cc:2272] T 72871f9ecbeb417fa5bd39d4714f0a90 P 6634fa3fbd594726a5ec3c1686900284 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:28.002036  3754 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.003006  3754 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:28.004880 27116 tablet_replica.cc:333] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:28.005261  3754 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:28.005857 27116 raft_consensus.cc:2243] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:28.007104 28330 leader_election.cc:304] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6634fa3fbd594726a5ec3c1686900284; no voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059
I20260709 12:20:28.006593 27116 raft_consensus.cc:2272] T 582bd3fd6d214e92aefb0baa58c76aed P 6634fa3fbd594726a5ec3c1686900284 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:28.009490  3795 raft_consensus.cc:2749] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:28.010809 27116 tablet_replica.cc:333] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:28.011686 27116 raft_consensus.cc:2243] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:28.013298 27116 raft_consensus.cc:2272] T 777967d6330c4654813298b12226a560 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:28.016592 27116 tablet_replica.cc:333] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:28.017397 27116 raft_consensus.cc:2243] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:28.018615 27116 raft_consensus.cc:2272] T 114fa146124747358c997ac1008e1b93 P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:28.020846 27116 tablet_replica.cc:333] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:28.021404 27116 raft_consensus.cc:2243] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:28.022009 27116 raft_consensus.cc:2272] T 2e3e7c85ff90423aa9f9d234d992379b P 6634fa3fbd594726a5ec3c1686900284 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:28.031373 28954 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:28.032171 28954 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:20:28.032501 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:28.033114  3805 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:28.034222 27116 tablet_replica.cc:333] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:28.035063 27116 raft_consensus.cc:2243] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:28.035727 27116 raft_consensus.cc:2272] T cdc4169a890e44e3ad14a98135e7071e P 6634fa3fbd594726a5ec3c1686900284 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:28.040307 27116 tablet_replica.cc:333] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: stopping tablet replica
I20260709 12:20:28.041064 27116 raft_consensus.cc:2243] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:28.041671 27116 pending_rounds.cc:70] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Trying to abort 1 pending ops.
I20260709 12:20:28.041930 27116 pending_rounds.cc:77] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284: Aborting op as it isn't in flight: id { term: 19 index: 13 } timestamp: 7305624070604935168 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } } new_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } } }
I20260709 12:20:28.042940 27116 raft_consensus.cc:2917] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Aborting config change with OpId 19.13: Aborted: Op aborted
I20260709 12:20:28.043377 27116 raft_consensus.cc:2889] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:20:28.045468  3800 tablet_bootstrap.cc:492] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:20:28.045799 27116 raft_consensus.cc:2272] T 21eeb096d8a4423bbc24e8935e0539fc P 6634fa3fbd594726a5ec3c1686900284 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:28.046620  3800 tablet_bootstrap.cc:492] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: Bootstrap complete.
I20260709 12:20:28.047434  3800 ts_tablet_manager.cc:1403] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: Time spent bootstrapping tablet: real 0.207s	user 0.045s	sys 0.004s
I20260709 12:20:28.051966  3800 raft_consensus.cc:359] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059 [term 16 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:28.053694  3800 raft_consensus.cc:740] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: f526369c0d0743b192be6e205540a059, State: Initialized, Role: LEARNER
I20260709 12:20:28.054447  3800 consensus_queue.cc:260] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:20:28.057458  3800 ts_tablet_manager.cc:1434] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:20:28.063091 28451 tablet_copy_service.cc:342] P 9aea2391841b4888b4e195e5a1f4623b: Request end of tablet copy session f526369c0d0743b192be6e205540a059-986f172991d84af28d4f5cbe8c517e6e received from {username='slave'} at 127.0.0.1:53280
I20260709 12:20:28.063537 28451 tablet_copy_service.cc:434] P 9aea2391841b4888b4e195e5a1f4623b: ending tablet copy session f526369c0d0743b192be6e205540a059-986f172991d84af28d4f5cbe8c517e6e on tablet 986f172991d84af28d4f5cbe8c517e6e with peer f526369c0d0743b192be6e205540a059
I20260709 12:20:28.094136 28984 raft_consensus.cc:1217] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.10->[16.11-16.11]   Dedup: 16.11->[]
W20260709 12:20:28.128947 28954 consensus_peers.cc:597] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:28.136302 28331 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.221679 28984 raft_consensus.cc:1217] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: Deduplicated request from leader. Original: 19.12->[19.13-19.13]   Dedup: 19.13->[]
W20260709 12:20:28.325567 28954 consensus_peers.cc:597] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:28.511920 28954 consensus_peers.cc:597] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:28.518422  3799 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.519157  3799 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:28.521521  3799 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:28.527001  3799 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:28.527248 28404 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:28.527624  3799 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:28.529752  3761 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.530372  3761 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:28.532676  3761 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:28.533496  3799 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:28.539402 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:28.539968 28404 leader_election.cc:336] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:28.544049  3787 ts_tablet_manager.cc:1011] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: Tablet Copy: Unable to fetch data from remote peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Unable to download WAL segment with seq. number 1: unable to fetch data from remote: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:28.544688  3787 tablet_replica.cc:333] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:28.545045  3787 raft_consensus.cc:2243] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Raft consensus shutting down.
I20260709 12:20:28.545464  3787 raft_consensus.cc:2272] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059 [term 18 LEARNER]: Raft consensus is shut down!
W20260709 12:20:28.548935  3787 tablet_copy_client.cc:1131] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:28.550674  3787 tablet_metadata.cc:476] failed to load DataDirGroup from superblock: Already present: tried to load directory group for tablet 85a68e4142444e5cb52829e957c1abf7 but one is already registered
I20260709 12:20:28.551139  3787 ts_tablet_manager.cc:1916] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:28.602030  3747 raft_consensus.cc:1064] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: attempting to promote NON_VOTER f526369c0d0743b192be6e205540a059 to VOTER
I20260709 12:20:28.605216  3747 consensus_queue.cc:237] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:28.617091  3787 ts_tablet_manager.cc:1929] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.0
I20260709 12:20:28.617705  3787 log.cc:1199] T 85a68e4142444e5cb52829e957c1abf7 P f526369c0d0743b192be6e205540a059: Deleting WAL directory at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0/minicluster-data/ts-3-root/wals/85a68e4142444e5cb52829e957c1abf7
I20260709 12:20:28.637187  3799 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:20:28.640452 28983 raft_consensus.cc:1275] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:28.643815  3745 consensus_queue.cc:1048] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:20:28.658473 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:28.702579 28954 consensus_peers.cc:597] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:28.708838  3745 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.709539  3745 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:28.711891  3745 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:28.721977  3747 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.722643  3747 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:28.724915  3747 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:28.730329 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:28.732537 28404 leader_election.cc:336] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:28.743108 28404 leader_election.cc:336] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:28.743737 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:28.744647  3747 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:28.758543  3747 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.759298  3747 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:28.760802  3805 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.761435  3805 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:28.765530 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:28.766336 28440 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
I20260709 12:20:28.767328  3805 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:28.769111 28956 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:28.770085  3804 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:28.770507  3804 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.770864  3804 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:28.775282  3794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46950 (request call id 1616) took 1673 ms. Trace:
I20260709 12:20:28.775761  3794 rpcz_store.cc:276] 0709 12:20:27.101632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.101805 (+   173us) service_pool.cc:224] Handling call
0709 12:20:27.183287 (+ 81482us) ts_tablet_manager.cc:934] T b0d8e709ec114a04bfd86247e24dd63c P f526369c0d0743b192be6e205540a059: Initiating tablet copy from peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
0709 12:20:28.775153 (+1591866us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":320,"tablet-copy.queue_time_us":79070,"thread_start_us":146179,"threads_started":2}
W20260709 12:20:28.777997 28957 connection.cc:466] server connection from 127.0.0.1:46950 torn down before Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46950 (request call id 1616) could send its response
W20260709 12:20:28.782279 28954 leader_election.cc:336] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:28.785526  3747 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:28.789707 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:28.785631  3804 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:28.797984  3804 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:28.801249 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
W20260709 12:20:28.801959 28954 leader_election.cc:336] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:28.802683 28440 raft_consensus.cc:3060] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:28.814661 28440 raft_consensus.cc:2468] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 23.
I20260709 12:20:28.816147 28956 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:28.817489  3804 raft_consensus.cc:2804] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:28.818058  3804 raft_consensus.cc:697] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 23 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:28.819056  3804 consensus_queue.cc:237] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:28.832899 28954 consensus_peers.cc:597] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:28.853529 28404 leader_election.cc:336] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:28.854236 28404 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:28.855129  3747 raft_consensus.cc:2749] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:28.858355 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:28.859385 28404 consensus_peers.cc:597] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:28.859477 28185 catalog_manager.cc:5697] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 22 to 23, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 23 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:28.860502 28404 consensus_peers.cc:597] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:28.861420 28404 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:28.863561 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:28.864511  3747 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:28.865787 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:28.866276 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:28.867070  3747 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:28.868103 28404 leader_election.cc:336] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:28.868589 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:28.869397  3747 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:28.870666 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:28.871193 28404 leader_election.cc:336] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:28.871625 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:28.872392  3747 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:28.879557 28404 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111) [suppressed 467 similar messages]
I20260709 12:20:28.886590  3747 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.887243  3747 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:28.889520  3747 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:28.896938 28404 consensus_peers.cc:597] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:28.903891  3761 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.904511  3761 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:20:28.906344  3745 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.906744  3761 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:28.907095  3745 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
W20260709 12:20:28.908424 28404 leader_election.cc:336] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:28.910276 28404 leader_election.cc:336] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:28.911283 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:28.917428  3761 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:28.909513  3745 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:28.925755  3804 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.926465  3804 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:28.930604  3804 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:28.932157 28404 leader_election.cc:336] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:28.939877 28404 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:28.940919 28404 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:28.941370 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:28.942232  3761 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:28.945298 28954 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:28.946164 28954 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:28.946636 28954 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:28.948386  3804 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:28.968421 28404 leader_election.cc:336] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:28.969139 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:28.970422  3745 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:28.980578 28404 consensus_peers.cc:597] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:28.990311  3745 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:28.990931  3745 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:28.993206  3745 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:28.994683 28404 consensus_peers.cc:597] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:29.004453 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.005342 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.005865 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.006722  3745 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:29.020555  3745 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.021152  3745 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.023526  3745 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:29.026681  3804 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.027281  3804 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.029482  3804 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:29.032270 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:29.033403 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:29.038401  3745 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.039043  3745 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:29.041203  3745 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:29.038275 28954 leader_election.cc:336] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.043054 28954 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:29.044447  3804 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:29.048840  3745 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.049494  3745 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:29.052294  3745 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:29.053130 28404 leader_election.cc:336] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.053709 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:29.054502  3745 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:29.055896  3804 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.056475  3804 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.058735  3804 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:29.062565 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:29.063287 28440 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 21.
I20260709 12:20:29.064703 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
W20260709 12:20:29.068734 28954 leader_election.cc:336] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.069219  3805 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:29.069700  3805 raft_consensus.cc:493] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.070052  3804 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.070225  3805 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:29.070683  3804 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.073011  3804 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:29.081061  3805 raft_consensus.cc:515] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.082841  3761 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:29.083681 28954 leader_election.cc:336] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.087911 28954 leader_election.cc:336] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:29.086268 28404 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:29.091614 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189fdd6055df4826a42af9c225596e34" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:29.092300 28440 raft_consensus.cc:3060] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:29.092818 28404 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.094713 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.095481  3761 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:29.085517  3804 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.098722  3804 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.089836  3816 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.104573  3804 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:29.104601  3816 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.114666  3816 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:29.088613  3805 leader_election.cc:290] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:29.107894 28438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:29.115582 28438 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 26.
I20260709 12:20:29.110018  3761 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.116966  3761 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:29.102398 28440 raft_consensus.cc:2468] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 22.
I20260709 12:20:29.101543 28954 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.118815  3816 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:29.120844  3761 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:29.123503  3745 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.124137  3745 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:29.124580 28404 leader_election.cc:336] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.125512 28404 leader_election.cc:336] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.125885 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.126201 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:29.127218 28956 leader_election.cc:304] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:29.126600  3745 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:29.128501  3818 raft_consensus.cc:2804] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:29.127072  3816 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:29.129304  3816 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.129761  3816 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:29.129839  3818 raft_consensus.cc:697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 22 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:29.127543  3761 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:29.130905  3818 consensus_queue.cc:237] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:29.138468 28954 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.144042 28954 leader_election.cc:336] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:29.144685 28954 leader_election.cc:336] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:29.145255 28954 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.145550 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.146183  3817 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:20:29.147188  3761 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:29.147737 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:29.148209 28404 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.148465 28954 consensus_peers.cc:597] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:29.148079  3761 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:29.148856 28404 leader_election.cc:336] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.149657 28404 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.148874  3816 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.150121 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.152029  3761 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
W20260709 12:20:29.152310 28404 leader_election.cc:336] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.152578  3761 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:29.153720 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.153079 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:29.154582 28440 raft_consensus.cc:3060] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:29.155648  3745 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:29.155608  3816 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:29.157469 28984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:29.162458 28404 leader_election.cc:336] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.163012 28404 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
W20260709 12:20:29.163201 28954 leader_election.cc:336] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.163851  3761 raft_consensus.cc:2749] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:29.164625 28440 raft_consensus.cc:2468] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 27.
I20260709 12:20:29.165647 28956 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:29.166862  3816 raft_consensus.cc:2804] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:29.167347  3816 raft_consensus.cc:697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 27 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
I20260709 12:20:29.168061  3816 consensus_queue.cc:237] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.171046  3817 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.171638  3817 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.175014  3817 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:29.178185 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b" is_pre_election: true
I20260709 12:20:29.178851 28440 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 15.
I20260709 12:20:29.180374 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 
I20260709 12:20:29.181284  3805 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:29.180742 28185 catalog_manager.cc:5697] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 21 to 22, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 22 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:29.181649 28954 leader_election.cc:336] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.181756  3805 raft_consensus.cc:493] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.182201  3761 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.182893  3805 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:29.183023  3761 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:29.185271  3761 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:29.194973  3805 raft_consensus.cc:515] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.204667  3805 leader_election.cc:290] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:29.204871 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66010729a96e49ebb751b59d4c7f5f3a" candidate_uuid: "f526369c0d0743b192be6e205540a059" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aea2391841b4888b4e195e5a1f4623b"
I20260709 12:20:29.206729 28440 raft_consensus.cc:3060] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:29.210752 28954 leader_election.cc:336] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.206951  3818 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.211642  3818 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.215534 28440 raft_consensus.cc:2468] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f526369c0d0743b192be6e205540a059 in term 16.
I20260709 12:20:29.217018 28956 leader_election.cc:304] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:29.216075  3818 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:29.206064 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.224705 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.225181 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.225986  3761 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:29.227751  3818 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.227502 28185 catalog_manager.cc:5697] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 26 to 27, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 27 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:29.228294  3818 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.230525  3818 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:29.218338  3805 raft_consensus.cc:2804] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:29.232990  3805 raft_consensus.cc:697] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Becoming Leader. State: Replica: f526369c0d0743b192be6e205540a059, State: Running, Role: LEADER
W20260709 12:20:29.231343 28954 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.234081 28954 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.234580 28440 raft_consensus.cc:1275] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:29.234529 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.235545  3805 consensus_queue.cc:237] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.237609  3818 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.238147  3816 consensus_queue.cc:1048] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:29.240257  3817 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:29.240186  3818 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.243793  3818 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:29.251566 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.260524 28404 consensus_peers.cc:597] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:29.255782  3818 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.265626  3818 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.271500  3818 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:29.278353 28954 leader_election.cc:336] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.278987 28954 leader_election.cc:336] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.279793 28954 leader_election.cc:336] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.280187 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:29.281101 28954 leader_election.cc:336] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.281560 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.281421  3816 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:29.282819  3816 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:29.292869 28954 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.294746 28954 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.295243 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.296033  3817 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:29.306699 28954 consensus_peers.cc:597] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:29.310159 28189 catalog_manager.cc:5697] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 reported cstate change: term changed from 15 to 16, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to f526369c0d0743b192be6e205540a059 (127.26.123.4). New cstate: current_term: 16 leader_uuid: "f526369c0d0743b192be6e205540a059" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:29.313473 27116 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 28388)
W20260709 12:20:29.335989 28404 consensus_peers.cc:597] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:29.341465  3745 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.342084  3745 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.344128  3745 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:29.351821 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:29.352507 28983 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 15.
W20260709 12:20:29.354564 28404 leader_election.cc:336] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.355629 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:29.356406  3745 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:29.356766  3745 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.357121  3745 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:29.360921  3805 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:29.361495  3805 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.363628  3805 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:29.369423 28954 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.370061 28954 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.370450 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.371505  3805 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:29.372733  3745 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.374914  3745 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Requested vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:29.377887 28983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdd1e39ef1841c382e814af04ea5bc9" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059"
I20260709 12:20:29.381716 28983 raft_consensus.cc:3060] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:29.389400 28983 raft_consensus.cc:2468] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aea2391841b4888b4e195e5a1f4623b in term 16.
W20260709 12:20:29.381107 28404 leader_election.cc:336] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.392397 28404 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b, f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284
I20260709 12:20:29.394081  3745 raft_consensus.cc:2804] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:29.396013  3745 raft_consensus.cc:697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Becoming Leader. State: Replica: 9aea2391841b4888b4e195e5a1f4623b, State: Running, Role: LEADER
I20260709 12:20:29.396996  3745 consensus_queue.cc:237] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:29.399590 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:29.451754 28189 catalog_manager.cc:5697] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b reported cstate change: term changed from 15 to 16, leader changed from 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2) to 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3). New cstate: current_term: 16 leader_uuid: "9aea2391841b4888b4e195e5a1f4623b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:29.472934 28404 consensus_peers.cc:597] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:29.480141  3761 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:29.483700 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.502130 28404 consensus_peers.cc:597] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.511745 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:29.551604 28954 consensus_peers.cc:597] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:29.558193 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.566160 28404 consensus_peers.cc:597] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.570108 28404 consensus_peers.cc:597] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.593436 28404 consensus_peers.cc:597] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.599900 28404 consensus_peers.cc:597] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.621760 28954 consensus_peers.cc:597] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.637676 28404 consensus_peers.cc:597] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:29.660854 28404 consensus_peers.cc:597] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:29.679412 28440 raft_consensus.cc:1275] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:29.682454  3817 consensus_queue.cc:1048] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:29.687896  3805 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.688463  3805 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:29.699003  3805 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:29.699561 28954 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:29.705596 28954 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:29.706195 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:29.708314  3805 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:29.759766 28438 raft_consensus.cc:1275] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 27 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:29.762809  3804 consensus_queue.cc:1048] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:20:29.791942 28954 consensus_peers.cc:597] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.793704 28954 consensus_peers.cc:597] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:29.803593 28404 consensus_peers.cc:597] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:29.847990 28954 consensus_peers.cc:597] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:29.886106 28983 raft_consensus.cc:1275] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Refusing update from remote peer 9aea2391841b4888b4e195e5a1f4623b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:20:29.948448 28440 raft_consensus.cc:1275] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Refusing update from remote peer f526369c0d0743b192be6e205540a059: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:20:29.950555  3805 consensus_queue.cc:1048] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:29.990900  3747 consensus_queue.cc:1048] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:30.048794 28954 consensus_peers.cc:597] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:30.063319  3804 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.063944  3804 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:30.067112  3804 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:30.178493 28954 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:30.186576 28954 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:30.187184 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.188030  3805 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:30.239780 28404 consensus_peers.cc:597] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:30.240491 28404 consensus_peers.cc:597] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:30.241173 28404 consensus_peers.cc:597] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:30.314347 27116 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 28388)
I20260709 12:20:30.319397  3745 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:30.324122 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:30.326135 28404 consensus_peers.cc:597] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:30.370064 28954 consensus_peers.cc:597] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:30.489673 28954 consensus_peers.cc:597] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:30.509914 28404 consensus_peers.cc:597] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:30.531200 28404 consensus_peers.cc:597] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:30.625165 28954 consensus_peers.cc:597] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:30.687377  3761 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.688596  3761 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:20:30.696008  3761 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:30.697074 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:30.706900 28404 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:30.707808 28404 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.708350 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.709209  3761 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:30.744168  3761 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.745103  3761 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:30.749374  3761 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:30.753924  3761 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.754781  3761 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:30.757264  3761 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:30.762933  3761 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.763902  3761 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:30.772498 28954 consensus_peers.cc:597] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:30.772877  3761 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:30.782374 28404 leader_election.cc:336] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:30.783128 28404 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:30.784528 28404 leader_election.cc:336] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.784997 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.785871  3761 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:30.787201 28404 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.787686 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.788497  3761 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:30.797076 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:30.802341 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.802958 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.803829  3761 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:30.812132  3761 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.812973  3761 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:30.815574  3761 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:30.833231 28404 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:30.836513  3832 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:30.840780 28404 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.841427 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.842319  3761 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:30.845834  3761 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.846645  3761 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:30.849210  3761 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:30.861390 28404 leader_election.cc:336] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:30.862550 28404 leader_election.cc:336] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.863045 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.864114  3761 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:30.874742  3761 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.874907  3804 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.875576  3761 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:30.875881  3804 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:30.878341  3761 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:30.881579  3804 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:30.884848 28954 leader_election.cc:336] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:30.888638 28954 leader_election.cc:336] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.889272 28954 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.890219  3804 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:30.904364  3761 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.905145  3761 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:30.907853  3761 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:30.912462 28404 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:30.920921 28404 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:30.921522 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.922392  3761 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:30.923691 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:30.930608  3761 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.931442  3761 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:30.934624  3747 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.935582  3747 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:20:30.938072  3747 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:30.933625  3761 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:30.939252 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:30.940011 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.940488 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.941269  3761 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:30.950251 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:30.955963 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:30.956620 28404 leader_election.cc:336] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:30.957284 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.957688 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.959344  3761 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:30.960551 28404 leader_election.cc:336] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:30.961014 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:30.961848  3761 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:30.974697  3804 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.975574  3804 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:30.978232  3804 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:30.982379  3804 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.983265  3804 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:30.985721  3804 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:30.987169  3745 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.988015  3745 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
W20260709 12:20:30.988618 28954 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:30.993965  3761 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:30.994870  3761 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:30.997138  3761 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:30.999766 28404 leader_election.cc:336] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:31.001022 28404 leader_election.cc:336] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:31.003563 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.004386  3747 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:31.006479 28954 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:31.007009 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.004563  3745 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:31.008441  3805 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:31.011845 28404 leader_election.cc:336] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:31.008392 28954 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:31.013049 28404 leader_election.cc:336] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:31.013648 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.014497  3745 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:31.019414 28954 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:31.020110 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.021004  3804 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:20:31.038703  3804 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:31.039536  3804 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:31.041960  3804 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:31.046556 28954 consensus_peers.cc:597] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:31.061100 28954 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:31.062327 28954 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:31.062851 28954 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.064155  3804 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:31.113812  3745 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:31.114579  3745 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:31.116946  3745 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:31.126032 28404 leader_election.cc:336] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:31.126881 28404 leader_election.cc:336] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:31.127353 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.128310  3745 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:31.137291 28404 consensus_peers.cc:597] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:20:31.143862  3834 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:31.144721  3834 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:31.147245  3834 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:31.152016 28954 leader_election.cc:336] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:31.152997 28954 leader_election.cc:336] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:31.153479 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.154376  3834 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:31.192382  3834 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:31.193154  3834 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:31.195608  3834 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:31.205317 28954 leader_election.cc:336] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:31.206243 28954 leader_election.cc:336] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:31.206737 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.207568  3834 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:31.235958 28954 consensus_peers.cc:597] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:31.254258 28404 consensus_peers.cc:597] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:31.276826 28954 consensus_peers.cc:597] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:31.315083 27116 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 28388)
I20260709 12:20:31.330729  3834 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:31.331743  3834 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:31.340828  3834 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:31.356772  3834 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:31.357640  3834 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:31.360683  3834 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:31.406085 28404 consensus_peers.cc:597] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:31.409368 28954 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:31.415761 28954 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:31.423794 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.429525  3835 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:31.437867  3747 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:31.443385 28954 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:31.446727 28954 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:31.452658 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:31.453600  3835 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:32.036574 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.037415 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.047868 28404 consensus_peers.cc:597] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:32.090551 28954 consensus_peers.cc:597] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.091697 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:32.092473 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:32.093501 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.094142 28954 consensus_peers.cc:597] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.094658 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:32.095047  3836 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:32.095355 28954 consensus_peers.cc:597] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:32.107102 28404 consensus_peers.cc:597] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.111950 28404 consensus_peers.cc:597] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:32.112756 28404 consensus_peers.cc:597] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:32.113394 28404 consensus_peers.cc:597] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:32.116881 28404 consensus_peers.cc:597] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.122262 28404 consensus_peers.cc:597] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.124660 28404 consensus_peers.cc:597] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.134652 28404 consensus_peers.cc:597] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.210342 28954 consensus_peers.cc:597] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:32.315702 27116 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 28388)
W20260709 12:20:32.393507 28404 consensus_peers.cc:597] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.396245 28404 consensus_peers.cc:597] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.422065 28954 consensus_peers.cc:597] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.448583 28954 consensus_peers.cc:597] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.449388 28954 consensus_peers.cc:597] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:32.453636 28404 consensus_peers.cc:597] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.461386 28404 consensus_peers.cc:597] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:20:32.467495  3836 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:32.468166  3836 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:32.471158  3836 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:32.475670 28404 consensus_peers.cc:597] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:32.484987 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:32.486155 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:32.486661 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:32.487825  3836 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:32.493415  3838 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:32.514981 28404 consensus_peers.cc:597] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.546811 28954 consensus_peers.cc:597] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:32.554807 28404 consensus_peers.cc:597] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.575496 28404 consensus_peers.cc:597] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:32.601339  3839 raft_consensus.cc:493] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:32.602039  3839 raft_consensus.cc:515] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:32.604300  3839 leader_election.cc:290] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:32.613678 28404 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:32.614625 28404 leader_election.cc:336] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:32.615015 28404 leader_election.cc:304] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:32.615834  3839 raft_consensus.cc:2749] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:32.639148 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:32.659701 28221 debug-util.cc:398] Leaking SignalData structure 0x7b0800172920 after lost signal to thread 27119
W20260709 12:20:32.661144 28221 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd340 after lost signal to thread 28224
W20260709 12:20:32.661541 28221 debug-util.cc:398] Leaking SignalData structure 0x7b080037e160 after lost signal to thread 28238
W20260709 12:20:32.663230 28221 debug-util.cc:398] Leaking SignalData structure 0x7b08000fdd20 after lost signal to thread 28465
W20260709 12:20:32.664690 28221 debug-util.cc:398] Leaking SignalData structure 0x7b08000574a0 after lost signal to thread 29015
I20260709 12:20:32.808998  3838 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:32.809955  3838 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:32.812575  3838 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:32.817152  3843 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:32.817873  3843 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:32.820199  3843 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:32.829178 28954 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:32.833371 28954 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:32.836015 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:32.835384 28404 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:32.838603  3843 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:20:32.840871 28404 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:32.842782 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:32.844657  3839 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:32.927608  3838 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:20:32.963799  3843 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:32.964453  3843 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:32.966814  3843 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:32.983385  3843 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:32.984475  3843 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
W20260709 12:20:32.987965 28954 consensus_peers.cc:597] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:32.988708  3836 raft_consensus.cc:493] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:32.993634  3836 raft_consensus.cc:515] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:32.996186  3836 leader_election.cc:290] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:33.020504  3843 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:33.030634 28954 leader_election.cc:336] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:33.034046 28954 leader_election.cc:336] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:33.034597 28954 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:33.041697  3805 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:33.042757 28954 leader_election.cc:336] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:33.046747 28954 leader_election.cc:336] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:33.047345 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:33.048269  3805 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:33.099005 28404 leader_election.cc:336] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:33.100517 28404 leader_election.cc:336] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:33.101094 28404 leader_election.cc:304] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:33.101142 28954 consensus_peers.cc:597] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:33.102470  3838 raft_consensus.cc:2749] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:33.103739 28404 consensus_peers.cc:597] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:33.121261 28404 consensus_peers.cc:597] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:20:33.136459  3838 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:33.137065  3838 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:33.139518  3838 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:33.150996 28404 leader_election.cc:336] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:33.154244 28404 leader_election.cc:336] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:33.154874 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:33.156502  3838 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:33.158529 28954 consensus_peers.cc:597] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:33.274358  3838 raft_consensus.cc:493] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:33.274983  3838 raft_consensus.cc:515] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:33.277402  3838 leader_election.cc:290] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:33.285861 28404 leader_election.cc:336] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:33.286791 28404 leader_election.cc:336] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:33.287228 28404 leader_election.cc:304] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:33.288048  3838 raft_consensus.cc:2749] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:33.316399 27116 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 28388)
I20260709 12:20:33.325802  3838 raft_consensus.cc:493] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:33.326927  3838 raft_consensus.cc:515] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:33.329528  3838 leader_election.cc:290] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:33.338419 28404 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:33.339375 28404 leader_election.cc:336] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:33.339908 28404 leader_election.cc:304] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:33.340775  3838 raft_consensus.cc:2749] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:33.353128  3838 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:33.353857  3838 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:33.354653  3847 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:33.355803  3847 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:33.358083  3847 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:33.360359  3838 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:33.368185 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:33.369168 28404 leader_election.cc:336] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:33.369879 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:33.370285 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
W20260709 12:20:33.371270 28404 leader_election.cc:336] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:33.371812 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:33.372685  3847 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:33.373449  3847 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:33.412252  3847 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:20:33.415364  3805 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:33.415962  3805 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:33.418344  3805 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:33.424772 28954 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:33.427342 28954 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:33.427861 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:33.428699  3805 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:33.449924 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:33.457921 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:33.523952 28404 consensus_peers.cc:597] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:33.594616 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:33.605662 28954 consensus_peers.cc:597] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:33.710525 28954 consensus_peers.cc:597] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:20:33.858290  3838 raft_consensus.cc:493] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:33.858953  3838 raft_consensus.cc:515] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:33.861100  3838 leader_election.cc:290] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:33.866341 28404 leader_election.cc:336] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:33.867296 28404 leader_election.cc:336] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:33.867830 28404 leader_election.cc:304] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:33.868857  3838 raft_consensus.cc:2749] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:33.879917 28404 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111) [suppressed 564 similar messages]
I20260709 12:20:33.948673  3849 raft_consensus.cc:493] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.949548  3849 raft_consensus.cc:515] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:33.993341  3849 leader_election.cc:290] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:34.003038  3851 raft_consensus.cc:493] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:34.068645  3851 raft_consensus.cc:515] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:34.071249  3851 leader_election.cc:290] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:34.138561  3850 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:34.139417  3850 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:34.142410  3850 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:34.274515  3849 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
W20260709 12:20:34.317036 27116 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 28388)
I20260709 12:20:34.275395  3849 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:34.655578  3853 raft_consensus.cc:493] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:34.656450  3853 raft_consensus.cc:515] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:20:34.667757  3853 leader_election.cc:290] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:34.752867  3858 raft_consensus.cc:493] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:34.753710  3858 raft_consensus.cc:515] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:34.766345  3858 leader_election.cc:290] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:34.805590  3860 raft_consensus.cc:493] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:34.899317  3860 raft_consensus.cc:515] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } attrs { replace: true } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } }
I20260709 12:20:34.902293  3860 leader_election.cc:290] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:34.927922  3860 raft_consensus.cc:493] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:34.928706  3860 raft_consensus.cc:515] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:34.931308  3860 leader_election.cc:290] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:34.994949  3860 raft_consensus.cc:493] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f526369c0d0743b192be6e205540a059)
I20260709 12:20:34.995558  3860 raft_consensus.cc:515] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:34.997900  3860 leader_election.cc:290] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), f526369c0d0743b192be6e205540a059 (127.26.123.4:39985)
I20260709 12:20:34.393486  3849 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:35.008705 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab1586b52be49808b90fcc88bdf2d60" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
I20260709 12:20:35.019542  3857 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:20:35.048223 28982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59ee067797b40d396f97683aa7eca81" candidate_uuid: "9aea2391841b4888b4e195e5a1f4623b" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f526369c0d0743b192be6e205540a059" is_pre_election: true
W20260709 12:20:35.049849 28404 consensus_peers.cc:597] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:35.050486 28404 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:35.052023 28404 consensus_peers.cc:597] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:35.052510 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:35.058310 28404 leader_election.cc:336] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:35.059161 28404 leader_election.cc:336] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:35.059628 28404 leader_election.cc:304] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:35.060407  3860 raft_consensus.cc:2749] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:35.061645 28404 leader_election.cc:336] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:35.061966  3849 raft_consensus.cc:493] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:35.062116 28404 leader_election.cc:304] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.062976  3860 raft_consensus.cc:2749] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:35.062681  3849 raft_consensus.cc:515] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:35.064822  3849 leader_election.cc:290] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:35.065626 28404 leader_election.cc:336] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:35.066102 28404 leader_election.cc:304] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.066851  3860 raft_consensus.cc:2749] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:35.071308 28404 leader_election.cc:336] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:35.071787 28404 leader_election.cc:304] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.072558  3860 raft_consensus.cc:2749] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:35.073652 28404 consensus_peers.cc:597] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:35.074453 28404 leader_election.cc:336] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:35.074769 28404 leader_election.cc:304] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, f526369c0d0743b192be6e205540a059
I20260709 12:20:35.075497  3860 raft_consensus.cc:2749] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:35.084769  3843 raft_consensus.cc:493] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.085553  3843 raft_consensus.cc:515] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:35.087774  3843 leader_election.cc:290] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:35.089923 28954 consensus_peers.cc:597] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:35.090497 28954 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:35.094835 28954 leader_election.cc:336] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:35.095299 28954 leader_election.cc:304] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.096084  3849 raft_consensus.cc:2749] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:35.096342 28954 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:35.097013 28954 leader_election.cc:336] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:35.097563 28954 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:35.103468 28954 leader_election.cc:336] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:35.103922 28954 leader_election.cc:304] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.104650  3849 raft_consensus.cc:2749] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:35.105039 28954 leader_election.cc:336] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:35.105499 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.106186  3849 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:35.106395 28954 consensus_peers.cc:597] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.106958 28954 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:35.107306 28954 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.107951  3849 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:35.108338 28954 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:35.110033 28954 leader_election.cc:336] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:35.110414 28954 leader_election.cc:304] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.111074  3849 raft_consensus.cc:2749] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:35.114099 28954 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:35.115212 28954 leader_election.cc:336] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:35.115609 28954 leader_election.cc:304] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.116305  3849 raft_consensus.cc:2749] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:35.184427 28404 consensus_peers.cc:597] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.189533 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.208066 28404 consensus_peers.cc:597] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:35.211565 28404 consensus_peers.cc:597] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.214860 28404 consensus_peers.cc:597] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.217864 28404 consensus_peers.cc:597] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.218905 28404 consensus_peers.cc:597] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:35.252926  3860 raft_consensus.cc:493] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:35.253553  3860 raft_consensus.cc:515] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:35.255885  3860 leader_election.cc:290] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:35.262727 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:35.263481 28404 leader_election.cc:336] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:35.263847 28404 leader_election.cc:304] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.264595  3852 raft_consensus.cc:2749] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:35.294032 28404 consensus_peers.cc:597] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:35.295357 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.298086 28404 consensus_peers.cc:597] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.317705 27116 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 28388)
W20260709 12:20:35.329617 28404 consensus_peers.cc:597] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:35.348150 28404 consensus_peers.cc:597] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:20:35.362620  3852 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:35.367770 28954 consensus_peers.cc:597] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:35.390429 28404 consensus_peers.cc:597] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.428537 28462 debug-util.cc:398] Leaking SignalData structure 0x7b08001cfc40 after lost signal to thread 27119
W20260709 12:20:35.429828 28462 debug-util.cc:398] Leaking SignalData structure 0x7b080019d220 after lost signal to thread 28224
W20260709 12:20:35.430183 28462 debug-util.cc:398] Leaking SignalData structure 0x7b0800209d20 after lost signal to thread 28238
W20260709 12:20:35.430949 28954 consensus_peers.cc:597] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.431519 28462 debug-util.cc:398] Leaking SignalData structure 0x7b08003e47a0 after lost signal to thread 28465
W20260709 12:20:35.432885 28462 debug-util.cc:398] Leaking SignalData structure 0x7b080018b820 after lost signal to thread 29015
W20260709 12:20:35.468876 28954 consensus_peers.cc:597] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.503660 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.517023 28404 consensus_peers.cc:597] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.575497 28954 consensus_peers.cc:597] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.598420 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.598964 28954 consensus_peers.cc:597] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:35.610440 28404 consensus_peers.cc:597] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:20:35.628527  3852 raft_consensus.cc:493] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:35.629197  3852 raft_consensus.cc:515] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:35.634769  3852 leader_election.cc:290] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:35.644972 28404 leader_election.cc:336] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:35.648531 28954 consensus_peers.cc:597] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:35.649269 28404 leader_election.cc:336] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:35.649804 28404 leader_election.cc:304] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:35.650589  3860 raft_consensus.cc:2749] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:35.655082 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:35.657855 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:35.662238 28404 consensus_peers.cc:597] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:35.664610 28404 consensus_peers.cc:597] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.685160 28404 consensus_peers.cc:597] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.719717 28404 consensus_peers.cc:597] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:20:35.810123  3852 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:35.865195 28954 consensus_peers.cc:597] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:35.892791 28954 consensus_peers.cc:597] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:20:36.073354 28404 consensus_peers.cc:597] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:36.163535 28404 consensus_peers.cc:597] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:20:36.181455  3860 raft_consensus.cc:493] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:36.182128  3860 raft_consensus.cc:515] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:36.187412  3860 leader_election.cc:290] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:36.197808 28404 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:36.198948 28404 leader_election.cc:336] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:36.199426 28404 leader_election.cc:304] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:36.200059  3860 raft_consensus.cc:2749] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:36.262957  3860 raft_consensus.cc:493] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:36.263629  3860 raft_consensus.cc:515] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:36.266170  3860 leader_election.cc:290] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:36.272599 28404 leader_election.cc:336] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:36.273471 28404 leader_election.cc:336] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:36.274026 28404 leader_election.cc:304] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:36.274857  3860 raft_consensus.cc:2749] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:36.370174  3860 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:20:36.423166 28954 consensus_peers.cc:597] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:36.443398  3870 raft_consensus.cc:493] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:36.444006  3870 raft_consensus.cc:515] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:36.446429  3870 leader_election.cc:290] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:36.456205 28954 leader_election.cc:336] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:36.457017 28954 leader_election.cc:336] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:36.457507 28954 leader_election.cc:304] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:36.458346  3870 raft_consensus.cc:2749] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:36.471556 28954 consensus_peers.cc:597] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:36.500751  3852 raft_consensus.cc:493] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:36.501412  3852 raft_consensus.cc:515] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } }
I20260709 12:20:36.503764  3852 leader_election.cc:290] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:36.513113 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:36.513949 28404 leader_election.cc:336] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:36.514402 28404 leader_election.cc:304] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aea2391841b4888b4e195e5a1f4623b; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:36.515772  3852 raft_consensus.cc:2749] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:36.543716 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:36.550272 28954 consensus_peers.cc:597] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:36.651558 28404 consensus_peers.cc:597] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:36.664928 28404 consensus_peers.cc:597] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:36.781930 28404 consensus_peers.cc:597] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:20:36.816799 27116 tablet_server.cc:196] TabletServer@127.26.123.2:0 shutdown complete.
W20260709 12:20:36.821801 28954 consensus_peers.cc:597] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:20:36.948580  3872 raft_consensus.cc:1027] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: attempt to promote peer f526369c0d0743b192be6e205540a059: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:20:36.955670  3873 raft_consensus.cc:493] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:36.956480  3873 raft_consensus.cc:515] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:36.957875 27116 tablet_server.cc:179] TabletServer@127.26.123.3:0 shutting down...
I20260709 12:20:36.960567  3873 leader_election.cc:290] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:36.968511 28954 leader_election.cc:336] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:36.969277 28954 leader_election.cc:336] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:36.970045 28954 leader_election.cc:304] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:36.971068  3873 raft_consensus.cc:2749] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:37.006850 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:37.007846 27116 tablet_replica.cc:333] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.008626 27116 raft_consensus.cc:2243] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.009424 27116 raft_consensus.cc:2272] T 39dfaebe2a90498f928f7cd6cc499fac P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.012501 27116 tablet_replica.cc:333] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
W20260709 12:20:37.013128 28956 consensus_peers.cc:597] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:37.013314 27116 raft_consensus.cc:2243] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.013993 27116 raft_consensus.cc:2272] T 6b2edb513f7343d2a2463b94d992b7a3 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.016824 27116 tablet_replica.cc:333] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.017519 27116 raft_consensus.cc:2243] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.018926 27116 raft_consensus.cc:2272] T d8850fe0201a48528da0da1cdac707f0 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.022904 27116 tablet_replica.cc:333] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.023810 27116 raft_consensus.cc:2243] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.024844 27116 raft_consensus.cc:2272] T 97c4af01d02243b2b0727baa5bbacf44 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.026993 27116 tablet_replica.cc:333] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.027637 27116 raft_consensus.cc:2243] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.028604 27116 raft_consensus.cc:2272] T 5235a8de06744285b8c3fe8f9f857223 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.030702 27116 tablet_replica.cc:333] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.031328 27116 raft_consensus.cc:2243] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.031891 27116 raft_consensus.cc:2272] T c0d1d6f8a6a04f6ea4e6f31327e92009 P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.033903 27116 tablet_replica.cc:333] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.034487 27116 raft_consensus.cc:2243] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.035405 27116 raft_consensus.cc:2272] T 8ab1586b52be49808b90fcc88bdf2d60 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.038081 27116 tablet_replica.cc:333] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.038939 27116 raft_consensus.cc:2243] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.040081 27116 raft_consensus.cc:2272] T 8fa1ea35643e43ea8ab0f71a59be9e29 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.042270 27116 tablet_replica.cc:333] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.042865 27116 raft_consensus.cc:2243] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.043391 27116 raft_consensus.cc:2272] T 189fdd6055df4826a42af9c225596e34 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.045231 27116 tablet_replica.cc:333] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.045907 27116 raft_consensus.cc:2243] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.047224 27116 raft_consensus.cc:2272] T 819ca6350ca84ff49e7254638d8755e7 P 9aea2391841b4888b4e195e5a1f4623b [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.050186 27116 tablet_replica.cc:333] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.051036 27116 raft_consensus.cc:2243] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.052376 27116 raft_consensus.cc:2272] T b5bac9d516d84373a10b8812e66d31ed P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.054611 27116 tablet_replica.cc:333] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.055189 27116 raft_consensus.cc:2243] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.055897 27116 raft_consensus.cc:2272] T 916dfa76a5254f63bfc746dbe9c5fde0 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.058066 27116 tablet_replica.cc:333] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.058854 27116 raft_consensus.cc:2243] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.059662 27116 raft_consensus.cc:2272] T 66010729a96e49ebb751b59d4c7f5f3a P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.062590 27116 tablet_replica.cc:333] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.063607 27116 raft_consensus.cc:2243] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.064800 27116 raft_consensus.cc:2272] T 448f610d25494d2c875c5042ec162f71 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.066849 27116 tablet_replica.cc:333] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.067507 27116 raft_consensus.cc:2243] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.068540 27116 raft_consensus.cc:2272] T 705e15df427147839fee08c92bc268dc P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.070597 27116 tablet_replica.cc:333] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.071197 27116 raft_consensus.cc:2243] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.071928 27116 raft_consensus.cc:2272] T 775a5161e47944f5ad46f1616a48b2fe P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.074044 27116 tablet_replica.cc:333] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.074627 27116 raft_consensus.cc:2243] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.075176 27116 raft_consensus.cc:2272] T 8d63db98cb72412194c214fa082eee94 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.077279 27116 tablet_replica.cc:333] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.077893 27116 raft_consensus.cc:2243] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.078568 27116 raft_consensus.cc:2272] T 85a68e4142444e5cb52829e957c1abf7 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.080730 27116 tablet_replica.cc:333] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.081322 27116 raft_consensus.cc:2243] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.082244 27116 raft_consensus.cc:2272] T 61d225f747aa45ff89df6c0ba308faad P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.084695 27116 tablet_replica.cc:333] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.085376 27116 raft_consensus.cc:2243] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.085942 27116 raft_consensus.cc:2272] T 54211b8ac4fe44c59e9f0338d9960177 P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.088555 27116 tablet_replica.cc:333] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.089318 27116 raft_consensus.cc:2243] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.089975 27116 raft_consensus.cc:2272] T 610cd41ac66c4074be8e2aa7b22d5e6c P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.092419 27116 tablet_replica.cc:333] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.093081 27116 raft_consensus.cc:2243] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.094345 27116 raft_consensus.cc:2272] T b0d8e709ec114a04bfd86247e24dd63c P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.097373 27116 tablet_replica.cc:333] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.098272 27116 raft_consensus.cc:2243] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.098919 27116 raft_consensus.cc:2272] T bea7beef1d4c49159d75e5f61eaaeb5a P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:37.100312 28954 consensus_peers.cc:597] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:37.101109 27116 tablet_replica.cc:333] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.101830 27116 raft_consensus.cc:2243] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.102378 27116 raft_consensus.cc:2272] T 2e3e7c85ff90423aa9f9d234d992379b P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.104892 27116 tablet_replica.cc:333] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.105576 27116 raft_consensus.cc:2243] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.106235 27116 raft_consensus.cc:2272] T 6c8442db4dc44c619fd67179aa212063 P 9aea2391841b4888b4e195e5a1f4623b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.108260 27116 tablet_replica.cc:333] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.108853 27116 raft_consensus.cc:2243] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.109370 27116 raft_consensus.cc:2272] T d6fd84f874b14098badd4d77b5703109 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.111136  3873 raft_consensus.cc:493] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:37.111481 27116 tablet_replica.cc:333] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.111646  3873 raft_consensus.cc:515] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:37.112212 27116 raft_consensus.cc:2243] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.112812 27116 raft_consensus.cc:2272] T 7f380821c5264bfc8a5f0bf36bc52fd4 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.113715  3873 leader_election.cc:290] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
I20260709 12:20:37.115484 27116 tablet_replica.cc:333] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.116225 27116 raft_consensus.cc:2243] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.117369 27116 raft_consensus.cc:2272] T 5fb143bfa1d44653b638d40442c7e955 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:37.119390 28956 consensus_peers.cc:597] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:37.119902 27116 tablet_replica.cc:333] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
W20260709 12:20:37.120476 28954 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:37.120630 27116 raft_consensus.cc:2243] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.122216 27116 raft_consensus.cc:2272] T 014b394597e7430489bd093d8616dcad P 9aea2391841b4888b4e195e5a1f4623b [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:37.122252 28954 leader_election.cc:336] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:37.122831 28954 leader_election.cc:304] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:37.123586  3873 raft_consensus.cc:2749] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:37.125294 27116 tablet_replica.cc:333] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.125999 27116 raft_consensus.cc:2243] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.126940 27116 raft_consensus.cc:2272] T 956fb13fa38243fd862789f3046aba97 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.128989 27116 tablet_replica.cc:333] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.129586 27116 raft_consensus.cc:2243] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.130774 27116 raft_consensus.cc:2272] T 9cdd1e39ef1841c382e814af04ea5bc9 P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.133603 27116 tablet_replica.cc:333] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.134531 27116 raft_consensus.cc:2243] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.135268 27116 raft_consensus.cc:2272] T 23a4c78997e24df7bce95fcf6d70c95c P 9aea2391841b4888b4e195e5a1f4623b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.138149 27116 tablet_replica.cc:333] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.138924 27116 raft_consensus.cc:2243] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.140247 27116 raft_consensus.cc:2272] T c59ee067797b40d396f97683aa7eca81 P 9aea2391841b4888b4e195e5a1f4623b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.143147 27116 tablet_replica.cc:333] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.143874 27116 raft_consensus.cc:2243] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Raft consensus shutting down.
W20260709 12:20:37.143950 28956 consensus_peers.cc:597] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:37.145377 27116 pending_rounds.cc:70] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Trying to abort 1 pending ops.
I20260709 12:20:37.145702 27116 pending_rounds.cc:77] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b: Aborting op as it isn't in flight: id { term: 16 index: 11 } timestamp: 7305624072076292096 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "986f172991d84af28d4f5cbe8c517e6e" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } } new_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } } }
I20260709 12:20:37.146878 27116 raft_consensus.cc:2917] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Aborting config change with OpId 16.11: Aborted: Op aborted
I20260709 12:20:37.147336 27116 raft_consensus.cc:2889] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:20:37.147790 27116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:33084 (request call id 6) took 10177 ms. Trace:
I20260709 12:20:37.148051 27116 rpcz_store.cc:276] 0709 12:20:26.970682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.970843 (+   161us) service_pool.cc:224] Handling call
0709 12:20:26.976911 (+  6068us) log.cc:844] Serialized 499 byte log entry
0709 12:20:27.018163 (+ 41252us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b5400210aa0. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85be7f0571 00007f85be800142 00007f85be7e0eaa 00007f85be7e7d53 00007f85be7e7cd9 00007f85be7e7c69 00007f85be7e7c31 00007f85be7e6dfd 0000000000388634 0000000000388479 00007f85bb79a475
0709 12:20:37.147723 (+10129560us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":604,"raft.run_cpu_time_us":3799,"raft.run_wall_time_us":24053,"spinlock_wait_cycles":3226240,"thread_start_us":26478,"threads_started":1,"wal-append.queue_time_us":61195,"wal-append.run_cpu_time_us":1747,"wal-append.run_wall_time_us":1005331}
I20260709 12:20:37.149096 27116 raft_consensus.cc:2272] T 986f172991d84af28d4f5cbe8c517e6e P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:37.150233 28238 auto_rebalancer.cc:629] Failed to schedule move for tablet 986f172991d84af28d4f5cbe8c517e6e: Aborted: Op aborted
I20260709 12:20:37.151618 27116 tablet_replica.cc:333] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.152266 27116 raft_consensus.cc:2243] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.153815 27116 raft_consensus.cc:2272] T e77b6d26dd064626ab3b950e5a6c272f P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.155803 27116 tablet_replica.cc:333] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.156401 27116 raft_consensus.cc:2243] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.157249 27116 raft_consensus.cc:2272] T 255e0140f2a64433a6c2dd64fe24e6b3 P 9aea2391841b4888b4e195e5a1f4623b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.159503 27116 tablet_replica.cc:333] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.160077 27116 raft_consensus.cc:2243] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.161288 27116 raft_consensus.cc:2272] T 4ac0d46a7b244f2494d81e6667cfa1bf P 9aea2391841b4888b4e195e5a1f4623b [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.163967 27116 tablet_replica.cc:333] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.164747 27116 raft_consensus.cc:2243] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.166230 27116 raft_consensus.cc:2272] T 65569448c1eb448799671a8f4ba82cd4 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.169179 27116 tablet_replica.cc:333] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.170008 27116 raft_consensus.cc:2243] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.170681 27116 raft_consensus.cc:2272] T 1fc9bc911cb145b0a259ee6481eefdf6 P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:37.172879 28956 consensus_peers.cc:597] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:37.173465 27116 tablet_replica.cc:333] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.174120 27116 raft_consensus.cc:2243] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.175781 27116 raft_consensus.cc:2272] T bce32c3c49414e228fe6d912d95aeaac P 9aea2391841b4888b4e195e5a1f4623b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.177986 27116 tablet_replica.cc:333] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.178555 27116 raft_consensus.cc:2243] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.179049 27116 raft_consensus.cc:2272] T 72871f9ecbeb417fa5bd39d4714f0a90 P 9aea2391841b4888b4e195e5a1f4623b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.181555 27116 tablet_replica.cc:333] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.182395 27116 raft_consensus.cc:2243] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.183698 27116 raft_consensus.cc:2272] T 9c99f01ab7374acfa37f7669b8fcdeb2 P 9aea2391841b4888b4e195e5a1f4623b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.186606 27116 tablet_replica.cc:333] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.187182 27116 raft_consensus.cc:2243] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.188336 27116 raft_consensus.cc:2272] T 582bd3fd6d214e92aefb0baa58c76aed P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.191011 27116 tablet_replica.cc:333] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.191665 27116 raft_consensus.cc:2243] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 12:20:37.192262 28956 consensus_peers.cc:597] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:37.193230 27116 raft_consensus.cc:2272] T 3ee4696ba90a447fa07c4d09407217f6 P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:37.195868 28956 consensus_peers.cc:597] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:37.196058 27116 tablet_replica.cc:333] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.196779 27116 raft_consensus.cc:2243] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.198235 27116 pending_rounds.cc:70] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Trying to abort 1 pending ops.
I20260709 12:20:37.198494 27116 pending_rounds.cc:77] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305624078759874560 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" old_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } } new_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } } }
I20260709 12:20:37.199326 27116 raft_consensus.cc:2917] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: Aborting config change with OpId 19.14: Aborted: Op aborted
I20260709 12:20:37.199712 27116 raft_consensus.cc:2889] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:20:37.200011 27116 raft_consensus.cc:2272] T 21eeb096d8a4423bbc24e8935e0539fc P 9aea2391841b4888b4e195e5a1f4623b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.202471 27116 tablet_replica.cc:333] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.203042 27116 raft_consensus.cc:2243] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:37.204226 27116 raft_consensus.cc:2272] T c474098832a04762b58521bdc2a92b5a P 9aea2391841b4888b4e195e5a1f4623b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:37.206583 27116 tablet_replica.cc:333] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b: stopping tablet replica
I20260709 12:20:37.207090 27116 raft_consensus.cc:2243] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:37.208252 27116 raft_consensus.cc:2272] T 23d6b97dd9e9466d9581f6088e681a11 P 9aea2391841b4888b4e195e5a1f4623b [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:37.229965 28954 consensus_peers.cc:597] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:37.255622 28956 consensus_peers.cc:597] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:37.333530 28956 consensus_peers.cc:597] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:37.351528 28954 consensus_peers.cc:597] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:20:37.352982 28956 consensus_peers.cc:597] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:37.358894 28956 consensus_peers.cc:597] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:37.361406 28956 consensus_peers.cc:597] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:37.386889 28956 consensus_peers.cc:597] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:37.392539 28956 consensus_peers.cc:597] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:37.408649 28956 consensus_peers.cc:597] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:37.483956 28956 consensus_peers.cc:597] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 -> Peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Couldn't send request to peer 9aea2391841b4888b4e195e5a1f4623b. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:37.500378  3873 raft_consensus.cc:493] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:37.501125  3873 raft_consensus.cc:515] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:37.503659  3873 leader_election.cc:290] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:37.507848 28954 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:37.511145 28954 leader_election.cc:336] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:37.512550 28954 leader_election.cc:304] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:37.520979  3873 raft_consensus.cc:2749] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:20:37.797650 28954 consensus_peers.cc:597] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:37.919059 28954 consensus_peers.cc:597] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:20:37.937566 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:20:37.969221 28954 consensus_peers.cc:597] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:38.049592  3870 raft_consensus.cc:493] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:38.050273  3870 raft_consensus.cc:515] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.052506  3870 leader_election.cc:290] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:38.058887 28954 leader_election.cc:336] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:38.059630 28954 leader_election.cc:336] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:38.060030 28954 leader_election.cc:304] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:38.060976  3870 raft_consensus.cc:2749] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:38.087394 28954 consensus_peers.cc:597] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:38.090955 27116 tablet_server.cc:196] TabletServer@127.26.123.3:0 shutdown complete.
W20260709 12:20:38.091653 28954 consensus_peers.cc:597] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 -> Peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Couldn't send request to peer a3de34b6f4cf4606a61944db6e2739c8. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:20:38.150475  3870 raft_consensus.cc:493] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:38.151340  3870 raft_consensus.cc:515] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.154090  3870 leader_election.cc:290] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:38.159647 28956 leader_election.cc:336] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111)
W20260709 12:20:38.162019 28954 leader_election.cc:336] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:38.162645 28954 leader_election.cc:304] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:38.163498  3870 raft_consensus.cc:2749] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:38.197190 27116 tablet_server.cc:179] TabletServer@127.26.123.4:0 shutting down...
I20260709 12:20:38.207373  3873 raft_consensus.cc:493] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:38.208081  3873 raft_consensus.cc:515] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.210381  3873 leader_election.cc:290] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:38.219396 28954 leader_election.cc:336] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:38.220048 28956 leader_election.cc:336] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111)
I20260709 12:20:38.220584 28956 leader_election.cc:304] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:38.221429  3873 raft_consensus.cc:2749] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:38.223489  3870 raft_consensus.cc:493] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6634fa3fbd594726a5ec3c1686900284)
I20260709 12:20:38.224046  3870 raft_consensus.cc:515] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.227993  3870 leader_election.cc:290] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701)
W20260709 12:20:38.228761 28954 consensus_peers.cc:597] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:20:38.230129  3870 raft_consensus.cc:493] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:38.230736  3870 raft_consensus.cc:515] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.233466  3870 leader_election.cc:290] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:38.243575 28956 leader_election.cc:336] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111)
W20260709 12:20:38.249513 28954 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:38.253021 28954 leader_election.cc:336] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:38.253662 28954 leader_election.cc:304] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:38.254557  3870 raft_consensus.cc:2749] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:38.255270  3870 raft_consensus.cc:493] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:38.255736  3870 raft_consensus.cc:515] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.258044  3870 leader_election.cc:290] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
W20260709 12:20:38.259660 28954 leader_election.cc:336] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:38.260293 28954 leader_election.cc:304] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:38.261210  3870 raft_consensus.cc:2749] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:38.262261 28954 consensus_peers.cc:597] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:38.267818 28954 leader_election.cc:336] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
W20260709 12:20:38.268730 28956 leader_election.cc:336] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111)
I20260709 12:20:38.269942 28956 leader_election.cc:304] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:38.270121 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:38.270933  3873 raft_consensus.cc:493] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:38.271215 27116 tablet_replica.cc:333] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.271559  3873 raft_consensus.cc:515] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.273622 27116 raft_consensus.cc:2243] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.272940  3870 raft_consensus.cc:2749] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:38.274583 27116 pending_rounds.cc:70] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: Trying to abort 1 pending ops.
I20260709 12:20:38.275520  3873 leader_election.cc:290] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:38.274897 27116 pending_rounds.cc:77] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305624078759874560 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "21eeb096d8a4423bbc24e8935e0539fc" old_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } } new_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } attrs { replace: true } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } } } }
I20260709 12:20:38.276091 27116 raft_consensus.cc:2917] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Aborting config change with OpId 19.14: Aborted: Op aborted
I20260709 12:20:38.276574 27116 raft_consensus.cc:2889] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:20:38.277127 27116 raft_consensus.cc:2272] T 21eeb096d8a4423bbc24e8935e0539fc P f526369c0d0743b192be6e205540a059 [term 19 LEARNER]: Raft consensus is shut down!
W20260709 12:20:38.280170 28954 leader_election.cc:336] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:38.280614 27116 tablet_replica.cc:333] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.281864 27116 raft_consensus.cc:2243] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.281829  3873 raft_consensus.cc:493] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
W20260709 12:20:38.282717 28956 leader_election.cc:336] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111)
I20260709 12:20:38.283233 27116 raft_consensus.cc:2272] T 9c99f01ab7374acfa37f7669b8fcdeb2 P f526369c0d0743b192be6e205540a059 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.282738  3873 raft_consensus.cc:515] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.283407 28956 leader_election.cc:304] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:38.284308  3870 raft_consensus.cc:2749] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:38.285770  3873 leader_election.cc:290] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:38.286957 27116 tablet_replica.cc:333] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.288033 27116 raft_consensus.cc:2243] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.288808 27116 raft_consensus.cc:2272] T 9cdd1e39ef1841c382e814af04ea5bc9 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:38.292349 28956 leader_election.cc:336] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111)
I20260709 12:20:38.292618 27116 tablet_replica.cc:333] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059: stopping tablet replica
W20260709 12:20:38.293005 28954 leader_election.cc:336] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111)
I20260709 12:20:38.293387  3873 raft_consensus.cc:493] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:38.293540 28954 leader_election.cc:304] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 6634fa3fbd594726a5ec3c1686900284, 9aea2391841b4888b4e195e5a1f4623b
I20260709 12:20:38.293802 27116 raft_consensus.cc:2243] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.294512  3870 raft_consensus.cc:2749] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:38.294008  3873 raft_consensus.cc:515] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.295842 27116 raft_consensus.cc:2272] T c474098832a04762b58521bdc2a92b5a P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.298167  3873 leader_election.cc:290] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:38.299922 27116 tablet_replica.cc:333] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.300927 27116 raft_consensus.cc:2243] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.302572 27116 raft_consensus.cc:2272] T 8ab1586b52be49808b90fcc88bdf2d60 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:38.306201 28956 leader_election.cc:336] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111)
I20260709 12:20:38.306345 27116 tablet_replica.cc:333] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059: stopping tablet replica
W20260709 12:20:38.306245 28954 consensus_peers.cc:597] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 -> Peer 6634fa3fbd594726a5ec3c1686900284 (127.26.123.2:44701): Couldn't send request to peer 6634fa3fbd594726a5ec3c1686900284. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:44701: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:20:38.307312 28954 leader_election.cc:336] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
I20260709 12:20:38.307588 27116 raft_consensus.cc:2243] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.307801 28954 leader_election.cc:304] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:38.308835 27116 raft_consensus.cc:2272] T d68008d2b3fd4f7bb72bf69f612b2998 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.309695  3873 raft_consensus.cc:2749] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:38.312629 27116 tablet_replica.cc:333] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.313483 27116 raft_consensus.cc:2243] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.314244 27116 raft_consensus.cc:2272] T 8e6bd3eccb6c4f4c8fedbc76fa829a96 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.317114 27116 tablet_replica.cc:333] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.317950 27116 raft_consensus.cc:2243] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.319048 27116 raft_consensus.cc:2272] T 5235a8de06744285b8c3fe8f9f857223 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.321818 27116 tablet_replica.cc:333] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.322490 27116 raft_consensus.cc:2243] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.323382 27116 raft_consensus.cc:2272] T b4036b67205e41c597049a10a71e9467 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.327314 27116 tablet_replica.cc:333] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.328372 27116 raft_consensus.cc:2243] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.329520 27116 raft_consensus.cc:2272] T 956fb13fa38243fd862789f3046aba97 P f526369c0d0743b192be6e205540a059 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.332849 27116 tablet_replica.cc:333] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.333565 27116 raft_consensus.cc:2243] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.334275 27116 raft_consensus.cc:2272] T 114fa146124747358c997ac1008e1b93 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.337518 27116 tablet_replica.cc:333] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.338435 27116 raft_consensus.cc:2243] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.339792 27116 raft_consensus.cc:2272] T 61d225f747aa45ff89df6c0ba308faad P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.342355 27116 tablet_replica.cc:333] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.343005 27116 raft_consensus.cc:2243] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.343626 27116 raft_consensus.cc:2272] T 819ca6350ca84ff49e7254638d8755e7 P f526369c0d0743b192be6e205540a059 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.346728 27116 tablet_replica.cc:333] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.347623 27116 raft_consensus.cc:2243] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.349057 27116 raft_consensus.cc:2272] T cdc4169a890e44e3ad14a98135e7071e P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.351754 27116 tablet_replica.cc:333] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.352414 27116 raft_consensus.cc:2243] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.353328 27116 raft_consensus.cc:2272] T 3ee4696ba90a447fa07c4d09407217f6 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.355749 27116 tablet_replica.cc:333] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.356372 27116 raft_consensus.cc:2243] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.357012 27116 raft_consensus.cc:2272] T ab6d72f1fb474adfafdd159090dc63b5 P f526369c0d0743b192be6e205540a059 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.360000 27116 tablet_replica.cc:333] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.360651 27116 raft_consensus.cc:2243] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.361711 27116 raft_consensus.cc:2272] T 66010729a96e49ebb751b59d4c7f5f3a P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.364055 27116 tablet_replica.cc:333] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.364724 27116 raft_consensus.cc:2243] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.365293 27116 raft_consensus.cc:2272] T 448f610d25494d2c875c5042ec162f71 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.367726 27116 tablet_replica.cc:333] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.368381 27116 raft_consensus.cc:2243] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.369306 27116 raft_consensus.cc:2272] T 2e3e7c85ff90423aa9f9d234d992379b P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.371969 27116 tablet_replica.cc:333] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.372602 27116 raft_consensus.cc:2243] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.373761 27116 raft_consensus.cc:2272] T bea7beef1d4c49159d75e5f61eaaeb5a P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.376489 27116 tablet_replica.cc:333] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.377223 27116 raft_consensus.cc:2243] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.378249 27116 raft_consensus.cc:2272] T 8d63db98cb72412194c214fa082eee94 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.380494 27116 tablet_replica.cc:333] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.381098 27116 raft_consensus.cc:2243] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.382087 27116 raft_consensus.cc:2272] T 6c8442db4dc44c619fd67179aa212063 P f526369c0d0743b192be6e205540a059 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.384423 27116 tablet_replica.cc:333] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.385054 27116 raft_consensus.cc:2243] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.385596 27116 raft_consensus.cc:2272] T 1410767867a5470e81621556e44c6560 P f526369c0d0743b192be6e205540a059 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.387909 27116 tablet_replica.cc:333] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.388567 27116 raft_consensus.cc:2243] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.389086 27116 raft_consensus.cc:2272] T 8fa1ea35643e43ea8ab0f71a59be9e29 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.391892 27116 tablet_replica.cc:333] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.392724 27116 raft_consensus.cc:2243] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.393489 27116 raft_consensus.cc:2272] T e2ac8850682043e5bf8caf7fc64fae05 P f526369c0d0743b192be6e205540a059 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.396341 27116 tablet_replica.cc:333] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.397027 27116 raft_consensus.cc:2243] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Raft consensus shutting down.
I20260709 12:20:38.397506 27116 pending_rounds.cc:70] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: Trying to abort 1 pending ops.
I20260709 12:20:38.397693 27116 pending_rounds.cc:77] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059: Aborting op as it isn't in flight: id { term: 16 index: 11 } timestamp: 7305624072076292096 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "986f172991d84af28d4f5cbe8c517e6e" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } } new_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } attrs { replace: true } } peers { permanent_uuid: "6634fa3fbd594726a5ec3c1686900284" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44701 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: NON_VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: true } } } }
I20260709 12:20:38.398499 27116 raft_consensus.cc:2917] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059 [term 16 LEARNER]: Aborting config change with OpId 16.11: Aborted: Op aborted
I20260709 12:20:38.398823 27116 raft_consensus.cc:2889] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059 [term 16 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:20:38.399089 27116 raft_consensus.cc:2272] T 986f172991d84af28d4f5cbe8c517e6e P f526369c0d0743b192be6e205540a059 [term 16 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 12:20:38.401525 27116 tablet_replica.cc:333] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.402209 27116 raft_consensus.cc:2243] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.402838 27116 raft_consensus.cc:2272] T 23f7ed53101a4cc3b4e19149d5a4a1a8 P f526369c0d0743b192be6e205540a059 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.405216 27116 tablet_replica.cc:333] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.406028 27116 raft_consensus.cc:2243] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.407141 27116 raft_consensus.cc:2272] T c59ee067797b40d396f97683aa7eca81 P f526369c0d0743b192be6e205540a059 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.409722 27116 tablet_replica.cc:333] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.410368 27116 raft_consensus.cc:2243] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.410858 27116 raft_consensus.cc:2272] T 5fb143bfa1d44653b638d40442c7e955 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.413095 27116 tablet_replica.cc:333] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.413661 27116 raft_consensus.cc:2243] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.414692 27116 raft_consensus.cc:2272] T d6fd84f874b14098badd4d77b5703109 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.416782 27116 tablet_replica.cc:333] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.417323 27116 raft_consensus.cc:2243] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.418391 27116 raft_consensus.cc:2272] T 189fdd6055df4826a42af9c225596e34 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.420823 27116 tablet_replica.cc:333] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.420740  3873 raft_consensus.cc:493] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aea2391841b4888b4e195e5a1f4623b)
I20260709 12:20:38.421608 27116 raft_consensus.cc:2243] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.421461  3873 raft_consensus.cc:515] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a3de34b6f4cf4606a61944db6e2739c8" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 42113 } } peers { permanent_uuid: "9aea2391841b4888b4e195e5a1f4623b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 33721 } } peers { permanent_uuid: "f526369c0d0743b192be6e205540a059" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 39985 } attrs { promote: false } }
I20260709 12:20:38.422716 27116 raft_consensus.cc:2272] T 775a5161e47944f5ad46f1616a48b2fe P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.424037  3873 leader_election.cc:290] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113), 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721)
I20260709 12:20:38.425196 27116 tablet_replica.cc:333] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.426060 27116 raft_consensus.cc:2243] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.427385 27116 raft_consensus.cc:2272] T 255e0140f2a64433a6c2dd64fe24e6b3 P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:38.429212 28954 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a3de34b6f4cf4606a61944db6e2739c8 (127.26.123.1:42113): Network error: Client connection negotiation failed: client connection to 127.26.123.1:42113: connect: Connection refused (error 111)
W20260709 12:20:38.429785 28956 leader_election.cc:336] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 9aea2391841b4888b4e195e5a1f4623b (127.26.123.3:33721): Network error: Client connection negotiation failed: client connection to 127.26.123.3:33721: connect: Connection refused (error 111)
I20260709 12:20:38.430203 27116 tablet_replica.cc:333] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.430244 28956 leader_election.cc:304] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f526369c0d0743b192be6e205540a059; no voters: 9aea2391841b4888b4e195e5a1f4623b, a3de34b6f4cf4606a61944db6e2739c8
I20260709 12:20:38.431180  3873 raft_consensus.cc:2749] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:38.431335 27116 raft_consensus.cc:2243] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.432097 27116 raft_consensus.cc:2272] T e77b6d26dd064626ab3b950e5a6c272f P f526369c0d0743b192be6e205540a059 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.434440 27116 tablet_replica.cc:333] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.435087 27116 raft_consensus.cc:2243] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.436247 27116 raft_consensus.cc:2272] T 9cb896260dae4cf6bb09ba7728c603f3 P f526369c0d0743b192be6e205540a059 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.439111 27116 tablet_replica.cc:333] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.439808 27116 raft_consensus.cc:2243] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.440344 27116 raft_consensus.cc:2272] T 23d6b97dd9e9466d9581f6088e681a11 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.442844 27116 tablet_replica.cc:333] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.443421 27116 raft_consensus.cc:2243] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.443887 27116 raft_consensus.cc:2272] T 9a388b90c9b140649cada36c135d7080 P f526369c0d0743b192be6e205540a059 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.446224 27116 tablet_replica.cc:333] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.446794 27116 raft_consensus.cc:2243] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.447559 27116 raft_consensus.cc:2272] T bce32c3c49414e228fe6d912d95aeaac P f526369c0d0743b192be6e205540a059 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.450258 27116 tablet_replica.cc:333] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059: stopping tablet replica
I20260709 12:20:38.450781 27116 raft_consensus.cc:2243] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:38.451339 27116 raft_consensus.cc:2272] T 777967d6330c4654813298b12226a560 P f526369c0d0743b192be6e205540a059 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.809361 27116 tablet_server.cc:196] TabletServer@127.26.123.4:0 shutdown complete.
I20260709 12:20:38.899886 27116 master.cc:562] Master@127.26.123.62:36149 shutting down...
W20260709 12:20:38.935413 28238 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:20:38.935788 28238 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:20:38.936009 28238 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 12:20:38.939774 27116 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:38.940478 27116 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:38.940806 27116 tablet_replica.cc:333] T 00000000000000000000000000000000 P b5f297f7df3d4c2da68fabf56f1654bd: stopping tablet replica
I20260709 12:20:39.130191 27116 master.cc:584] Master@127.26.123.62:36149 shutdown complete.
I20260709 12:20:39.171741 27116 test_util.cc:182] -----------------------------------------------
I20260709 12:20:39.172005 27116 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599329322122-27116-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (249238 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:20:39.175771 27116 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.123.62:34017
I20260709 12:20:39.177047 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:39.183269  3878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:39.183537  3879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:39.184026  3881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:39.185609 27116 server_base.cc:1061] running on GCE node
I20260709 12:20:39.189085 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:39.189307 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:39.189421 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599639189410 us; error 0 us; skew 500 ppm
I20260709 12:20:39.189993 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:39.192559 27116 webserver.cc:533] Webserver started at http://127.26.123.62:34029/ using document root <none> and password file <none>
I20260709 12:20:39.193063 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:39.193224 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:39.193454 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:39.194849 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/master-0-root/instance:
uuid: "24e095ac04c346e8bc5cdb69e7f59d95"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.199708 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:20:39.203398  3886 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.204180 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:20:39.204500 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/master-0-root
uuid: "24e095ac04c346e8bc5cdb69e7f59d95"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.204782 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:39.214397 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:39.215878 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:39.261616 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.62:34017
I20260709 12:20:39.261766  3937 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.62:34017 every 8 connection(s)
I20260709 12:20:39.266202  3938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:39.279105  3938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95: Bootstrap starting.
I20260709 12:20:39.284425  3938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.289324  3938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95: No bootstrap required, opened a new log
I20260709 12:20:39.291946  3938 raft_consensus.cc:359] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e095ac04c346e8bc5cdb69e7f59d95" member_type: VOTER }
I20260709 12:20:39.292389  3938 raft_consensus.cc:385] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.292663  3938 raft_consensus.cc:740] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24e095ac04c346e8bc5cdb69e7f59d95, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.293305  3938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e095ac04c346e8bc5cdb69e7f59d95" member_type: VOTER }
I20260709 12:20:39.293876  3938 raft_consensus.cc:399] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:39.294111  3938 raft_consensus.cc:493] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:39.294396  3938 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.300942  3938 raft_consensus.cc:515] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e095ac04c346e8bc5cdb69e7f59d95" member_type: VOTER }
I20260709 12:20:39.301664  3938 leader_election.cc:304] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24e095ac04c346e8bc5cdb69e7f59d95; no voters: 
I20260709 12:20:39.303074  3938 leader_election.cc:290] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:39.303422  3941 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.304934  3941 raft_consensus.cc:697] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 1 LEADER]: Becoming Leader. State: Replica: 24e095ac04c346e8bc5cdb69e7f59d95, State: Running, Role: LEADER
I20260709 12:20:39.305634  3941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e095ac04c346e8bc5cdb69e7f59d95" member_type: VOTER }
I20260709 12:20:39.306574  3938 sys_catalog.cc:565] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:39.310437  3943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24e095ac04c346e8bc5cdb69e7f59d95. Latest consensus state: current_term: 1 leader_uuid: "24e095ac04c346e8bc5cdb69e7f59d95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e095ac04c346e8bc5cdb69e7f59d95" member_type: VOTER } }
I20260709 12:20:39.311208  3943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:39.310369  3942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24e095ac04c346e8bc5cdb69e7f59d95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24e095ac04c346e8bc5cdb69e7f59d95" member_type: VOTER } }
I20260709 12:20:39.312347  3942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:39.315632  3947 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:39.321936  3947 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:39.324661 27116 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:39.332131  3947 catalog_manager.cc:1383] Generated new cluster ID: cd378d0aa38941cdaccba0c3524f89ed
I20260709 12:20:39.332489  3947 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:39.362591  3947 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:39.364094  3947 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:39.382157  3947 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95: Generated new TSK 0
I20260709 12:20:39.382893  3947 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:39.391494 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:39.398401  3959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:39.399827  3960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:39.403365  3962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:39.403450 27116 server_base.cc:1061] running on GCE node
I20260709 12:20:39.406019 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:39.406255 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:39.406440 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599639406417 us; error 0 us; skew 500 ppm
I20260709 12:20:39.407111 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:39.409822 27116 webserver.cc:533] Webserver started at http://127.26.123.1:42587/ using document root <none> and password file <none>
I20260709 12:20:39.410351 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:39.410547 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:39.410816 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:39.412132 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/instance:
uuid: "41009fde51f045c49be6461b76a27543"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.417269 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:20:39.421139  3967 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.422001 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:20:39.422308 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "41009fde51f045c49be6461b76a27543"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.422610 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:39.438427 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:39.439782 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:39.441407 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:39.443979 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:39.444185 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.444418 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:39.444610 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.492309 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:41077
I20260709 12:20:39.492450  4029 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:41077 every 8 connection(s)
I20260709 12:20:39.498008 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:39.506054  4034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:39.507936  4035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:39.510170  4037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:39.510394 27116 server_base.cc:1061] running on GCE node
I20260709 12:20:39.513022 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:39.513314 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:39.513532 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599639513499 us; error 0 us; skew 500 ppm
I20260709 12:20:39.514170 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:39.516885  4030 heartbeater.cc:344] Connected to a master server at 127.26.123.62:34017
I20260709 12:20:39.517313 27116 webserver.cc:533] Webserver started at http://127.26.123.2:35271/ using document root <none> and password file <none>
I20260709 12:20:39.517328  4030 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:39.518029 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:39.518278 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:39.518427  4030 heartbeater.cc:507] Master 127.26.123.62:34017 requested a full tablet report, sending...
I20260709 12:20:39.518625 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:39.520129 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/instance:
uuid: "25902f00fb5d41679245f23dd19654aa"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.521163  3903 ts_manager.cc:194] Registered new tserver with Master: 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:39.523952  3903 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48752
I20260709 12:20:39.528098 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.001s	sys 0.008s
I20260709 12:20:39.532198  4042 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.533026 27116 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:20:39.533329 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-1-root
uuid: "25902f00fb5d41679245f23dd19654aa"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.533612 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:39.553036 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:39.554422 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:39.556063 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:39.559031 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:39.559286 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.559561 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:39.559720 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.610606 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.2:43667
I20260709 12:20:39.610787  4104 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.2:43667 every 8 connection(s)
I20260709 12:20:39.615676 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:39.623463  4108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:39.624847  4109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:39.627755  4111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:39.628924  4105 heartbeater.cc:344] Connected to a master server at 127.26.123.62:34017
I20260709 12:20:39.629014 27116 server_base.cc:1061] running on GCE node
I20260709 12:20:39.629590  4105 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:39.630359 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:39.630700 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:39.630656  4105 heartbeater.cc:507] Master 127.26.123.62:34017 requested a full tablet report, sending...
I20260709 12:20:39.631085 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599639631062 us; error 0 us; skew 500 ppm
I20260709 12:20:39.631897 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:39.633389  3903 ts_manager.cc:194] Registered new tserver with Master: 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:39.634814 27116 webserver.cc:533] Webserver started at http://127.26.123.3:44283/ using document root <none> and password file <none>
I20260709 12:20:39.635529 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:39.635510  3903 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48754
I20260709 12:20:39.635967 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:39.636348 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:39.638026 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/instance:
uuid: "6994df41341e478d89f7c105f550ff93"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.642946 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:20:39.646663  4116 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.647455 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:20:39.647771 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-2-root
uuid: "6994df41341e478d89f7c105f550ff93"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.648052 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:39.667145 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:39.668484 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:39.670154 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:39.672623 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:39.672853 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.673112 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:39.673274 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.719882 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.3:39469
I20260709 12:20:39.719985  4178 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.3:39469 every 8 connection(s)
I20260709 12:20:39.725296 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:39.733891  4182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:39.734386  4183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:39.737288 27116 server_base.cc:1061] running on GCE node
I20260709 12:20:39.738373  4179 heartbeater.cc:344] Connected to a master server at 127.26.123.62:34017
I20260709 12:20:39.738792  4179 heartbeater.cc:461] Registering TS with master...
W20260709 12:20:39.739053  4185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:39.739650  4179 heartbeater.cc:507] Master 127.26.123.62:34017 requested a full tablet report, sending...
I20260709 12:20:39.740139 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:39.740398 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:39.740597 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599639740575 us; error 0 us; skew 500 ppm
I20260709 12:20:39.741217 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:39.741835  3903 ts_manager.cc:194] Registered new tserver with Master: 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:39.743464  3903 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48766
I20260709 12:20:39.743996 27116 webserver.cc:533] Webserver started at http://127.26.123.4:46587/ using document root <none> and password file <none>
I20260709 12:20:39.744661 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:39.744911 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:39.745180 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:39.746340 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-3-root/instance:
uuid: "72940e4e52944adb81608e286ad58a59"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.751034 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:20:39.754522  4190 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.755324 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:20:39.755666 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-3-root
uuid: "72940e4e52944adb81608e286ad58a59"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:39.755966 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:39.814332 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:39.815745 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:39.817147 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:39.819504 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:39.819725 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.819967 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:39.820129 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:39.867043 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.4:42389
I20260709 12:20:39.867184  4252 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.4:42389 every 8 connection(s)
I20260709 12:20:39.882651  4253 heartbeater.cc:344] Connected to a master server at 127.26.123.62:34017
I20260709 12:20:39.883230  4253 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:39.884132  4253 heartbeater.cc:507] Master 127.26.123.62:34017 requested a full tablet report, sending...
I20260709 12:20:39.886472  3903 ts_manager.cc:194] Registered new tserver with Master: 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:39.887583 27116 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.017104306s
I20260709 12:20:39.888025  3903 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48782
I20260709 12:20:39.916163  3903 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48792:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:20:40.171774  4218 tablet_service.cc:1511] Processing CreateTablet for tablet 91cdbe8d3035425abc337e6e27846e42 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:40.173585  4218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91cdbe8d3035425abc337e6e27846e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.174819  4144 tablet_service.cc:1511] Processing CreateTablet for tablet 73f2a3fbee56421aa8768a4d7f275ba9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:40.176491  4144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f2a3fbee56421aa8768a4d7f275ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.179207  4143 tablet_service.cc:1511] Processing CreateTablet for tablet ac91fc64d6874b49a749e20bbf04ad07 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:40.182509  4215 tablet_service.cc:1511] Processing CreateTablet for tablet 44ae7ddfcc1e49d4b6847e9c895edde7 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:40.189719  4142 tablet_service.cc:1511] Processing CreateTablet for tablet 7ded6cf162b34debb6ea7d81c85685da (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:40.192667  4213 tablet_service.cc:1511] Processing CreateTablet for tablet 69fa505a55eb4823a412608c712fb378 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:40.193765  4142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ded6cf162b34debb6ea7d81c85685da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.197849  4216 tablet_service.cc:1511] Processing CreateTablet for tablet 7ded6cf162b34debb6ea7d81c85685da (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:40.199504  4216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ded6cf162b34debb6ea7d81c85685da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.203111  4212 tablet_service.cc:1511] Processing CreateTablet for tablet 01529dc6d8944b9697b9a87c4e4bf450 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:40.204730  4212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01529dc6d8944b9697b9a87c4e4bf450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.205773  4217 tablet_service.cc:1511] Processing CreateTablet for tablet ac91fc64d6874b49a749e20bbf04ad07 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:40.207449  4217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac91fc64d6874b49a749e20bbf04ad07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.208940  4069 tablet_service.cc:1511] Processing CreateTablet for tablet 73f2a3fbee56421aa8768a4d7f275ba9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:40.210445  4069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f2a3fbee56421aa8768a4d7f275ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.212004  4214 tablet_service.cc:1511] Processing CreateTablet for tablet 525557ea38a44858b86a97b0c444ecc4 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:40.213608  4214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 525557ea38a44858b86a97b0c444ecc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.215058  4068 tablet_service.cc:1511] Processing CreateTablet for tablet 7ded6cf162b34debb6ea7d81c85685da (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:40.216634  4068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ded6cf162b34debb6ea7d81c85685da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.207093  4070 tablet_service.cc:1511] Processing CreateTablet for tablet 91cdbe8d3035425abc337e6e27846e42 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:40.218680  4070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91cdbe8d3035425abc337e6e27846e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.219053  4067 tablet_service.cc:1511] Processing CreateTablet for tablet 7fc90d55c744447aaa17e745cf58157c (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:40.220446  4067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc90d55c744447aaa17e745cf58157c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.222972  4066 tablet_service.cc:1511] Processing CreateTablet for tablet 44ae7ddfcc1e49d4b6847e9c895edde7 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:40.224499  4066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44ae7ddfcc1e49d4b6847e9c895edde7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.224344  4140 tablet_service.cc:1511] Processing CreateTablet for tablet 525557ea38a44858b86a97b0c444ecc4 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:40.227687  4139 tablet_service.cc:1511] Processing CreateTablet for tablet 69fa505a55eb4823a412608c712fb378 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:40.227927  4065 tablet_service.cc:1511] Processing CreateTablet for tablet 69fa505a55eb4823a412608c712fb378 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:40.222800  4141 tablet_service.cc:1511] Processing CreateTablet for tablet 7fc90d55c744447aaa17e745cf58157c (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:40.229681  4065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69fa505a55eb4823a412608c712fb378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.231026  4140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 525557ea38a44858b86a97b0c444ecc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.232681  4141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc90d55c744447aaa17e745cf58157c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.232245  4064 tablet_service.cc:1511] Processing CreateTablet for tablet cb8a7e4c51b446578c698dcf9a253001 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:40.233979  4139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69fa505a55eb4823a412608c712fb378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.235404  4143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac91fc64d6874b49a749e20bbf04ad07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.220350  4215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44ae7ddfcc1e49d4b6847e9c895edde7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.237702  4213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69fa505a55eb4823a412608c712fb378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.234002  4064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8a7e4c51b446578c698dcf9a253001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.298923  4211 tablet_service.cc:1511] Processing CreateTablet for tablet 48a9a743ab6141218e529314c318f16b (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:40.300717  4211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48a9a743ab6141218e529314c318f16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.303669  4138 tablet_service.cc:1511] Processing CreateTablet for tablet cb8a7e4c51b446578c698dcf9a253001 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:40.305380  4138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8a7e4c51b446578c698dcf9a253001. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:40.320459  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:40.323155  4210 tablet_service.cc:1511] Processing CreateTablet for tablet 63fcc76766934d05b1a2f4cc1f997eb9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:40.324798  4210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fcc76766934d05b1a2f4cc1f997eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.333087  4273 tablet_bootstrap.cc:492] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:40.335496  4209 tablet_service.cc:1511] Processing CreateTablet for tablet 7c72e9bb69f746c191320d2437c3ce19 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:40.337294  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c72e9bb69f746c191320d2437c3ce19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.348587  4273 tablet_bootstrap.cc:654] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.356824  4063 tablet_service.cc:1511] Processing CreateTablet for tablet 63fcc76766934d05b1a2f4cc1f997eb9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:40.358551  4063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fcc76766934d05b1a2f4cc1f997eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.359601  4137 tablet_service.cc:1511] Processing CreateTablet for tablet 48a9a743ab6141218e529314c318f16b (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:40.361146  4137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48a9a743ab6141218e529314c318f16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.363037  4062 tablet_service.cc:1511] Processing CreateTablet for tablet 11d7a192af654f9fa50d695be708ca81 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:40.364688  4062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11d7a192af654f9fa50d695be708ca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.367197  4061 tablet_service.cc:1511] Processing CreateTablet for tablet 7c72e9bb69f746c191320d2437c3ce19 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:40.369014  4061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c72e9bb69f746c191320d2437c3ce19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.372748  4273 tablet_bootstrap.cc:492] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:40.378947  4273 ts_tablet_manager.cc:1403] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.004s
I20260709 12:20:40.384680  4136 tablet_service.cc:1511] Processing CreateTablet for tablet 63fcc76766934d05b1a2f4cc1f997eb9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:40.386731  4136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fcc76766934d05b1a2f4cc1f997eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.382547  4065 tablet_service.cc:1511] Processing CreateTablet for tablet 01529dc6d8944b9697b9a87c4e4bf450 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:40.391057  4065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01529dc6d8944b9697b9a87c4e4bf450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.394219  4273 raft_consensus.cc:359] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:40.398208  4273 raft_consensus.cc:385] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.399184  4273 raft_consensus.cc:740] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.399786  4135 tablet_service.cc:1511] Processing CreateTablet for tablet 11d7a192af654f9fa50d695be708ca81 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:40.406734  4135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11d7a192af654f9fa50d695be708ca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.407228  4273 consensus_queue.cc:260] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:40.416838  3995 tablet_service.cc:1511] Processing CreateTablet for tablet 91cdbe8d3035425abc337e6e27846e42 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:40.418498  3995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91cdbe8d3035425abc337e6e27846e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.429092  4105 heartbeater.cc:499] Master 127.26.123.62:34017 was elected leader, sending a full tablet report...
I20260709 12:20:40.429356  4273 ts_tablet_manager.cc:1434] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.007s
I20260709 12:20:40.433851  3994 tablet_service.cc:1511] Processing CreateTablet for tablet 73f2a3fbee56421aa8768a4d7f275ba9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:40.435534  3994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73f2a3fbee56421aa8768a4d7f275ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.446355  4210 tablet_service.cc:1511] Processing CreateTablet for tablet 64bc045a9f78405ebd413d12e6236f61 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:40.447917  4210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64bc045a9f78405ebd413d12e6236f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.458454  4273 tablet_bootstrap.cc:492] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:40.459542  4062 tablet_service.cc:1511] Processing CreateTablet for tablet e7536eba19a0467ab642af1d051f5b77 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:40.461130  4062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7536eba19a0467ab642af1d051f5b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.461004  3993 tablet_service.cc:1511] Processing CreateTablet for tablet ac91fc64d6874b49a749e20bbf04ad07 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:40.462749  3993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac91fc64d6874b49a749e20bbf04ad07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.465574  3992 tablet_service.cc:1511] Processing CreateTablet for tablet 7fc90d55c744447aaa17e745cf58157c (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:40.467375  3992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc90d55c744447aaa17e745cf58157c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.471637  3991 tablet_service.cc:1511] Processing CreateTablet for tablet 44ae7ddfcc1e49d4b6847e9c895edde7 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:40.472231  4273 tablet_bootstrap.cc:654] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.473302  3991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44ae7ddfcc1e49d4b6847e9c895edde7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.478998  4209 tablet_service.cc:1511] Processing CreateTablet for tablet e7536eba19a0467ab642af1d051f5b77 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:40.480611  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7536eba19a0467ab642af1d051f5b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.494338  4061 tablet_service.cc:1511] Processing CreateTablet for tablet f0991ca4c9bd47b393e9566d1a6ecfc5 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:20:40.496071  4061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0991ca4c9bd47b393e9566d1a6ecfc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.506870  4063 tablet_service.cc:1511] Processing CreateTablet for tablet 75948129b8264d75972f4660893bd024 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:20:40.507459  3990 tablet_service.cc:1511] Processing CreateTablet for tablet 525557ea38a44858b86a97b0c444ecc4 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:40.508574  4063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75948129b8264d75972f4660893bd024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.509053  3990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 525557ea38a44858b86a97b0c444ecc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.511310  3988 tablet_service.cc:1511] Processing CreateTablet for tablet 01529dc6d8944b9697b9a87c4e4bf450 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:40.512933  3988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01529dc6d8944b9697b9a87c4e4bf450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.515421  3987 tablet_service.cc:1511] Processing CreateTablet for tablet 48a9a743ab6141218e529314c318f16b (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:40.517185  3987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48a9a743ab6141218e529314c318f16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.518724  4067 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd58e37cb1847738fdf9dd220c2280a (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:20:40.519269  3986 tablet_service.cc:1511] Processing CreateTablet for tablet 11d7a192af654f9fa50d695be708ca81 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:40.520699  4067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd58e37cb1847738fdf9dd220c2280a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.522274  3989 tablet_service.cc:1511] Processing CreateTablet for tablet cb8a7e4c51b446578c698dcf9a253001 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:40.526208  3986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11d7a192af654f9fa50d695be708ca81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.527330  4030 heartbeater.cc:499] Master 127.26.123.62:34017 was elected leader, sending a full tablet report...
I20260709 12:20:40.529815  3989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8a7e4c51b446578c698dcf9a253001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.540599  4068 tablet_service.cc:1511] Processing CreateTablet for tablet bcefb699ae684288b994f3255f4c89b5 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:20:40.540928  4211 tablet_service.cc:1511] Processing CreateTablet for tablet f0991ca4c9bd47b393e9566d1a6ecfc5 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:20:40.542410  4211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0991ca4c9bd47b393e9566d1a6ecfc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.553989  4069 tablet_service.cc:1511] Processing CreateTablet for tablet 9d4786cea7ba49e49a9800f80ef684b8 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:20:40.570585  4273 tablet_bootstrap.cc:492] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:40.571441  4273 ts_tablet_manager.cc:1403] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.007s
I20260709 12:20:40.575551  4273 raft_consensus.cc:359] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:40.576526  4273 raft_consensus.cc:385] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.576864  4273 raft_consensus.cc:740] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.580614  4272 tablet_bootstrap.cc:492] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:40.586136  4068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcefb699ae684288b994f3255f4c89b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.599553  4275 raft_consensus.cc:493] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.600239  4275 raft_consensus.cc:515] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:40.605804  4214 tablet_service.cc:1511] Processing CreateTablet for tablet 98ec68263b9c4d5dbdff0fbfd53629cd (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:20:40.607347  4214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ec68263b9c4d5dbdff0fbfd53629cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.609509  4138 tablet_service.cc:1511] Processing CreateTablet for tablet e7536eba19a0467ab642af1d051f5b77 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:40.615968  4069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d4786cea7ba49e49a9800f80ef684b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.627117  4137 tablet_service.cc:1511] Processing CreateTablet for tablet 64bc045a9f78405ebd413d12e6236f61 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:40.627791  4135 tablet_service.cc:1511] Processing CreateTablet for tablet 75948129b8264d75972f4660893bd024 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
W20260709 12:20:40.628628  4106 tablet.cc:2406] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:20:40.631280  4065 tablet_service.cc:1511] Processing CreateTablet for tablet 7e92ca9c3274410aa6ea1aaa11671569 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:20:40.639878  4066 tablet_service.cc:1511] Processing CreateTablet for tablet 3331199805b64fa8bf0d5e8847ad17b8 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:20:40.617049  4138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7536eba19a0467ab642af1d051f5b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.654069  4209 tablet_service.cc:1511] Processing CreateTablet for tablet 9d4786cea7ba49e49a9800f80ef684b8 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:20:40.655642  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d4786cea7ba49e49a9800f80ef684b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.663162  4216 tablet_service.cc:1511] Processing CreateTablet for tablet bcefb699ae684288b994f3255f4c89b5 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:20:40.664973  4216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcefb699ae684288b994f3255f4c89b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.666813  4065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e92ca9c3274410aa6ea1aaa11671569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.668385  4142 tablet_service.cc:1511] Processing CreateTablet for tablet 98ec68263b9c4d5dbdff0fbfd53629cd (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:20:40.672571  4066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3331199805b64fa8bf0d5e8847ad17b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.663767  4217 tablet_service.cc:1511] Processing CreateTablet for tablet 26349fc0d5664798a248220d968f5269 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:20:40.674278  4217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26349fc0d5664798a248220d968f5269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.676677  4209 tablet_service.cc:1511] Processing CreateTablet for tablet 7e92ca9c3274410aa6ea1aaa11671569 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:20:40.676939  4143 tablet_service.cc:1511] Processing CreateTablet for tablet 7e92ca9c3274410aa6ea1aaa11671569 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:20:40.679057  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e92ca9c3274410aa6ea1aaa11671569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.679318  4136 tablet_service.cc:1511] Processing CreateTablet for tablet 9d4786cea7ba49e49a9800f80ef684b8 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:20:40.614665  4273 consensus_queue.cc:260] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:40.694056  4213 tablet_service.cc:1511] Processing CreateTablet for tablet a0f07f64b93d44269e6666cc4482c983 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:20:40.700212  4275 leader_election.cc:290] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
W20260709 12:20:40.705569  4218 thread.cc:641] tablet-open [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.048s	sys 0.086s
I20260709 12:20:40.709156  4280 tablet_bootstrap.cc:492] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:40.713411  4272 tablet_bootstrap.cc:654] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.716976  4280 tablet_bootstrap.cc:654] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.714736  4277 tablet_bootstrap.cc:492] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:40.762595  3994 tablet_service.cc:1511] Processing CreateTablet for tablet 64bc045a9f78405ebd413d12e6236f61 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:40.767516  3994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64bc045a9f78405ebd413d12e6236f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.797330  4062 tablet_service.cc:1511] Processing CreateTablet for tablet 38b7f39842e94931a878f3003c1353e1 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:20:40.800787  4062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b7f39842e94931a878f3003c1353e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.819721  4070 tablet_service.cc:1511] Processing CreateTablet for tablet a0f07f64b93d44269e6666cc4482c983 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:20:40.822427  4137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64bc045a9f78405ebd413d12e6236f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.828132  4179 heartbeater.cc:499] Master 127.26.123.62:34017 was elected leader, sending a full tablet report...
I20260709 12:20:40.836211  4070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0f07f64b93d44269e6666cc4482c983. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:40.707370  4218 thread.cc:608] tablet-open [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.050s	sys 0.086s
I20260709 12:20:40.864892  4218 tablet_service.cc:1511] Processing CreateTablet for tablet 72ce1dc8db9548058ab94a9115c111f2 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
W20260709 12:20:40.874342  4045 leader_election.cc:343] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Not found: Tablet not found: 69fa505a55eb4823a412608c712fb378
I20260709 12:20:40.878664  4135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75948129b8264d75972f4660893bd024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.878433  4281 raft_consensus.cc:493] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.879452  4281 raft_consensus.cc:515] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:40.856280  4144 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd58e37cb1847738fdf9dd220c2280a (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:20:40.915210  4209 tablet_service.cc:1511] Processing CreateTablet for tablet 38b7f39842e94931a878f3003c1353e1 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:20:40.916656  4143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e92ca9c3274410aa6ea1aaa11671569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.939819  4215 tablet_service.cc:1511] Processing CreateTablet for tablet ebb6ad79685348f4a82858f9f7844c48 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:20:40.907137  4273 ts_tablet_manager.cc:1434] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.305s	user 0.010s	sys 0.012s
I20260709 12:20:40.943041  4218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ce1dc8db9548058ab94a9115c111f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.942723  4063 tablet_service.cc:1511] Processing CreateTablet for tablet e203dae959394f4e90051ad4746a9291 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:20:40.920250  4281 leader_election.cc:290] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:40.923518  4217 tablet_service.cc:1511] Processing CreateTablet for tablet e203dae959394f4e90051ad4746a9291 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:20:40.930245  4068 tablet_service.cc:1511] Processing CreateTablet for tablet a252087aa0c9426cb3067f2f9004dd29 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:20:40.917521  4253 heartbeater.cc:499] Master 127.26.123.62:34017 was elected leader, sending a full tablet report...
I20260709 12:20:40.954151  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d7a192af654f9fa50d695be708ca81" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:40.958081  4045 leader_election.cc:343] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.959447  3995 tablet_service.cc:1511] Processing CreateTablet for tablet 75948129b8264d75972f4660893bd024 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:20:40.961127  3995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75948129b8264d75972f4660893bd024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.959355  3993 tablet_service.cc:1511] Processing CreateTablet for tablet 7c72e9bb69f746c191320d2437c3ce19 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:40.972352  3993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c72e9bb69f746c191320d2437c3ce19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.980542  4142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ec68263b9c4d5dbdff0fbfd53629cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.982365  3991 tablet_service.cc:1511] Processing CreateTablet for tablet f0991ca4c9bd47b393e9566d1a6ecfc5 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:20:40.983204  4213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0f07f64b93d44269e6666cc4482c983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.983666  4273 tablet_bootstrap.cc:492] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:40.983927  3988 tablet_service.cc:1511] Processing CreateTablet for tablet 26349fc0d5664798a248220d968f5269 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:20:40.987090  3991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0991ca4c9bd47b393e9566d1a6ecfc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.989125  4139 tablet_service.cc:1511] Processing CreateTablet for tablet 72ce1dc8db9548058ab94a9115c111f2 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:20:40.990675  4139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ce1dc8db9548058ab94a9115c111f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.995160  4068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a252087aa0c9426cb3067f2f9004dd29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.003229  3989 tablet_service.cc:1511] Processing CreateTablet for tablet 98ec68263b9c4d5dbdff0fbfd53629cd (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:20:41.004922  3989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98ec68263b9c4d5dbdff0fbfd53629cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.006246  4063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e203dae959394f4e90051ad4746a9291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.007701  4215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb6ad79685348f4a82858f9f7844c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.008409  4140 tablet_service.cc:1511] Processing CreateTablet for tablet 3331199805b64fa8bf0d5e8847ad17b8 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:20:41.009433  4067 tablet_service.cc:1511] Processing CreateTablet for tablet 74b93cd1c6f94fdf869528b9411a7959 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:20:41.013410  4067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74b93cd1c6f94fdf869528b9411a7959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.015971  4212 tablet_service.cc:1511] Processing CreateTablet for tablet 74b93cd1c6f94fdf869528b9411a7959 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:20:41.017455  4212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74b93cd1c6f94fdf869528b9411a7959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.020316  4217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e203dae959394f4e90051ad4746a9291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.021159  4064 tablet_service.cc:1511] Processing CreateTablet for tablet 65e4899b95af40fd9836e281f4c23e17 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:20:41.022725  4064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65e4899b95af40fd9836e281f4c23e17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:40.973019  4141 tablet_service.cc:1511] Processing CreateTablet for tablet 26349fc0d5664798a248220d968f5269 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:20:41.030409  3987 tablet_service.cc:1511] Processing CreateTablet for tablet 6dd58e37cb1847738fdf9dd220c2280a (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:20:41.031875  3987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd58e37cb1847738fdf9dd220c2280a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.036289  4211 tablet_service.cc:1511] Processing CreateTablet for tablet d14347d13af8448bb12300afb6a90639 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:20:41.037206  4140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3331199805b64fa8bf0d5e8847ad17b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.044088  4273 tablet_bootstrap.cc:654] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.055617  3990 tablet_service.cc:1511] Processing CreateTablet for tablet bcefb699ae684288b994f3255f4c89b5 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:20:41.081776  3990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcefb699ae684288b994f3255f4c89b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.087719  4062 tablet_service.cc:1511] Processing CreateTablet for tablet d14347d13af8448bb12300afb6a90639 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:20:41.089468  4062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d14347d13af8448bb12300afb6a90639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.090534  4136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d4786cea7ba49e49a9800f80ef684b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.096165  3994 tablet_service.cc:1511] Processing CreateTablet for tablet a0f07f64b93d44269e6666cc4482c983 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:20:41.062515  4272 tablet_bootstrap.cc:492] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:41.097532  4272 ts_tablet_manager.cc:1403] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.517s	user 0.026s	sys 0.023s
I20260709 12:20:41.098232  4144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd58e37cb1847738fdf9dd220c2280a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.077787  4211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d14347d13af8448bb12300afb6a90639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.102165  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b7f39842e94931a878f3003c1353e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.108355  3994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0f07f64b93d44269e6666cc4482c983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.111012  3988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26349fc0d5664798a248220d968f5269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.114846  4280 tablet_bootstrap.cc:492] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:41.115612  4280 ts_tablet_manager.cc:1403] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.407s	user 0.038s	sys 0.015s
I20260709 12:20:41.120122  4280 raft_consensus.cc:359] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:41.121021  4280 raft_consensus.cc:385] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.121367  4280 raft_consensus.cc:740] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.122599  4280 consensus_queue.cc:260] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:41.124083  4272 raft_consensus.cc:359] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:41.130719  4272 raft_consensus.cc:385] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.137156  4141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26349fc0d5664798a248220d968f5269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.150391  4277 tablet_bootstrap.cc:654] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.150238  4138 tablet_service.cc:1511] Processing CreateTablet for tablet a252087aa0c9426cb3067f2f9004dd29 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:20:41.162532  3992 tablet_service.cc:1511] Processing CreateTablet for tablet 3331199805b64fa8bf0d5e8847ad17b8 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:20:41.164013  3992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3331199805b64fa8bf0d5e8847ad17b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.170619  4216 tablet_service.cc:1511] Processing CreateTablet for tablet fcb63c6a1eed482f9d368536f4aaadf1 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:20:41.172073  4216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb63c6a1eed482f9d368536f4aaadf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.173616  4280 ts_tablet_manager.cc:1434] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.018s
I20260709 12:20:41.161316  3986 tablet_service.cc:1511] Processing CreateTablet for tablet 72ce1dc8db9548058ab94a9115c111f2 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:20:41.174180  4138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a252087aa0c9426cb3067f2f9004dd29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.175457  4137 tablet_service.cc:1511] Processing CreateTablet for tablet 38b7f39842e94931a878f3003c1353e1 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:20:41.187850  4137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38b7f39842e94931a878f3003c1353e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.190477  3986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72ce1dc8db9548058ab94a9115c111f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.176172  4273 tablet_bootstrap.cc:492] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:41.192966  4273 ts_tablet_manager.cc:1403] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.210s	user 0.012s	sys 0.009s
I20260709 12:20:41.195490  4061 tablet_service.cc:1511] Processing CreateTablet for tablet e9fe17de2e1845dd959d6b3d892cda55 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:20:41.196985  4061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9fe17de2e1845dd959d6b3d892cda55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.207957  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d7a192af654f9fa50d695be708ca81" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:41.209703  4280 tablet_bootstrap.cc:492] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:41.225564  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69fa505a55eb4823a412608c712fb378" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:41.146628  4272 raft_consensus.cc:740] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
W20260709 12:20:41.229512  4046 leader_election.cc:343] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.231693  4046 leader_election.cc:304] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:41.230932  4213 tablet_service.cc:1511] Processing CreateTablet for tablet e9fe17de2e1845dd959d6b3d892cda55 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:20:41.233181  4213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9fe17de2e1845dd959d6b3d892cda55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.233943  4281 raft_consensus.cc:2749] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.234349  4273 raft_consensus.cc:359] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:41.235256  4273 raft_consensus.cc:385] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:20:41.235388  4046 leader_election.cc:343] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.235706  4273 raft_consensus.cc:740] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.236740  4273 consensus_queue.cc:260] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:41.240517  4135 tablet_service.cc:1511] Processing CreateTablet for tablet 74b93cd1c6f94fdf869528b9411a7959 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:20:41.249584  4070 tablet_service.cc:1511] Processing CreateTablet for tablet c07edbbf4ef64ddbbad929d20ce78c4e (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:20:41.250053  4210 tablet_service.cc:1511] Processing CreateTablet for tablet b08446fff0954c05b5311d3e76153e58 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:20:41.251091  4070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07edbbf4ef64ddbbad929d20ce78c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.251519  4210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b08446fff0954c05b5311d3e76153e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.256695  4069 tablet_service.cc:1511] Processing CreateTablet for tablet b08446fff0954c05b5311d3e76153e58 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:20:41.258177  4069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b08446fff0954c05b5311d3e76153e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.261273  4214 tablet_service.cc:1511] Processing CreateTablet for tablet 76d413f51fcd4460aee2adb16b951a7c (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:20:41.262712  4214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d413f51fcd4460aee2adb16b951a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.267498  4287 raft_consensus.cc:493] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.268677  4046 leader_election.cc:304] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:41.268535  4287 raft_consensus.cc:515] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:41.207849  4136 tablet_service.cc:1511] Processing CreateTablet for tablet ebb6ad79685348f4a82858f9f7844c48 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:20:41.290990  3993 tablet_service.cc:1511] Processing CreateTablet for tablet e203dae959394f4e90051ad4746a9291 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:20:41.292580  3993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e203dae959394f4e90051ad4746a9291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.294121  3995 tablet_service.cc:1511] Processing CreateTablet for tablet a252087aa0c9426cb3067f2f9004dd29 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:20:41.298195  4135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74b93cd1c6f94fdf869528b9411a7959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.298516  4272 consensus_queue.cc:260] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:41.265241  4142 tablet_service.cc:1511] Processing CreateTablet for tablet 65e4899b95af40fd9836e281f4c23e17 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
W20260709 12:20:41.309921  3890 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 44892.9us
I20260709 12:20:41.307636  4281 raft_consensus.cc:2749] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.312255  4273 ts_tablet_manager.cc:1434] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.119s	user 0.004s	sys 0.004s
I20260709 12:20:41.313464  4273 tablet_bootstrap.cc:492] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:41.316562  4142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65e4899b95af40fd9836e281f4c23e17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.317584  4136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb6ad79685348f4a82858f9f7844c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.318356  3995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a252087aa0c9426cb3067f2f9004dd29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.318969  4281 raft_consensus.cc:493] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.319511  4281 raft_consensus.cc:515] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:41.308373  4272 ts_tablet_manager.cc:1434] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.210s	user 0.004s	sys 0.005s
I20260709 12:20:41.329627  4140 tablet_service.cc:1511] Processing CreateTablet for tablet fcb63c6a1eed482f9d368536f4aaadf1 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:20:41.332367  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fcc76766934d05b1a2f4cc1f997eb9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:41.335462  4272 tablet_bootstrap.cc:492] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
W20260709 12:20:41.336206  4045 leader_election.cc:343] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.337926  3994 tablet_service.cc:1511] Processing CreateTablet for tablet 65e4899b95af40fd9836e281f4c23e17 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:20:41.339259  3994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65e4899b95af40fd9836e281f4c23e17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.344059  4277 tablet_bootstrap.cc:492] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:41.344704  4277 ts_tablet_manager.cc:1403] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.630s	user 0.027s	sys 0.000s
W20260709 12:20:41.347285  3889 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 82423.1us
I20260709 12:20:41.348305  4277 raft_consensus.cc:359] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:41.349092  4277 raft_consensus.cc:385] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.349457  4277 raft_consensus.cc:740] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
W20260709 12:20:41.349622  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:41.350420  4277 consensus_queue.cc:260] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:41.331986  4281 leader_election.cc:290] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:41.363799  4287 leader_election.cc:290] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:41.355188  4140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb63c6a1eed482f9d368536f4aaadf1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:41.355542  3890 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.26.123.1:41077, user_credentials={real_user=slave}} blocked reactor thread for 44781.3us
I20260709 12:20:41.382480  4064 tablet_service.cc:1511] Processing CreateTablet for tablet 724697c71b61499aa0daadd2e66af805 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:20:41.388700  4209 tablet_service.cc:1511] Processing CreateTablet for tablet 724697c71b61499aa0daadd2e66af805 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:20:41.393154  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 724697c71b61499aa0daadd2e66af805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.417938  4064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 724697c71b61499aa0daadd2e66af805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.417183  4273 tablet_bootstrap.cc:654] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.431514  3988 tablet_service.cc:1511] Processing CreateTablet for tablet d14347d13af8448bb12300afb6a90639 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:20:41.433058  3988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d14347d13af8448bb12300afb6a90639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.433599  3986 tablet_service.cc:1511] Processing CreateTablet for tablet ebb6ad79685348f4a82858f9f7844c48 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:20:41.434908  3986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebb6ad79685348f4a82858f9f7844c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.438032  4141 tablet_service.cc:1511] Processing CreateTablet for tablet c07edbbf4ef64ddbbad929d20ce78c4e (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:20:41.439543  4141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07edbbf4ef64ddbbad929d20ce78c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.441882  3990 tablet_service.cc:1511] Processing CreateTablet for tablet c07edbbf4ef64ddbbad929d20ce78c4e (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:20:41.350459  4143 tablet_service.cc:1511] Processing CreateTablet for tablet e9fe17de2e1845dd959d6b3d892cda55 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:20:41.460345  4144 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45792 (request call id 15) took 1056 ms. Trace:
I20260709 12:20:41.466593  4143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9fe17de2e1845dd959d6b3d892cda55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.460506  3991 tablet_service.cc:1511] Processing CreateTablet for tablet b08446fff0954c05b5311d3e76153e58 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:20:41.460906  4144 rpcz_store.cc:276] 0709 12:20:40.403890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:40.854927 (+451037us) service_pool.cc:224] Handling call
0709 12:20:40.931045 (+ 76118us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:20:40.951476 (+ 20431us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:20:41.161326 (+209850us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:41.457527 (+296201us) tablet_replica.cc:206] Creating consensus instance
0709 12:20:41.460271 (+  2744us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:41.475881  3990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07edbbf4ef64ddbbad929d20ce78c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.473656  4280 tablet_bootstrap.cc:654] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.486630  4295 raft_consensus.cc:493] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.487265  4295 raft_consensus.cc:515] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
W20260709 12:20:41.464499  4046 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 229128us
I20260709 12:20:41.441762  3989 tablet_service.cc:1511] Processing CreateTablet for tablet fcb63c6a1eed482f9d368536f4aaadf1 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:20:41.497777  3991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b08446fff0954c05b5311d3e76153e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.503082  4061 tablet_service.cc:1511] Processing CreateTablet for tablet d10e7c5b29b14b4e8c90d79a5201bf26 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:20:41.510550  3989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb63c6a1eed482f9d368536f4aaadf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.503248  3992 tablet_service.cc:1511] Processing CreateTablet for tablet 76d413f51fcd4460aee2adb16b951a7c (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:20:41.511643  3992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d413f51fcd4460aee2adb16b951a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.514120  4211 tablet_service.cc:1511] Processing CreateTablet for tablet 389645cf278940c6803751b614ba1dfa (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:20:41.514539  4069 tablet_service.cc:1511] Processing CreateTablet for tablet 389645cf278940c6803751b614ba1dfa (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:20:41.515417  4211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389645cf278940c6803751b614ba1dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.509339  4272 tablet_bootstrap.cc:654] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.518559  4212 tablet_service.cc:1511] Processing CreateTablet for tablet 3e9cdd46c01a43e3a92c6065a0a15e12 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:20:41.519866  4212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e9cdd46c01a43e3a92c6065a0a15e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.510673  4061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d10e7c5b29b14b4e8c90d79a5201bf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.525446  4277 ts_tablet_manager.cc:1434] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.180s	user 0.006s	sys 0.011s
I20260709 12:20:41.526782  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fcc76766934d05b1a2f4cc1f997eb9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:41.530573  4046 leader_election.cc:343] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.531044  4046 leader_election.cc:304] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:41.532047  4292 raft_consensus.cc:2749] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.542035  4069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389645cf278940c6803751b614ba1dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.543475  4068 tablet_service.cc:1511] Processing CreateTablet for tablet 9dc6c1656d03459babbca1538b0ae0ef (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:20:41.544998  4068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dc6c1656d03459babbca1538b0ae0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.550704  4214 tablet_service.cc:1511] Processing CreateTablet for tablet 9dc6c1656d03459babbca1538b0ae0ef (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:20:41.552141  4214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dc6c1656d03459babbca1538b0ae0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.558827  4139 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45792 (request call id 19) took 1120 ms. Trace:
I20260709 12:20:41.559324  4139 rpcz_store.cc:276] 0709 12:20:40.438718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:40.987884 (+549166us) service_pool.cc:224] Handling call
0709 12:20:40.990086 (+  2202us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:20:40.990215 (+   129us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:20:41.394595 (+404380us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:41.558315 (+163720us) tablet_replica.cc:206] Creating consensus instance
0709 12:20:41.558759 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:41.526691  4277 tablet_bootstrap.cc:492] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:41.509860  4295 leader_election.cc:290] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:41.585258  4277 tablet_bootstrap.cc:654] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.586133  4137 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45792 (request call id 20) took 1110 ms. Trace:
I20260709 12:20:41.586645  4137 rpcz_store.cc:276] 0709 12:20:40.476042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.144549 (+668507us) service_pool.cc:224] Handling call
0709 12:20:41.187323 (+ 42774us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:20:41.187441 (+   118us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:20:41.343798 (+156357us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:41.493794 (+149996us) tablet_replica.cc:206] Creating consensus instance
0709 12:20:41.586045 (+ 92251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:41.588889  4137 tablet_service.cc:1511] Processing CreateTablet for tablet 724697c71b61499aa0daadd2e66af805 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:20:41.590320  4137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 724697c71b61499aa0daadd2e66af805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.599798  4062 tablet_service.cc:1511] Processing CreateTablet for tablet 4ae1da0dfadb4b17bebb0c20ce3ec355 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:20:41.601500  4062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ae1da0dfadb4b17bebb0c20ce3ec355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.601675  4067 tablet_service.cc:1511] Processing CreateTablet for tablet 81ff51ff620749a591f77c58c287e374 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:20:41.606734  4067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ff51ff620749a591f77c58c287e374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.615330  4216 tablet_service.cc:1511] Processing CreateTablet for tablet 9fda924a0568433fa70ab72d4a0c3968 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:20:41.616786  4216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fda924a0568433fa70ab72d4a0c3968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.600528  4214 tablet_service.cc:1511] Processing CreateTablet for tablet 81ff51ff620749a591f77c58c287e374 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:20:41.632223  4144 tablet_service.cc:1511] Processing CreateTablet for tablet 76d413f51fcd4460aee2adb16b951a7c (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:20:41.633715  4144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d413f51fcd4460aee2adb16b951a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.660782  3992 tablet_service.cc:1511] Processing CreateTablet for tablet d10e7c5b29b14b4e8c90d79a5201bf26 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:20:41.664480  4291 raft_consensus.cc:493] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.667217  3992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d10e7c5b29b14b4e8c90d79a5201bf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.621273  4070 tablet_service.cc:1511] Processing CreateTablet for tablet 435b4d42ac35468f8a95973a14a9d6e3 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:20:41.674573  4214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ff51ff620749a591f77c58c287e374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.646914  4139 tablet_service.cc:1511] Processing CreateTablet for tablet d10e7c5b29b14b4e8c90d79a5201bf26 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:20:41.685298  4138 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45792 (request call id 21) took 1158 ms. Trace:
I20260709 12:20:41.685472  4139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d10e7c5b29b14b4e8c90d79a5201bf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.685719  4138 rpcz_store.cc:276] 0709 12:20:40.529094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.148883 (+619789us) service_pool.cc:224] Handling call
0709 12:20:41.151251 (+  2368us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:20:41.151388 (+   137us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:20:41.314159 (+162771us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:41.678415 (+364256us) tablet_replica.cc:206] Creating consensus instance
0709 12:20:41.685227 (+  6812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:41.687557  4142 tablet_service.cc:1511] Processing CreateTablet for tablet 3e9cdd46c01a43e3a92c6065a0a15e12 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:20:41.690127  3987 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46182 (request call id 15) took 1179 ms. Trace:
I20260709 12:20:41.691844  4142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e9cdd46c01a43e3a92c6065a0a15e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.674419  4291 raft_consensus.cc:515] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:41.691041  3987 rpcz_store.cc:276] 0709 12:20:40.510522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.029278 (+518756us) service_pool.cc:224] Handling call
0709 12:20:41.031387 (+  2109us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:20:41.031502 (+   115us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:20:41.279189 (+247687us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:41.679171 (+399982us) tablet_replica.cc:206] Creating consensus instance
0709 12:20:41.690001 (+ 10830us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:41.699167  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f2a3fbee56421aa8768a4d7f275ba9" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:41.700295  3993 tablet_service.cc:1511] Processing CreateTablet for tablet 3e9cdd46c01a43e3a92c6065a0a15e12 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:20:41.701881  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f2a3fbee56421aa8768a4d7f275ba9" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:41.702983  4273 tablet_bootstrap.cc:492] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:41.703537  4273 ts_tablet_manager.cc:1403] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.390s	user 0.022s	sys 0.029s
I20260709 12:20:41.704001  4135 tablet_service.cc:1511] Processing CreateTablet for tablet 4ae1da0dfadb4b17bebb0c20ce3ec355 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:20:41.705404  4135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ae1da0dfadb4b17bebb0c20ce3ec355. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:41.705659  3971 leader_election.cc:343] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:41.706772  3970 leader_election.cc:343] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.707276  3970 leader_election.cc:304] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:41.708351  4287 raft_consensus.cc:2749] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.700305  3987 tablet_service.cc:1511] Processing CreateTablet for tablet 4ae1da0dfadb4b17bebb0c20ce3ec355 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:20:41.712249  4070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435b4d42ac35468f8a95973a14a9d6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.715863  4291 leader_election.cc:290] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:41.687736  4138 tablet_service.cc:1511] Processing CreateTablet for tablet 9dc6c1656d03459babbca1538b0ae0ef (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:20:41.717464  3994 tablet_service.cc:1511] Processing CreateTablet for tablet 389645cf278940c6803751b614ba1dfa (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:20:41.720081  4280 tablet_bootstrap.cc:492] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:41.720686  4280 ts_tablet_manager.cc:1403] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.511s	user 0.008s	sys 0.016s
I20260709 12:20:41.722764  3993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e9cdd46c01a43e3a92c6065a0a15e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.746163  3987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ae1da0dfadb4b17bebb0c20ce3ec355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.749718  4138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dc6c1656d03459babbca1538b0ae0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.737103  4061 tablet_service.cc:1511] Processing CreateTablet for tablet 35f92fa80bd34e60924d5c77f79df421 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:20:41.754007  4067 tablet_service.cc:1511] Processing CreateTablet for tablet 778cd7f085494ba1842e5f758d7f9828 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:20:41.757102  4209 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1c0613280b411f98f4807034b0a1cb (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:20:41.754601  4140 tablet_service.cc:1511] Processing CreateTablet for tablet 9fda924a0568433fa70ab72d4a0c3968 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:20:41.758749  4209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1c0613280b411f98f4807034b0a1cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.759163  4140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fda924a0568433fa70ab72d4a0c3968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.761184  3994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 389645cf278940c6803751b614ba1dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.763257  4061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f92fa80bd34e60924d5c77f79df421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.766326  4067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778cd7f085494ba1842e5f758d7f9828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.789878  4213 tablet_service.cc:1511] Processing CreateTablet for tablet 435b4d42ac35468f8a95973a14a9d6e3 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:20:41.791486  4213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435b4d42ac35468f8a95973a14a9d6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.793501  3988 tablet_service.cc:1511] Processing CreateTablet for tablet 81ff51ff620749a591f77c58c287e374 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:20:41.779150  3995 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46182 (request call id 21) took 1192 ms. Trace:
I20260709 12:20:41.787742  4273 raft_consensus.cc:359] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:41.798050  4273 raft_consensus.cc:385] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.798396  4273 raft_consensus.cc:740] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.799023  4272 tablet_bootstrap.cc:492] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:41.799687  4272 ts_tablet_manager.cc:1403] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.464s	user 0.019s	sys 0.014s
I20260709 12:20:41.799449  4273 consensus_queue.cc:260] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:41.802574  4273 ts_tablet_manager.cc:1434] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.099s	user 0.009s	sys 0.002s
I20260709 12:20:41.803920  4273 tablet_bootstrap.cc:492] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:41.804589  4280 raft_consensus.cc:359] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:41.797911  3995 rpcz_store.cc:276] 0709 12:20:40.586332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.224797 (+638465us) service_pool.cc:224] Handling call
0709 12:20:41.294981 (+ 70184us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:20:41.295094 (+   113us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:20:41.577330 (+282236us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:41.741407 (+164077us) tablet_replica.cc:206] Creating consensus instance
0709 12:20:41.776844 (+ 35437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:41.805387  4280 raft_consensus.cc:385] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.795749  4136 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45792 (request call id 22) took 1179 ms. Trace:
I20260709 12:20:41.802207  3990 tablet_service.cc:1511] Processing CreateTablet for tablet 9fda924a0568433fa70ab72d4a0c3968 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:20:41.812758  3995 tablet_service.cc:1511] Processing CreateTablet for tablet 35f92fa80bd34e60924d5c77f79df421 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:20:41.815196  4272 raft_consensus.cc:359] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:41.816046  4272 raft_consensus.cc:385] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.816350  4272 raft_consensus.cc:740] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.817160  4272 consensus_queue.cc:260] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:41.820309  4272 ts_tablet_manager.cc:1434] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 12:20:41.753973  4211 tablet_service.cc:1511] Processing CreateTablet for tablet 778cd7f085494ba1842e5f758d7f9828 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:20:41.822775  4211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778cd7f085494ba1842e5f758d7f9828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.823609  3991 tablet_service.cc:1511] Processing CreateTablet for tablet 778cd7f085494ba1842e5f758d7f9828 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:20:41.824443  4277 tablet_bootstrap.cc:492] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:41.825073  4277 ts_tablet_manager.cc:1403] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.299s	user 0.012s	sys 0.006s
I20260709 12:20:41.826213  3988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81ff51ff620749a591f77c58c287e374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.829125  4292 raft_consensus.cc:493] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.829646  4292 raft_consensus.cc:515] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:41.832115  4292 leader_election.cc:290] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:41.807319  4136 rpcz_store.cc:276] 0709 12:20:40.616778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.185261 (+568483us) service_pool.cc:224] Handling call
0709 12:20:41.286436 (+101175us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:20:41.286655 (+   219us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:20:41.663234 (+376579us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:41.795155 (+131921us) tablet_replica.cc:206] Creating consensus instance
0709 12:20:41.795674 (+   519us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:41.821574  4272 tablet_bootstrap.cc:492] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:41.806504  4280 raft_consensus.cc:740] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.854948  4280 consensus_queue.cc:260] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:41.842456  3986 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46182 (request call id 23) took 1001 ms. Trace:
I20260709 12:20:41.843480  4143 tablet_service.cc:1511] Processing CreateTablet for tablet 35f92fa80bd34e60924d5c77f79df421 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:20:41.845808  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c72e9bb69f746c191320d2437c3ce19" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:41.846975  3995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f92fa80bd34e60924d5c77f79df421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.851079  4136 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1c0613280b411f98f4807034b0a1cb (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:20:41.828699  4277 raft_consensus.cc:359] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:41.869344  4136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1c0613280b411f98f4807034b0a1cb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:41.869681  4046 leader_election.cc:343] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.875257  4143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35f92fa80bd34e60924d5c77f79df421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.860219  3986 rpcz_store.cc:276] 0709 12:20:40.841489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.427034 (+585545us) service_pool.cc:224] Handling call
0709 12:20:41.434451 (+  7417us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:20:41.434551 (+   100us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:20:41.569716 (+135165us) tablet_metadata.cc:665] Metadata flushed
0709 12:20:41.814039 (+244323us) tablet_replica.cc:206] Creating consensus instance
0709 12:20:41.842031 (+ 27992us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b5800002798. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb762b71 00007f85bb7631a2 00007f85c46a4889 00007f85c37d5d88 00007f85c389c973 00007f85c389870f 00007f85c3898ecd 00007f85c3847400 00007f85bccccaf4 00007f85bcccca82
0709 12:20:41.842375 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48921856}
I20260709 12:20:41.877404  3991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778cd7f085494ba1842e5f758d7f9828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.877678  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fcc76766934d05b1a2f4cc1f997eb9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:41.878569  4080 raft_consensus.cc:2468] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:41.891019  3990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fda924a0568433fa70ab72d4a0c3968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.885382  4120 leader_election.cc:304] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:41.898655  4295 raft_consensus.cc:2804] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:41.899066  4295 raft_consensus.cc:493] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.899605  4295 raft_consensus.cc:3060] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.871414  4277 raft_consensus.cc:385] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.902387  4277 raft_consensus.cc:740] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.903296  4277 consensus_queue.cc:260] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:41.909677  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c72e9bb69f746c191320d2437c3ce19" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:41.909920  4295 raft_consensus.cc:515] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:41.899055  4280 ts_tablet_manager.cc:1434] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.178s	user 0.004s	sys 0.007s
I20260709 12:20:41.912379  4295 leader_election.cc:290] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:41.913130  4280 tablet_bootstrap.cc:492] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:41.915105  4068 tablet_service.cc:1511] Processing CreateTablet for tablet 581a73aef577436b9cc60234f338e9b9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:20:41.916051  4273 tablet_bootstrap.cc:654] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.916486  4068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 581a73aef577436b9cc60234f338e9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.934015  4141 tablet_service.cc:1511] Processing CreateTablet for tablet 435b4d42ac35468f8a95973a14a9d6e3 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:20:41.935652  4141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435b4d42ac35468f8a95973a14a9d6e3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:41.945837  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 61521.4us
W20260709 12:20:41.946233  4046 leader_election.cc:343] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.946820  4046 leader_election.cc:304] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:41.947697  4292 raft_consensus.cc:2749] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.947796  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fcc76766934d05b1a2f4cc1f997eb9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:41.948544  4080 raft_consensus.cc:3060] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.957389  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fcc76766934d05b1a2f4cc1f997eb9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:41.958273  4228 raft_consensus.cc:2468] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:41.959223  4080 raft_consensus.cc:2468] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:41.959848  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fcc76766934d05b1a2f4cc1f997eb9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:41.960500  4228 raft_consensus.cc:3060] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.960852  4120 leader_election.cc:304] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:41.961927  4295 raft_consensus.cc:2804] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:41.962466  4295 raft_consensus.cc:697] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:41.963562  4295 consensus_queue.cc:237] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:41.977692  4061 tablet_service.cc:1511] Processing CreateTablet for tablet c2adcdb890e0483095cb9923dce850a1 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:20:41.979314  4061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2adcdb890e0483095cb9923dce850a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.991742  3992 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1c0613280b411f98f4807034b0a1cb (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:20:41.992609  4214 tablet_service.cc:1511] Processing CreateTablet for tablet 581a73aef577436b9cc60234f338e9b9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:20:41.993556  4228 raft_consensus.cc:2468] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:41.994030  4214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 581a73aef577436b9cc60234f338e9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.997385  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91cdbe8d3035425abc337e6e27846e42" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:41.964390  4272 tablet_bootstrap.cc:654] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.993528  3992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1c0613280b411f98f4807034b0a1cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:41.997200  3987 tablet_service.cc:1511] Processing CreateTablet for tablet c2adcdb890e0483095cb9923dce850a1 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:20:42.002177  4280 tablet_bootstrap.cc:654] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.022382  3987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2adcdb890e0483095cb9923dce850a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.024132  3902 catalog_manager.cc:5697] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.026736  4142 tablet_service.cc:1511] Processing CreateTablet for tablet 581a73aef577436b9cc60234f338e9b9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
W20260709 12:20:42.029979  4194 leader_election.cc:343] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.031931  4277 ts_tablet_manager.cc:1434] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.207s	user 0.006s	sys 0.011s
I20260709 12:20:42.033260  4068 tablet_service.cc:1511] Processing CreateTablet for tablet 9233488a2b6745f8a012d9edc348333a (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:20:42.034646  4139 tablet_service.cc:1511] Processing CreateTablet for tablet 18dadd11589447aea90415c88b77e312 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:20:42.033825  3994 tablet_service.cc:1511] Processing CreateTablet for tablet 9233488a2b6745f8a012d9edc348333a (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:20:42.035979  4216 tablet_service.cc:1511] Processing CreateTablet for tablet 18dadd11589447aea90415c88b77e312 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:20:42.034827  4068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9233488a2b6745f8a012d9edc348333a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.039592  3994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9233488a2b6745f8a012d9edc348333a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.040289  4069 tablet_service.cc:1511] Processing CreateTablet for tablet fe179cc2289146d98b10419917a7a36d (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:20:42.042600  4144 tablet_service.cc:1511] Processing CreateTablet for tablet fe179cc2289146d98b10419917a7a36d (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:20:42.043346  4212 tablet_service.cc:1511] Processing CreateTablet for tablet fe179cc2289146d98b10419917a7a36d (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:20:42.041419  4135 tablet_service.cc:1511] Processing CreateTablet for tablet c2adcdb890e0483095cb9923dce850a1 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:20:42.059271  4144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe179cc2289146d98b10419917a7a36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.044158  4216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18dadd11589447aea90415c88b77e312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.068504  4211 tablet_service.cc:1511] Processing CreateTablet for tablet 9233488a2b6745f8a012d9edc348333a (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:20:42.070331  4142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 581a73aef577436b9cc60234f338e9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.072695  4211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9233488a2b6745f8a012d9edc348333a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.073942  4212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe179cc2289146d98b10419917a7a36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.082950  4069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe179cc2289146d98b10419917a7a36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.083393  4139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18dadd11589447aea90415c88b77e312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.085178  4135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2adcdb890e0483095cb9923dce850a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.086826  4273 tablet_bootstrap.cc:492] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:42.087455  4273 ts_tablet_manager.cc:1403] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.284s	user 0.015s	sys 0.006s
I20260709 12:20:42.043429  3993 tablet_service.cc:1511] Processing CreateTablet for tablet 18dadd11589447aea90415c88b77e312 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:20:42.091068  3993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18dadd11589447aea90415c88b77e312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.109557  4280 tablet_bootstrap.cc:492] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:42.112186  4280 ts_tablet_manager.cc:1403] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.199s	user 0.010s	sys 0.007s
I20260709 12:20:42.111120  3988 tablet_service.cc:1511] Processing CreateTablet for tablet f0bab1596ed74079b6efee7c9fefe4f9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:20:42.123374  3988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0bab1596ed74079b6efee7c9fefe4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.130903  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91cdbe8d3035425abc337e6e27846e42" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:42.129204  4280 raft_consensus.cc:359] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.134361  4280 raft_consensus.cc:385] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.136420  4280 raft_consensus.cc:740] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.137506  4280 consensus_queue.cc:260] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.140307  4273 raft_consensus.cc:359] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.146416  4273 raft_consensus.cc:385] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.146785  4273 raft_consensus.cc:740] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.147644  4273 consensus_queue.cc:260] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.150780  4273 ts_tablet_manager.cc:1434] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.002s
I20260709 12:20:42.154088  4273 tablet_bootstrap.cc:492] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:42.157929  4277 tablet_bootstrap.cc:492] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:42.163128  4067 tablet_service.cc:1511] Processing CreateTablet for tablet f0bab1596ed74079b6efee7c9fefe4f9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:20:42.173228  4005 raft_consensus.cc:2468] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:42.174634  4212 tablet_service.cc:1511] Processing CreateTablet for tablet 24281667f1364c6fac542e54ed413502 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:20:42.176323  4212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24281667f1364c6fac542e54ed413502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.178076  4136 tablet_service.cc:1511] Processing CreateTablet for tablet f0bab1596ed74079b6efee7c9fefe4f9 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:20:42.179018  4194 leader_election.cc:304] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa
I20260709 12:20:42.179768  4136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0bab1596ed74079b6efee7c9fefe4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.180428  4291 raft_consensus.cc:2804] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.180896  4291 raft_consensus.cc:493] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.181218  4291 raft_consensus.cc:3060] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.184590  4067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0bab1596ed74079b6efee7c9fefe4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.184917  4306 raft_consensus.cc:493] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.178076  3987 tablet_service.cc:1511] Processing CreateTablet for tablet 24281667f1364c6fac542e54ed413502 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:20:42.186712  4306 raft_consensus.cc:515] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.196123  4273 tablet_bootstrap.cc:654] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.203884  4306 leader_election.cc:290] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:42.206681  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d7a192af654f9fa50d695be708ca81" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:42.207618  4080 raft_consensus.cc:2468] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:42.196103  3987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24281667f1364c6fac542e54ed413502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.214286  4120 leader_election.cc:304] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:42.217104  4295 raft_consensus.cc:2804] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.217552  4295 raft_consensus.cc:493] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.217931  4295 raft_consensus.cc:3060] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.220837  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d7a192af654f9fa50d695be708ca81" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:42.224391  4061 tablet_service.cc:1511] Processing CreateTablet for tablet 24281667f1364c6fac542e54ed413502 (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:20:42.226143  4061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24281667f1364c6fac542e54ed413502. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:42.226992  4120 leader_election.cc:343] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.245702  4140 tablet_service.cc:1511] Processing CreateTablet for tablet e08603ff2d9145e0b85e975bac56312c (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:20:42.247393  4140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08603ff2d9145e0b85e975bac56312c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.247797  4214 tablet_service.cc:1511] Processing CreateTablet for tablet e08603ff2d9145e0b85e975bac56312c (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:20:42.252835  4272 tablet_bootstrap.cc:492] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:42.254544  4272 ts_tablet_manager.cc:1403] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.433s	user 0.014s	sys 0.009s
I20260709 12:20:42.258316  4280 ts_tablet_manager.cc:1434] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.140s	user 0.008s	sys 0.000s
I20260709 12:20:42.259718  4214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08603ff2d9145e0b85e975bac56312c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:42.259759  4280 tablet_bootstrap.cc:492] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:42.260316  4272 raft_consensus.cc:359] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.266464  4272 raft_consensus.cc:385] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.270319  4272 raft_consensus.cc:740] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.274936  4272 consensus_queue.cc:260] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.273584  4287 raft_consensus.cc:493] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.278275  4287 raft_consensus.cc:515] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.273952  4280 tablet_bootstrap.cc:654] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.268988  4277 tablet_bootstrap.cc:654] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.282804  4287 leader_election.cc:290] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:42.315946  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ae7ddfcc1e49d4b6847e9c895edde7" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:42.316272  3987 tablet_service.cc:1511] Processing CreateTablet for tablet e08603ff2d9145e0b85e975bac56312c (DEFAULT_TABLE table=test-workload [id=4f8cf4dff4ce425baf6f4f0a66a2c687]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:20:42.317803  3987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08603ff2d9145e0b85e975bac56312c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:42.318141  3971 leader_election.cc:343] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.281678  4291 raft_consensus.cc:515] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.323764  4273 tablet_bootstrap.cc:492] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:42.324436  4273 ts_tablet_manager.cc:1403] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.171s	user 0.026s	sys 0.036s
I20260709 12:20:42.242841  4295 raft_consensus.cc:515] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.324849  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91cdbe8d3035425abc337e6e27846e42" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:42.326581  4295 leader_election.cc:290] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:20:42.328217  4194 leader_election.cc:343] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.332489  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d7a192af654f9fa50d695be708ca81" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:42.333711  4080 raft_consensus.cc:3060] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.334476  4272 ts_tablet_manager.cc:1434] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.079s	user 0.011s	sys 0.002s
I20260709 12:20:42.334692  4291 leader_election.cc:290] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:42.334939  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91cdbe8d3035425abc337e6e27846e42" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:42.335634  4005 raft_consensus.cc:3060] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.335701  4272 tablet_bootstrap.cc:492] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:42.338673  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d7a192af654f9fa50d695be708ca81" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
W20260709 12:20:42.340471  4120 leader_election.cc:343] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.328092  4273 raft_consensus.cc:359] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:42.352795  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:42.354735  4273 raft_consensus.cc:385] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.354409  4291 raft_consensus.cc:493] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.375159  4291 raft_consensus.cc:515] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.378574  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c72e9bb69f746c191320d2437c3ce19" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:42.379851  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c72e9bb69f746c191320d2437c3ce19" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:42.380709  4079 raft_consensus.cc:2468] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
W20260709 12:20:42.381283  4194 leader_election.cc:343] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.381999  4291 leader_election.cc:290] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:42.382221  4194 leader_election.cc:304] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:42.383795  4291 raft_consensus.cc:2804] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.384284  4291 raft_consensus.cc:493] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.384688  4291 raft_consensus.cc:3060] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.387715  4080 raft_consensus.cc:2468] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:42.369386  4273 raft_consensus.cc:740] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.389230  4120 leader_election.cc:304] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:42.371956  4005 raft_consensus.cc:2468] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:42.390537  4295 raft_consensus.cc:2804] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.391062  4295 raft_consensus.cc:697] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:42.394783  4194 leader_election.cc:304] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa
I20260709 12:20:42.396864  4295 consensus_queue.cc:237] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.398483  4315 raft_consensus.cc:2804] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.394142  4273 consensus_queue.cc:260] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.400668  4280 tablet_bootstrap.cc:492] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:42.401286  4280 ts_tablet_manager.cc:1403] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.142s	user 0.022s	sys 0.002s
I20260709 12:20:42.402294  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ae7ddfcc1e49d4b6847e9c895edde7" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:42.404186  3971 leader_election.cc:343] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.404700  3971 leader_election.cc:304] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:42.404871  4273 ts_tablet_manager.cc:1434] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.003s
I20260709 12:20:42.406073  4287 raft_consensus.cc:2749] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.406314  4273 tablet_bootstrap.cc:492] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:42.409052  4280 raft_consensus.cc:359] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.420408  4280 raft_consensus.cc:385] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.420804  4280 raft_consensus.cc:740] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.414227  4273 tablet_bootstrap.cc:654] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.402609  4272 tablet_bootstrap.cc:654] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.417111  4277 tablet_bootstrap.cc:492] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:42.424103  4277 ts_tablet_manager.cc:1403] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.315s	user 0.019s	sys 0.021s
I20260709 12:20:42.426054  4280 consensus_queue.cc:260] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.431604  4277 raft_consensus.cc:359] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.432565  4277 raft_consensus.cc:385] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.432873  4277 raft_consensus.cc:740] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.435717  4280 ts_tablet_manager.cc:1434] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.034s	user 0.001s	sys 0.006s
I20260709 12:20:42.437170  4280 tablet_bootstrap.cc:492] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:42.423971  4291 raft_consensus.cc:515] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.441686  4291 leader_election.cc:290] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:42.442698  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c72e9bb69f746c191320d2437c3ce19" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
W20260709 12:20:42.444991  4194 leader_election.cc:343] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.446780  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c72e9bb69f746c191320d2437c3ce19" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:42.447611  4080 raft_consensus.cc:3060] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.452477  3902 catalog_manager.cc:5697] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.474220  4277 consensus_queue.cc:260] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.478190  4315 raft_consensus.cc:697] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:42.479043  4272 tablet_bootstrap.cc:492] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:42.479660  4272 ts_tablet_manager.cc:1403] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.144s	user 0.036s	sys 0.003s
I20260709 12:20:42.479342  4315 consensus_queue.cc:237] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.480959  4080 raft_consensus.cc:2468] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:42.482596  4194 leader_election.cc:304] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:42.483234  4272 raft_consensus.cc:359] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.483927  4277 ts_tablet_manager.cc:1434] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.001s
I20260709 12:20:42.484266  4272 raft_consensus.cc:385] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.484597  4272 raft_consensus.cc:740] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.485263  4277 tablet_bootstrap.cc:492] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:42.488330  4273 tablet_bootstrap.cc:492] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:42.489084  4273 ts_tablet_manager.cc:1403] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.083s	user 0.019s	sys 0.017s
I20260709 12:20:42.492621  4273 raft_consensus.cc:359] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.493467  4273 raft_consensus.cc:385] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.493796  4273 raft_consensus.cc:740] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.485587  4272 consensus_queue.cc:260] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.495249  4291 raft_consensus.cc:2804] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.497915  4295 consensus_queue.cc:1048] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.502837  4272 ts_tablet_manager.cc:1434] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:20:42.504346  4272 tablet_bootstrap.cc:492] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:42.506357  4273 consensus_queue.cc:260] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.507097  4287 raft_consensus.cc:493] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.507699  4287 raft_consensus.cc:515] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.507516  3902 catalog_manager.cc:5697] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.510504  4273 ts_tablet_manager.cc:1434] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 12:20:42.511144  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:42.514865  4272 tablet_bootstrap.cc:654] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.517640  4292 raft_consensus.cc:493] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.518254  4292 raft_consensus.cc:515] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.522354  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:42.522544  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:42.523375  4287 leader_election.cc:290] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
W20260709 12:20:42.523957  3970 leader_election.cc:343] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.524652  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:42.525003  4046 leader_election.cc:343] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:42.527022  4045 leader_election.cc:343] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.527560  4045 leader_election.cc:304] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
W20260709 12:20:42.527877  3971 leader_election.cc:343] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.528108  4292 leader_election.cc:290] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:42.528533  3971 leader_election.cc:304] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:42.528729  4292 raft_consensus.cc:2749] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.529958  4287 raft_consensus.cc:2749] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.531301  4280 tablet_bootstrap.cc:654] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.538161  4295 consensus_queue.cc:1048] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.551332  4277 tablet_bootstrap.cc:654] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.554283  4291 raft_consensus.cc:697] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:42.556105  4273 tablet_bootstrap.cc:492] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:42.556337  4291 consensus_queue.cc:237] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.579062  4280 tablet_bootstrap.cc:492] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:42.579897  4280 ts_tablet_manager.cc:1403] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.143s	user 0.014s	sys 0.000s
I20260709 12:20:42.583720  4280 raft_consensus.cc:359] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.589254  4273 tablet_bootstrap.cc:654] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.585500  4280 raft_consensus.cc:385] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:20:42.585678  4045 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 58737.6us
I20260709 12:20:42.592712  4306 raft_consensus.cc:493] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.603724  4306 raft_consensus.cc:515] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.595358  4280 raft_consensus.cc:740] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.611347  3901 catalog_manager.cc:5697] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.613693  4272 tablet_bootstrap.cc:492] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:42.615692  4272 ts_tablet_manager.cc:1403] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.112s	user 0.032s	sys 0.006s
I20260709 12:20:42.616243  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb8a7e4c51b446578c698dcf9a253001" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:42.616690  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb8a7e4c51b446578c698dcf9a253001" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:42.614867  4306 leader_election.cc:290] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:20:42.618377  4120 leader_election.cc:343] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:42.619303  4120 leader_election.cc:343] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.620590  4120 leader_election.cc:304] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:42.606225  4327 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.619318  4280 consensus_queue.cc:260] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.622985  4327 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.636709  4327 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:42.638907  4280 ts_tablet_manager.cc:1434] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 12:20:42.623307  4306 raft_consensus.cc:2749] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.642395  4280 tablet_bootstrap.cc:492] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:42.643829  4272 raft_consensus.cc:359] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.645895  4272 raft_consensus.cc:385] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.646271  4272 raft_consensus.cc:740] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.647475  4273 tablet_bootstrap.cc:492] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:42.647460  4277 tablet_bootstrap.cc:492] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:42.648893  4277 ts_tablet_manager.cc:1403] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.164s	user 0.020s	sys 0.010s
I20260709 12:20:42.648281  4272 consensus_queue.cc:260] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.649708  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:42.651309  4272 ts_tablet_manager.cc:1434] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.035s	user 0.002s	sys 0.005s
I20260709 12:20:42.646143  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:42.648267  4273 ts_tablet_manager.cc:1403] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.093s	user 0.010s	sys 0.021s
I20260709 12:20:42.652459  4277 raft_consensus.cc:359] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.653491  4277 raft_consensus.cc:385] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.653834  4277 raft_consensus.cc:740] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.654670  4277 consensus_queue.cc:260] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.652529  4272 tablet_bootstrap.cc:492] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:42.657224  4277 ts_tablet_manager.cc:1434] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:20:42.655742  4273 raft_consensus.cc:359] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.657913  4273 raft_consensus.cc:385] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.658213  4273 raft_consensus.cc:740] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.658634  4277 tablet_bootstrap.cc:492] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:42.649572  4280 tablet_bootstrap.cc:654] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:42.651474  4046 leader_election.cc:343] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:42.660341  4045 leader_election.cc:343] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.659004  4273 consensus_queue.cc:260] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.661434  4045 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:42.663421  4292 raft_consensus.cc:2749] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.663617  4273 ts_tablet_manager.cc:1434] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:20:42.664876  4273 tablet_bootstrap.cc:492] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:42.666966  4277 tablet_bootstrap.cc:654] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.666966  4272 tablet_bootstrap.cc:654] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.677471  4273 tablet_bootstrap.cc:654] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.696384  4280 tablet_bootstrap.cc:492] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:42.697073  4280 ts_tablet_manager.cc:1403] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.055s	user 0.007s	sys 0.012s
I20260709 12:20:42.697958  4291 raft_consensus.cc:493] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.698714  4291 raft_consensus.cc:515] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.700575  4280 raft_consensus.cc:359] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.701442  4280 raft_consensus.cc:385] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.704155  4291 leader_election.cc:290] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:42.704857  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:42.713244  4280 raft_consensus.cc:740] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.709944  4306 raft_consensus.cc:493] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.713995  4306 raft_consensus.cc:515] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:42.714397  4194 leader_election.cc:343] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.715158  4280 consensus_queue.cc:260] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.716202  4306 leader_election.cc:290] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:42.718129  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:42.718809  4277 tablet_bootstrap.cc:492] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:42.718961  4228 raft_consensus.cc:2468] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:42.719419  4277 ts_tablet_manager.cc:1403] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.061s	user 0.028s	sys 0.024s
I20260709 12:20:42.721236  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
W20260709 12:20:42.723145  4120 leader_election.cc:343] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.723078  4277 raft_consensus.cc:359] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.724054  4277 raft_consensus.cc:385] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.724427  4277 raft_consensus.cc:740] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.724140  4120 leader_election.cc:304] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:42.725505  4306 raft_consensus.cc:2804] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.725455  4277 consensus_queue.cc:260] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.726048  4306 raft_consensus.cc:493] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.711069  4273 tablet_bootstrap.cc:492] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:42.727082  4273 ts_tablet_manager.cc:1403] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.002s
I20260709 12:20:42.728096  4277 ts_tablet_manager.cc:1434] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:42.726514  4306 raft_consensus.cc:3060] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.729482  4277 tablet_bootstrap.cc:492] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:42.730625  4273 raft_consensus.cc:359] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.731366  4273 raft_consensus.cc:385] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.728096  4280 ts_tablet_manager.cc:1434] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.003s
I20260709 12:20:42.731648  4273 raft_consensus.cc:740] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.737766  4272 tablet_bootstrap.cc:492] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:42.742277  4272 ts_tablet_manager.cc:1403] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.090s	user 0.015s	sys 0.003s
I20260709 12:20:42.745946  4272 raft_consensus.cc:359] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.738095  4273 consensus_queue.cc:260] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.749611  4272 raft_consensus.cc:385] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.749663  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:42.751489  4193 leader_election.cc:343] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.752005  4193 leader_election.cc:304] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:42.739079  4277 tablet_bootstrap.cc:654] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.747779  4280 tablet_bootstrap.cc:492] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:42.753306  4291 raft_consensus.cc:2749] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.752008  4273 ts_tablet_manager.cc:1434] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.005s
I20260709 12:20:42.755316  4273 tablet_bootstrap.cc:492] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:42.761071  4280 tablet_bootstrap.cc:654] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.763065  4273 tablet_bootstrap.cc:654] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.748787  4306 raft_consensus.cc:515] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.776959  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:42.777256  4280 tablet_bootstrap.cc:492] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:42.777700  4228 raft_consensus.cc:3060] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.778007  4280 ts_tablet_manager.cc:1403] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.008s
I20260709 12:20:42.781497  4280 raft_consensus.cc:359] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.782327  4280 raft_consensus.cc:385] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.782644  4280 raft_consensus.cc:740] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.785630  4277 tablet_bootstrap.cc:492] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:42.750219  4272 raft_consensus.cc:740] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.786309  4277 ts_tablet_manager.cc:1403] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.003s
I20260709 12:20:42.786343  4287 raft_consensus.cc:493] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.787092  4287 raft_consensus.cc:515] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.788206  4327 raft_consensus.cc:493] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.790282  4327 raft_consensus.cc:515] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.791188  4280 consensus_queue.cc:260] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.790195  4277 raft_consensus.cc:359] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.792004  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:42.792378  4277 raft_consensus.cc:385] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.792716  4327 leader_election.cc:290] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:42.792765  4277 raft_consensus.cc:740] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.794014  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f2a3fbee56421aa8768a4d7f275ba9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:42.792941  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb8a7e4c51b446578c698dcf9a253001" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:42.789638  4306 leader_election.cc:290] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:42.794739  4004 raft_consensus.cc:2468] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:42.794432  4272 consensus_queue.cc:260] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.795868  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f2a3fbee56421aa8768a4d7f275ba9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:42.797240  4046 leader_election.cc:304] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:42.794421  4277 consensus_queue.cc:260] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.798295  4327 raft_consensus.cc:2804] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.798738  4327 raft_consensus.cc:493] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.799094  4327 raft_consensus.cc:3060] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.800148  4277 ts_tablet_manager.cc:1434] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:20:42.801155  4154 raft_consensus.cc:2468] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:42.802847  3970 leader_election.cc:304] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:42.793493  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb8a7e4c51b446578c698dcf9a253001" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:42.805182  4277 tablet_bootstrap.cc:492] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
W20260709 12:20:42.805758  3971 leader_election.cc:343] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:42.807883  4120 leader_election.cc:343] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:42.808823  4045 leader_election.cc:343] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.809917  4287 leader_election.cc:290] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:42.810597  4287 raft_consensus.cc:2804] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.811053  4287 raft_consensus.cc:493] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.811434  4287 raft_consensus.cc:3060] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.808697  4280 ts_tablet_manager.cc:1434] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.005s
I20260709 12:20:42.814855  4280 tablet_bootstrap.cc:492] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:42.815092  4335 raft_consensus.cc:493] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.815681  4335 raft_consensus.cc:515] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.819093  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:42.819808  4080 raft_consensus.cc:2468] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:42.821465  4120 leader_election.cc:304] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:42.823382  4335 leader_election.cc:290] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:42.823767  4277 tablet_bootstrap.cc:654] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.824101  4335 raft_consensus.cc:2804] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.824712  4335 raft_consensus.cc:493] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.825119  4335 raft_consensus.cc:3060] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.798249  4272 ts_tablet_manager.cc:1434] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.002s
I20260709 12:20:42.826973  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:42.827668  4280 tablet_bootstrap.cc:654] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.827790  4227 raft_consensus.cc:2468] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:42.830363  4327 raft_consensus.cc:515] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.787719  4228 raft_consensus.cc:2468] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:42.832641  4120 leader_election.cc:304] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:42.833611  4295 raft_consensus.cc:2804] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.834144  4295 raft_consensus.cc:697] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:42.835929  4295 consensus_queue.cc:237] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.828922  4272 tablet_bootstrap.cc:492] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:42.857607  4347 raft_consensus.cc:493] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.857120  3903 catalog_manager.cc:5697] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.858244  4347 raft_consensus.cc:515] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.859284  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f2a3fbee56421aa8768a4d7f275ba9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:42.860057  4005 raft_consensus.cc:3060] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.860587  4347 leader_election.cc:290] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:42.863575  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01529dc6d8944b9697b9a87c4e4bf450" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:42.865325  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73f2a3fbee56421aa8768a4d7f275ba9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
W20260709 12:20:42.866820  4045 leader_election.cc:343] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.867317  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01529dc6d8944b9697b9a87c4e4bf450" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:42.865875  4327 leader_election.cc:290] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:42.868038  4080 raft_consensus.cc:2468] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
W20260709 12:20:42.866756  3971 leader_election.cc:343] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.870761  3971 leader_election.cc:304] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:42.872138  4347 raft_consensus.cc:2804] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.872537  4347 raft_consensus.cc:493] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.872848  4347 raft_consensus.cc:3060] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.879799  4272 tablet_bootstrap.cc:654] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.881831  4005 raft_consensus.cc:2468] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:42.883653  4046 leader_election.cc:304] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:42.885084  4327 raft_consensus.cc:2804] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.886345  4273 tablet_bootstrap.cc:492] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:42.886885  4273 ts_tablet_manager.cc:1403] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.132s	user 0.016s	sys 0.015s
I20260709 12:20:42.886161  4327 raft_consensus.cc:697] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:42.888676  4327 consensus_queue.cc:237] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.889429  4347 raft_consensus.cc:515] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.893548  4280 tablet_bootstrap.cc:492] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:42.894282  4280 ts_tablet_manager.cc:1403] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.080s	user 0.018s	sys 0.016s
I20260709 12:20:42.895787  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01529dc6d8944b9697b9a87c4e4bf450" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
W20260709 12:20:42.897665  3971 leader_election.cc:343] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.898530  4277 tablet_bootstrap.cc:492] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:42.900501  4277 ts_tablet_manager.cc:1403] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.096s	user 0.022s	sys 0.015s
I20260709 12:20:42.897699  4280 raft_consensus.cc:359] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.902851  4280 raft_consensus.cc:385] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.903107  4280 raft_consensus.cc:740] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.901826  4335 raft_consensus.cc:515] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.905603  4280 consensus_queue.cc:260] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.906172  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01529dc6d8944b9697b9a87c4e4bf450" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:42.906786  4080 raft_consensus.cc:3060] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.908581  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:42.909320  4079 raft_consensus.cc:3060] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.908605  4280 ts_tablet_manager.cc:1434] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:20:42.910481  4287 raft_consensus.cc:515] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:42.912807  4287 leader_election.cc:290] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
W20260709 12:20:42.898069  3970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 95334.8us
I20260709 12:20:42.914932  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb8a7e4c51b446578c698dcf9a253001" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:42.908576  4277 raft_consensus.cc:359] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.916610  4277 raft_consensus.cc:385] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.916350  4280 tablet_bootstrap.cc:492] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:42.916981  4277 raft_consensus.cc:740] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.918045  4277 consensus_queue.cc:260] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.920616  4277 ts_tablet_manager.cc:1434] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:20:42.916636  4154 raft_consensus.cc:3060] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.921883  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb8a7e4c51b446578c698dcf9a253001" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:42.922437  4277 tablet_bootstrap.cc:492] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
W20260709 12:20:42.923765  3971 leader_election.cc:343] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.928356  4078 raft_consensus.cc:3060] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.929867  4280 tablet_bootstrap.cc:654] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.932505  4335 leader_election.cc:290] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:42.933727  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:42.934607  4228 raft_consensus.cc:3060] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.935001  4080 raft_consensus.cc:2468] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:42.936026  4291 consensus_queue.cc:1048] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.937866  3971 leader_election.cc:304] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:42.938587  4079 raft_consensus.cc:2468] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:42.902616  4347 leader_election.cc:290] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:42.939976  4120 leader_election.cc:304] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:42.941238  4273 raft_consensus.cc:359] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.941763  4154 raft_consensus.cc:2468] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:42.942158  4273 raft_consensus.cc:385] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.942466  4273 raft_consensus.cc:740] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.940033  4347 raft_consensus.cc:2804] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.943297  4335 raft_consensus.cc:2804] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.943447  4347 raft_consensus.cc:697] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:42.944228  4273 consensus_queue.cc:260] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.946204  4335 raft_consensus.cc:697] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:42.950230  4335 consensus_queue.cc:237] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.952013  4272 tablet_bootstrap.cc:492] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:42.952760  4272 ts_tablet_manager.cc:1403] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.124s	user 0.013s	sys 0.012s
I20260709 12:20:42.954218  3970 leader_election.cc:304] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa
I20260709 12:20:42.957647  3903 catalog_manager.cc:5697] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:42.962510  4120 consensus_peers.cc:597] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:42.961665  4287 raft_consensus.cc:2804] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.957640  4272 raft_consensus.cc:359] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.965548  4272 raft_consensus.cc:385] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.966015  4272 raft_consensus.cc:740] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.966866  4272 consensus_queue.cc:260] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:42.969358  4272 ts_tablet_manager.cc:1434] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 12:20:42.970739  4277 tablet_bootstrap.cc:654] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.972167  4272 tablet_bootstrap.cc:492] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:42.979905  4347 consensus_queue.cc:237] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:42.980559  4280 tablet_bootstrap.cc:492] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:42.982026  4280 ts_tablet_manager.cc:1403] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.002s
I20260709 12:20:42.986985  4273 ts_tablet_manager.cc:1434] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.100s	user 0.008s	sys 0.000s
I20260709 12:20:42.986027  4280 raft_consensus.cc:359] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:42.991918  4280 raft_consensus.cc:385] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.992672  4280 raft_consensus.cc:740] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.994148  4306 consensus_queue.cc:1048] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.994454  4280 consensus_queue.cc:260] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.001216  4280 ts_tablet_manager.cc:1434] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:20:43.001659  4291 consensus_queue.cc:1048] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:43.002923  4327 raft_consensus.cc:493] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.005626  4273 tablet_bootstrap.cc:492] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:43.008235  4280 tablet_bootstrap.cc:492] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:43.010497  4327 raft_consensus.cc:515] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.020104  4327 leader_election.cc:290] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:43.021899  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91cdbe8d3035425abc337e6e27846e42" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:43.022097  3903 catalog_manager.cc:5697] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.023633  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91cdbe8d3035425abc337e6e27846e42" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:43.009985  4287 raft_consensus.cc:697] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:43.028874  4273 tablet_bootstrap.cc:654] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.029036  4287 consensus_queue.cc:237] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.031545  4046 leader_election.cc:304] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:43.046767  4272 tablet_bootstrap.cc:654] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.051621  4292 raft_consensus.cc:2749] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:43.047830  4228 raft_consensus.cc:2468] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:43.063796  4325 consensus_queue.cc:1048] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:20:43.066592  4277 tablet_bootstrap.cc:492] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:43.067281  4277 ts_tablet_manager.cc:1403] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.008s
I20260709 12:20:43.068491  4335 raft_consensus.cc:493] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.069108  4335 raft_consensus.cc:515] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.071487  4335 leader_election.cc:290] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:43.072813  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:43.073673  4004 raft_consensus.cc:2468] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:43.075557  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:43.078169  4120 leader_election.cc:343] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.079138  4120 leader_election.cc:304] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:43.085160  4335 raft_consensus.cc:2804] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.086444  4335 raft_consensus.cc:493] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.087775  4292 raft_consensus.cc:493] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.090400  4292 raft_consensus.cc:515] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.091310  3901 catalog_manager.cc:5697] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.095439  4292 leader_election.cc:290] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:43.098107  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ae7ddfcc1e49d4b6847e9c895edde7" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:43.098960  4004 raft_consensus.cc:2468] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:43.100728  4046 leader_election.cc:304] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:43.096876  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ae7ddfcc1e49d4b6847e9c895edde7" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:43.105680  4046 leader_election.cc:343] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.108342  4292 raft_consensus.cc:2804] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.108990  4292 raft_consensus.cc:493] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.109434  4292 raft_consensus.cc:3060] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.109886  4335 raft_consensus.cc:3060] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.089253  4277 raft_consensus.cc:359] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.116947  4277 raft_consensus.cc:385] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.120092  4277 raft_consensus.cc:740] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.127038  4277 consensus_queue.cc:260] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.134310  3900 catalog_manager.cc:5697] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.137835  4292 raft_consensus.cc:515] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.141345  4292 leader_election.cc:290] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:43.143080  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ae7ddfcc1e49d4b6847e9c895edde7" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:43.143332  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ae7ddfcc1e49d4b6847e9c895edde7" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:43.143900  4005 raft_consensus.cc:3060] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:43.145190  4046 leader_election.cc:343] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.148134  4280 tablet_bootstrap.cc:654] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.164383  4005 raft_consensus.cc:2468] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:43.167567  4277 ts_tablet_manager.cc:1434] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.100s	user 0.009s	sys 0.000s
I20260709 12:20:43.168921  4277 tablet_bootstrap.cc:492] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:43.170248  4273 tablet_bootstrap.cc:492] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:43.170782  4273 ts_tablet_manager.cc:1403] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.183s	user 0.012s	sys 0.004s
I20260709 12:20:43.176908  4273 raft_consensus.cc:359] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.177886  4273 raft_consensus.cc:385] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.173493  4046 leader_election.cc:304] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:43.178284  4273 raft_consensus.cc:740] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.179191  4292 raft_consensus.cc:2804] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.179000  4273 consensus_queue.cc:260] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.179745  4292 raft_consensus.cc:697] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:43.180670  4292 consensus_queue.cc:237] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.188875  4335 raft_consensus.cc:515] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.198958  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
W20260709 12:20:43.200896  4120 leader_election.cc:343] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.215903  4273 ts_tablet_manager.cc:1434] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.045s	user 0.003s	sys 0.005s
I20260709 12:20:43.217247  4273 tablet_bootstrap.cc:492] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:43.218968  4277 tablet_bootstrap.cc:654] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.218766  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:43.219627  4005 raft_consensus.cc:3060] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.236438  4287 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.237313  4273 tablet_bootstrap.cc:654] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.237129  4287 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.239624  4287 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:43.242005  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:43.244230  3970 leader_election.cc:343] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.245268  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:43.246096  4078 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:43.246686  4277 tablet_bootstrap.cc:492] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:43.247282  4277 ts_tablet_manager.cc:1403] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.079s	user 0.019s	sys 0.011s
I20260709 12:20:43.247155  4327 raft_consensus.cc:493] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.247519  3971 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.248291  4327 raft_consensus.cc:515] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.248876  4287 raft_consensus.cc:2804] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.249346  4287 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.249855  4287 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.250454  4277 raft_consensus.cc:359] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.251210  4277 raft_consensus.cc:385] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.251479  4277 raft_consensus.cc:740] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.251895  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69fa505a55eb4823a412608c712fb378" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:43.252233  4277 consensus_queue.cc:260] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:43.253548  4046 leader_election.cc:343] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.254782  4277 ts_tablet_manager.cc:1434] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:43.255438  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69fa505a55eb4823a412608c712fb378" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:43.255865  4277 tablet_bootstrap.cc:492] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
W20260709 12:20:43.257126  4045 leader_election.cc:343] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.257656  4045 leader_election.cc:304] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:43.258644  4327 leader_election.cc:290] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.258523  3903 catalog_manager.cc:5697] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.259606  4327 raft_consensus.cc:2749] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.262773  4004 raft_consensus.cc:3060] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.263468  4277 tablet_bootstrap.cc:654] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.264484  4273 tablet_bootstrap.cc:492] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:43.265098  4273 ts_tablet_manager.cc:1403] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.005s
I20260709 12:20:43.269218  4273 raft_consensus.cc:359] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.270177  4273 raft_consensus.cc:385] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.270470  4273 raft_consensus.cc:740] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.271607  4273 consensus_queue.cc:260] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.273833  4273 ts_tablet_manager.cc:1434] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:43.274055  4365 raft_consensus.cc:493] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.274672  4365 raft_consensus.cc:515] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.275086  4273 tablet_bootstrap.cc:492] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:43.278402  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7536eba19a0467ab642af1d051f5b77" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:43.280084  4193 leader_election.cc:343] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.281116  4365 leader_election.cc:290] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:43.284579  4273 tablet_bootstrap.cc:654] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.286823  4335 leader_election.cc:290] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:43.293336  4272 tablet_bootstrap.cc:492] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:43.295328  4272 ts_tablet_manager.cc:1403] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.323s	user 0.064s	sys 0.085s
I20260709 12:20:43.298924  4272 raft_consensus.cc:359] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.299763  4272 raft_consensus.cc:385] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.300072  4272 raft_consensus.cc:740] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.294047  4154 raft_consensus.cc:3060] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.300916  4272 consensus_queue.cc:260] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.303377  4272 ts_tablet_manager.cc:1434] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:43.304440  4272 tablet_bootstrap.cc:492] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:43.306998  4356 raft_consensus.cc:493] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.307601  4356 raft_consensus.cc:515] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.311012  4356 leader_election.cc:290] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.311515  4347 raft_consensus.cc:493] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.312076  4347 raft_consensus.cc:515] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.312811  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:43.314240  4347 leader_election.cc:290] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.315531  4371 consensus_queue.cc:1048] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:43.315914  4194 consensus_peers.cc:597] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:43.316426  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:43.317131  4227 raft_consensus.cc:2468] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:43.316360  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:43.318126  4153 raft_consensus.cc:2468] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:43.318576  4005 raft_consensus.cc:2468] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:43.315848  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:43.300832  4280 tablet_bootstrap.cc:492] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:43.319742  4228 raft_consensus.cc:2468] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:43.320395  4280 ts_tablet_manager.cc:1403] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.315s	user 0.012s	sys 0.003s
I20260709 12:20:43.318902  3971 leader_election.cc:304] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:43.321753  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7536eba19a0467ab642af1d051f5b77" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:43.322510  4079 raft_consensus.cc:2468] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:43.324092  4347 raft_consensus.cc:2804] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.324558  4347 raft_consensus.cc:493] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.324911  4347 raft_consensus.cc:3060] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.325266  4194 leader_election.cc:304] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.325239  4287 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.326280  4365 raft_consensus.cc:2804] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.326658  4365 raft_consensus.cc:493] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.326761  4120 leader_election.cc:304] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:43.327049  4365 raft_consensus.cc:3060] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.327620  4306 raft_consensus.cc:2804] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.328154  4306 raft_consensus.cc:697] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:43.328630  3971 leader_election.cc:304] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.329070  4306 consensus_queue.cc:237] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.330091  4295 consensus_queue.cc:1048] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:43.327242  4280 raft_consensus.cc:359] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.333374  4280 raft_consensus.cc:385] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.333642  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:43.333667  4280 raft_consensus.cc:740] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.334435  4152 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.334923  4280 consensus_queue.cc:260] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.338640  4272 tablet_bootstrap.cc:654] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.333797  4287 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:43.340167  4356 raft_consensus.cc:2804] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:43.340620  4356 raft_consensus.cc:493] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.340970  4356 raft_consensus.cc:3060] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.341678  4280 ts_tablet_manager.cc:1434] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:20:43.347383  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:43.348141  4079 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.347695  4280 tablet_bootstrap.cc:492] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:43.351300  4356 raft_consensus.cc:515] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.352644  4004 raft_consensus.cc:1240] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Rejecting Update request from peer 6994df41341e478d89f7c105f550ff93 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:20:43.354974  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:43.355688  4228 raft_consensus.cc:3060] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.356308  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:43.362746  4335 consensus_queue.cc:1059] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:43.366108  4371 consensus_queue.cc:1048] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:43.366364  4295 raft_consensus.cc:3055] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:43.343449  4152 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:43.366936  4295 raft_consensus.cc:740] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:43.368183  4280 tablet_bootstrap.cc:654] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.367892  4295 consensus_queue.cc:260] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.368518  3970 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:43.369287  4295 raft_consensus.cc:3060] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.369606  4287 raft_consensus.cc:2804] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.339815  4347 raft_consensus.cc:515] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.376328  4347 leader_election.cc:290] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.380110  4287 raft_consensus.cc:697] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:43.377362  4356 leader_election.cc:290] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.377542  4228 raft_consensus.cc:2468] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:20:43.393000  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:43.393858  4227 raft_consensus.cc:3060] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.394479  4273 tablet_bootstrap.cc:492] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:43.395020  4273 ts_tablet_manager.cc:1403] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.120s	user 0.026s	sys 0.015s
I20260709 12:20:43.395187  4365 raft_consensus.cc:515] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.387416  4287 consensus_queue.cc:237] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.398440  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7536eba19a0467ab642af1d051f5b77" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:43.398655  4365 leader_election.cc:290] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:43.399082  4078 raft_consensus.cc:3060] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.399706  4273 raft_consensus.cc:359] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.400416  4273 raft_consensus.cc:385] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.400671  4273 raft_consensus.cc:740] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.401562  4273 consensus_queue.cc:260] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.402318  4306 consensus_queue.cc:1048] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:43.404992  4356 raft_consensus.cc:493] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.405184  4273 ts_tablet_manager.cc:1434] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:20:43.406886  3971 leader_election.cc:304] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.407682  4356 raft_consensus.cc:515] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.408399  4348 raft_consensus.cc:2804] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:43.411078  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7536eba19a0467ab642af1d051f5b77" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
W20260709 12:20:43.412796  4193 leader_election.cc:343] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.412819  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:43.414250  3971 consensus_peers.cc:597] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 -> Peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Couldn't send request to peer 72940e4e52944adb81608e286ad58a59. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:43.415007  3971 leader_election.cc:343] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.387394  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:43.409968  4356 leader_election.cc:290] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.417310  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:43.420784  3970 leader_election.cc:343] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.421299  3970 leader_election.cc:304] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:43.422883  4356 raft_consensus.cc:2749] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.416983  4152 raft_consensus.cc:3060] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.424811  4273 tablet_bootstrap.cc:492] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:43.431080  4078 raft_consensus.cc:2468] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:43.432821  4194 leader_election.cc:304] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.434301  4374 raft_consensus.cc:2804] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.434804  4374 raft_consensus.cc:697] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:43.435482  4277 tablet_bootstrap.cc:492] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:43.435664  4374 consensus_queue.cc:237] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.436950  4273 tablet_bootstrap.cc:654] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.438903  4227 raft_consensus.cc:2468] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:43.439970  4277 ts_tablet_manager.cc:1403] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.184s	user 0.023s	sys 0.027s
I20260709 12:20:43.415190  4079 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:43.454263  3971 leader_election.cc:304] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:43.456131  4347 raft_consensus.cc:2804] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.466645  4347 raft_consensus.cc:697] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:43.467747  4347 consensus_queue.cc:237] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.474174  4277 raft_consensus.cc:359] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.475169  4277 raft_consensus.cc:385] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.475456  4277 raft_consensus.cc:740] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.476194  4277 consensus_queue.cc:260] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.477843  4371 raft_consensus.cc:493] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.453254  4348 raft_consensus.cc:697] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:43.482738  4287 consensus_queue.cc:1048] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:20:43.478452  4371 raft_consensus.cc:515] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.494606  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75948129b8264d75972f4660893bd024" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:43.494886  4306 consensus_queue.cc:1048] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:43.489599  3901 catalog_manager.cc:5697] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:43.497160  3971 consensus_peers.cc:597] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:43.498353  4046 leader_election.cc:343] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.483146  4356 consensus_queue.cc:1048] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
W20260709 12:20:43.478998  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:43.502650  4277 ts_tablet_manager.cc:1434] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.001s
I20260709 12:20:43.486690  4348 consensus_queue.cc:237] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.503926  4277 tablet_bootstrap.cc:492] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:43.513940  4327 raft_consensus.cc:493] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.515235  4327 raft_consensus.cc:515] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.525005  4272 tablet_bootstrap.cc:492] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:43.525764  4272 ts_tablet_manager.cc:1403] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.222s	user 0.022s	sys 0.029s
I20260709 12:20:43.529007  4280 tablet_bootstrap.cc:492] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:43.529623  4280 ts_tablet_manager.cc:1403] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.183s	user 0.014s	sys 0.000s
I20260709 12:20:43.517688  4327 leader_election.cc:290] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:43.501446  4152 raft_consensus.cc:2468] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:43.536859  4272 raft_consensus.cc:359] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.543592  4272 raft_consensus.cc:385] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.544083  4272 raft_consensus.cc:740] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.539949  4277 tablet_bootstrap.cc:654] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.520032  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:43.544996  4272 consensus_queue.cc:260] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.519917  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:43.547079  4371 leader_election.cc:290] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
W20260709 12:20:43.550098  4046 leader_election.cc:343] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.550545  4272 ts_tablet_manager.cc:1434] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:20:43.550354  3902 catalog_manager.cc:5697] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.560092  4280 raft_consensus.cc:359] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.561093  4280 raft_consensus.cc:385] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.561401  4280 raft_consensus.cc:740] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
W20260709 12:20:43.552752  4046 leader_election.cc:343] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.563160  4046 leader_election.cc:304] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:43.563302  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75948129b8264d75972f4660893bd024" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:43.564141  4371 raft_consensus.cc:2749] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.564607  4280 consensus_queue.cc:260] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:43.565263  4045 leader_election.cc:343] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.565893  4045 leader_election.cc:304] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.567425  4371 raft_consensus.cc:2749] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.568588  4280 ts_tablet_manager.cc:1434] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 12:20:43.570369  4280 tablet_bootstrap.cc:492] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:43.580680  4272 tablet_bootstrap.cc:492] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:43.576439  3901 catalog_manager.cc:5697] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.588742  4273 tablet_bootstrap.cc:492] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:43.589376  4273 ts_tablet_manager.cc:1403] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.174s	user 0.063s	sys 0.036s
I20260709 12:20:43.599320  4273 raft_consensus.cc:359] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.600133  4273 raft_consensus.cc:385] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.600445  4273 raft_consensus.cc:740] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.602319  3901 catalog_manager.cc:5697] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 6994df41341e478d89f7c105f550ff93 (127.26.123.3) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.604445  4280 tablet_bootstrap.cc:654] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.606508  4277 tablet_bootstrap.cc:492] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:43.610704  4277 ts_tablet_manager.cc:1403] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.107s	user 0.014s	sys 0.006s
I20260709 12:20:43.613934  4277 raft_consensus.cc:359] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.614759  4277 raft_consensus.cc:385] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.615067  4277 raft_consensus.cc:740] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.615777  4277 consensus_queue.cc:260] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.601136  4273 consensus_queue.cc:260] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.619151  4273 ts_tablet_manager.cc:1434] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:20:43.620584  4273 tablet_bootstrap.cc:492] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:43.632068  4273 tablet_bootstrap.cc:654] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.638374  4277 ts_tablet_manager.cc:1434] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:20:43.642483  4277 tablet_bootstrap.cc:492] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:43.643939  4306 raft_consensus.cc:493] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.641259  4280 tablet_bootstrap.cc:492] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:43.644799  4280 ts_tablet_manager.cc:1403] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.075s	user 0.020s	sys 0.000s
I20260709 12:20:43.644534  4306 raft_consensus.cc:515] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.646134  4272 tablet_bootstrap.cc:654] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.646936  4306 leader_election.cc:290] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.649586  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7536eba19a0467ab642af1d051f5b77" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:43.652015  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7536eba19a0467ab642af1d051f5b77" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:43.653148  4078 raft_consensus.cc:2393] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6994df41341e478d89f7c105f550ff93 in current term 1: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:43.653716  4277 tablet_bootstrap.cc:654] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.654390  4120 leader_election.cc:304] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
W20260709 12:20:43.659756  4046 consensus_peers.cc:597] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa -> Peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Couldn't send request to peer 72940e4e52944adb81608e286ad58a59. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:43.655825  4306 raft_consensus.cc:3060] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.663501  4272 tablet_bootstrap.cc:492] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:43.664152  4272 ts_tablet_manager.cc:1403] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.000s
I20260709 12:20:43.652019  4280 raft_consensus.cc:359] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.667721  4280 raft_consensus.cc:385] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.667987  4280 raft_consensus.cc:740] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.668723  4272 raft_consensus.cc:359] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.669029  4280 consensus_queue.cc:260] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.671386  4280 ts_tablet_manager.cc:1434] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 12:20:43.670849  3900 catalog_manager.cc:5697] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:43.672066  4277 tablet_bootstrap.cc:492] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:43.672497  4280 tablet_bootstrap.cc:492] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:43.672648  4277 ts_tablet_manager.cc:1403] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.003s
I20260709 12:20:43.673621  4306 raft_consensus.cc:2749] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.675908  4277 raft_consensus.cc:359] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.676749  4277 raft_consensus.cc:385] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.677034  4277 raft_consensus.cc:740] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.677984  4277 consensus_queue.cc:260] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.679636  4280 tablet_bootstrap.cc:654] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.680301  4277 ts_tablet_manager.cc:1434] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:20:43.681493  4277 tablet_bootstrap.cc:492] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:43.682219  4374 raft_consensus.cc:493] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.682792  4374 raft_consensus.cc:515] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.686344  4374 leader_election.cc:290] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:43.687119  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:43.688140  4277 tablet_bootstrap.cc:654] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.669526  4272 raft_consensus.cc:385] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.690284  4272 raft_consensus.cc:740] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.691253  4272 consensus_queue.cc:260] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.696060  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:43.698264  4273 tablet_bootstrap.cc:492] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:43.698959  4273 ts_tablet_manager.cc:1403] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.079s	user 0.015s	sys 0.010s
I20260709 12:20:43.700891  4272 ts_tablet_manager.cc:1434] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:20:43.702365  4272 tablet_bootstrap.cc:492] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:43.688060  4004 raft_consensus.cc:2393] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 6994df41341e478d89f7c105f550ff93 in this term.
I20260709 12:20:43.704319  4194 leader_election.cc:304] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.706733  4365 raft_consensus.cc:3060] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.709369  4292 consensus_queue.cc:1048] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:43.709937  4273 raft_consensus.cc:359] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.711500  4273 raft_consensus.cc:385] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.711825  4273 raft_consensus.cc:740] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.712613  4273 consensus_queue.cc:260] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.715654  4272 tablet_bootstrap.cc:654] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.717283  4277 tablet_bootstrap.cc:492] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:43.717991  4277 ts_tablet_manager.cc:1403] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.011s
I20260709 12:20:43.720068  4280 tablet_bootstrap.cc:492] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:43.720695  4280 ts_tablet_manager.cc:1403] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.004s
I20260709 12:20:43.720240  4273 ts_tablet_manager.cc:1434] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:20:43.721227  4277 raft_consensus.cc:359] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.722049  4277 raft_consensus.cc:385] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.722326  4277 raft_consensus.cc:740] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.722575  4273 tablet_bootstrap.cc:492] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:43.723086  4277 consensus_queue.cc:260] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.725386  4277 ts_tablet_manager.cc:1434] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:20:43.726506  4280 raft_consensus.cc:359] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.727396  4280 raft_consensus.cc:385] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.727756  4280 raft_consensus.cc:740] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.729074  4272 tablet_bootstrap.cc:492] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:43.728708  4280 consensus_queue.cc:260] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.729686  4272 ts_tablet_manager.cc:1403] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.004s
I20260709 12:20:43.730993  4273 tablet_bootstrap.cc:654] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.734437  4280 ts_tablet_manager.cc:1434] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.004s
I20260709 12:20:43.736652  4280 tablet_bootstrap.cc:492] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:43.737344  4272 raft_consensus.cc:359] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.738242  4272 raft_consensus.cc:385] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.738534  4272 raft_consensus.cc:740] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.735558  4277 tablet_bootstrap.cc:492] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:43.743069  4273 tablet_bootstrap.cc:492] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:43.743701  4273 ts_tablet_manager.cc:1403] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 12:20:43.744196  4280 tablet_bootstrap.cc:654] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.746682  4273 raft_consensus.cc:359] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.747505  4273 raft_consensus.cc:385] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.747820  4273 raft_consensus.cc:740] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.748890  4273 consensus_queue.cc:260] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.751760  4371 raft_consensus.cc:493] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.749519  4272 consensus_queue.cc:260] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.752285  4371 raft_consensus.cc:515] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.755271  4272 ts_tablet_manager.cc:1434] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:20:43.756414  4272 tablet_bootstrap.cc:492] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:43.757864  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e92ca9c3274410aa6ea1aaa11671569" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:43.759610  4045 leader_election.cc:343] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.762110  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e92ca9c3274410aa6ea1aaa11671569" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:43.762915  4272 tablet_bootstrap.cc:654] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.762840  4227 raft_consensus.cc:2468] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:43.763298  4371 leader_election.cc:290] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.751487  4277 tablet_bootstrap.cc:654] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.764113  4046 leader_election.cc:304] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.765460  4371 raft_consensus.cc:2804] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.765908  4371 raft_consensus.cc:493] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.766275  4371 raft_consensus.cc:3060] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.775774  4371 raft_consensus.cc:515] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.780942  4347 raft_consensus.cc:493] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.782130  4273 ts_tablet_manager.cc:1434] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.038s	user 0.011s	sys 0.002s
I20260709 12:20:43.781638  4347 raft_consensus.cc:515] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.784493  4347 leader_election.cc:290] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:43.786645  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d7a192af654f9fa50d695be708ca81" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:43.791010  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11d7a192af654f9fa50d695be708ca81" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:43.793915  4348 raft_consensus.cc:493] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.794705  4348 raft_consensus.cc:515] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.799535  3971 leader_election.cc:304] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.801040  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75948129b8264d75972f4660893bd024" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:43.801892  4078 raft_consensus.cc:2468] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:43.805029  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75948129b8264d75972f4660893bd024" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:43.807663  4385 raft_consensus.cc:2749] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.810187  3971 leader_election.cc:304] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
W20260709 12:20:43.811687  3970 leader_election.cc:343] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.797489  4348 leader_election.cc:290] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:43.817930  4385 raft_consensus.cc:2804] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.818539  4385 raft_consensus.cc:493] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.818895  4385 raft_consensus.cc:3060] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.780328  4365 raft_consensus.cc:2749] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.822335  4347 consensus_queue.cc:1048] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:43.832963  4347 raft_consensus.cc:493] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.833634  4347 raft_consensus.cc:515] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.835862  4347 leader_election.cc:290] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:43.836637  4280 tablet_bootstrap.cc:492] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:43.837119  4280 ts_tablet_manager.cc:1403] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.101s	user 0.032s	sys 0.050s
I20260709 12:20:43.839573  4280 raft_consensus.cc:359] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.840291  4280 raft_consensus.cc:385] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.840684  4280 raft_consensus.cc:740] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.841542  4280 consensus_queue.cc:260] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.842752  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c72e9bb69f746c191320d2437c3ce19" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:43.845466  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c72e9bb69f746c191320d2437c3ce19" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:43.849493  4280 ts_tablet_manager.cc:1434] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:20:43.850922  4280 tablet_bootstrap.cc:492] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:43.856876  4273 tablet_bootstrap.cc:492] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:43.862504  4280 tablet_bootstrap.cc:654] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.864771  4385 raft_consensus.cc:515] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.868701  4277 tablet_bootstrap.cc:492] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:43.869325  4277 ts_tablet_manager.cc:1403] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.134s	user 0.030s	sys 0.028s
I20260709 12:20:43.870064  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e92ca9c3274410aa6ea1aaa11671569" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:43.871465  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e92ca9c3274410aa6ea1aaa11671569" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
W20260709 12:20:43.871719  4045 leader_election.cc:343] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.872151  4227 raft_consensus.cc:3060] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.872447  4277 raft_consensus.cc:359] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.873162  4277 raft_consensus.cc:385] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.873453  4277 raft_consensus.cc:740] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.874257  4306 raft_consensus.cc:493] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.874524  4277 consensus_queue.cc:260] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.874836  4306 raft_consensus.cc:515] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.877102  4306 leader_election.cc:290] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.877296  4277 ts_tablet_manager.cc:1434] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:20:43.877029  4280 tablet_bootstrap.cc:492] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:43.878153  4280 ts_tablet_manager.cc:1403] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.007s
I20260709 12:20:43.878476  4277 tablet_bootstrap.cc:492] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:43.878671  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69fa505a55eb4823a412608c712fb378" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:43.879401  4079 raft_consensus.cc:2468] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:43.880877  4120 leader_election.cc:304] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:43.881263  4280 raft_consensus.cc:359] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.882257  4280 raft_consensus.cc:385] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.882557  4280 raft_consensus.cc:740] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.883302  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69fa505a55eb4823a412608c712fb378" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:43.884016  4226 raft_consensus.cc:2468] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:43.883741  4280 consensus_queue.cc:260] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:43.885342  4306 raft_consensus.cc:2804] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.885754  4306 raft_consensus.cc:493] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.886103  4306 raft_consensus.cc:3060] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.886626  4280 ts_tablet_manager.cc:1434] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:43.887696  4280 tablet_bootstrap.cc:492] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:43.878705  4365 raft_consensus.cc:493] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.891045  4365 raft_consensus.cc:515] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:43.892602  4277 tablet_bootstrap.cc:654] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.882771  4374 consensus_queue.cc:1048] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:20:43.894457  4280 tablet_bootstrap.cc:654] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.895274  4365 leader_election.cc:290] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:43.898228  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:43.899083  4152 raft_consensus.cc:2468] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:43.904291  4277 tablet_bootstrap.cc:492] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:43.904937  4277 ts_tablet_manager.cc:1403] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.008s
I20260709 12:20:43.905702  4193 leader_election.cc:304] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:43.908313  4277 raft_consensus.cc:359] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.909157  4277 raft_consensus.cc:385] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.909488  4277 raft_consensus.cc:740] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.908885  3971 leader_election.cc:304] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:43.910282  4277 consensus_queue.cc:260] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.911573  4152 raft_consensus.cc:1275] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:20:43.911981  4152 pending_rounds.cc:85] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93: Aborting all ops after (but not including) 0
I20260709 12:20:43.912248  4152 pending_rounds.cc:107] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:20:43.912576  4152 raft_consensus.cc:2889] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:43.913131  4277 ts_tablet_manager.cc:1434] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:20:43.914078  4348 raft_consensus.cc:2749] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.914291  4277 tablet_bootstrap.cc:492] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:43.914858  4348 consensus_queue.cc:1048] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:43.915925  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
W20260709 12:20:43.917579  4194 leader_election.cc:343] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.917572  4348 consensus_queue.cc:1048] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:43.915267  4365 raft_consensus.cc:2804] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.920560  4365 raft_consensus.cc:493] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.920933  4365 raft_consensus.cc:3060] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.921341  4277 tablet_bootstrap.cc:654] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.928133  4335 consensus_queue.cc:1048] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:43.932741  4280 tablet_bootstrap.cc:492] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:43.933424  4280 ts_tablet_manager.cc:1403] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.005s
I20260709 12:20:43.935027  4277 tablet_bootstrap.cc:492] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:43.935616  4277 ts_tablet_manager.cc:1403] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:20:43.936442  4325 consensus_queue.cc:1048] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:43.936647  4280 raft_consensus.cc:359] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.937335  4280 raft_consensus.cc:385] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.937647  4280 raft_consensus.cc:740] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.938472  4280 consensus_queue.cc:260] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.938627  4277 raft_consensus.cc:359] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.939380  4277 raft_consensus.cc:385] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.939705  4277 raft_consensus.cc:740] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.940526  4277 consensus_queue.cc:260] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:43.942924  4227 raft_consensus.cc:2468] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:43.943289  4272 tablet_bootstrap.cc:492] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:43.944022  4371 leader_election.cc:290] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:43.962957  4272 ts_tablet_manager.cc:1403] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.207s	user 0.022s	sys 0.035s
I20260709 12:20:43.946818  4277 ts_tablet_manager.cc:1434] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:20:43.947140  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75948129b8264d75972f4660893bd024" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:43.947623  4385 leader_election.cc:290] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:43.949455  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75948129b8264d75972f4660893bd024" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:43.951609  4280 ts_tablet_manager.cc:1434] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:20:43.951880  4335 consensus_queue.cc:1048] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:43.968549  4080 raft_consensus.cc:3060] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.969318  4280 tablet_bootstrap.cc:492] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:43.969609  4272 raft_consensus.cc:359] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.971652  4272 raft_consensus.cc:385] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.971935  4272 raft_consensus.cc:740] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.972709  4272 consensus_queue.cc:260] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:43.975287  4226 raft_consensus.cc:1275] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:20:43.976826  4280 tablet_bootstrap.cc:654] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.964624  4277 tablet_bootstrap.cc:492] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:43.965885  4153 raft_consensus.cc:3060] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.967466  4079 raft_consensus.cc:3060] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.981670  4272 ts_tablet_manager.cc:1434] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:20:43.989935  4272 tablet_bootstrap.cc:492] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:43.992357  4046 leader_election.cc:304] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:43.995890  4371 raft_consensus.cc:2804] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.003810  4371 raft_consensus.cc:697] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:43.996029  4365 raft_consensus.cc:515] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.005234  4371 consensus_queue.cc:237] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.007198  4365 leader_election.cc:290] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:44.010327  4273 tablet_bootstrap.cc:654] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.011165  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:44.015413  4277 tablet_bootstrap.cc:654] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:44.023432  4194 leader_election.cc:343] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.024657  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:44.025360  4152 raft_consensus.cc:3060] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.963094  4306 raft_consensus.cc:515] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.017313  4385 consensus_queue.cc:1048] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:20:44.043203  4347 raft_consensus.cc:493] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.043292  4080 raft_consensus.cc:2468] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:44.043924  4347 raft_consensus.cc:515] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.046185  4347 leader_election.cc:290] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:44.048029  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.058957  4348 consensus_queue.cc:1048] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:44.060925  4005 raft_consensus.cc:3060] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.064069  4280 tablet_bootstrap.cc:492] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:44.064795  4280 ts_tablet_manager.cc:1403] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.096s	user 0.017s	sys 0.014s
I20260709 12:20:44.065339  4272 tablet_bootstrap.cc:654] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.068758  4153 raft_consensus.cc:2468] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:44.068778  4306 leader_election.cc:290] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:44.075315  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69fa505a55eb4823a412608c712fb378" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:44.076067  4077 raft_consensus.cc:3060] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.077005  4348 raft_consensus.cc:493] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.077594  4348 raft_consensus.cc:515] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.080663  3970 leader_election.cc:304] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:44.081051  4348 leader_election.cc:290] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:44.081801  4348 raft_consensus.cc:2804] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.071556  4152 raft_consensus.cc:2468] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:44.082296  4348 raft_consensus.cc:697] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:44.086481  4348 consensus_queue.cc:237] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.082389  4327 raft_consensus.cc:493] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.093273  4327 raft_consensus.cc:515] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.096282  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:44.099803  4280 raft_consensus.cc:359] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.100780  4280 raft_consensus.cc:385] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.101135  4280 raft_consensus.cc:740] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.102085  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69fa505a55eb4823a412608c712fb378" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:44.102797  4227 raft_consensus.cc:3060] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.101986  4280 consensus_queue.cc:260] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.125551  4193 leader_election.cc:304] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa
I20260709 12:20:44.126868  4365 raft_consensus.cc:2804] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.127421  4365 raft_consensus.cc:697] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:44.128341  4365 consensus_queue.cc:237] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.131494  4077 raft_consensus.cc:2468] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:44.132918  4120 leader_election.cc:304] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:44.133651  4306 raft_consensus.cc:2804] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.134228  4306 raft_consensus.cc:697] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:44.135145  4306 consensus_queue.cc:237] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.135721  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd58e37cb1847738fdf9dd220c2280a" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:44.136248  3971 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 45769.9us
W20260709 12:20:44.137388  4045 leader_election.cc:343] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.138932  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.140313  4280 ts_tablet_manager.cc:1434] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.075s	user 0.005s	sys 0.004s
I20260709 12:20:44.141758  4280 tablet_bootstrap.cc:492] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:44.135035  4327 leader_election.cc:290] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
W20260709 12:20:44.139312  3971 leader_election.cc:343] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.159387  4273 tablet_bootstrap.cc:492] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:44.160123  4273 ts_tablet_manager.cc:1403] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.304s	user 0.014s	sys 0.002s
I20260709 12:20:44.162554  4295 raft_consensus.cc:493] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.163282  4295 raft_consensus.cc:515] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.164647  4227 raft_consensus.cc:2468] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
W20260709 12:20:44.166368  3971 leader_election.cc:343] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.166944  3971 leader_election.cc:304] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:44.167788  4347 raft_consensus.cc:2749] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:44.172659  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd58e37cb1847738fdf9dd220c2280a" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:44.153224  4335 raft_consensus.cc:493] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.174222  4335 raft_consensus.cc:515] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.173695  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3331199805b64fa8bf0d5e8847ad17b8" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
W20260709 12:20:44.175341  3971 leader_election.cc:343] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.176090  4280 tablet_bootstrap.cc:654] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.176535  4335 leader_election.cc:290] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:44.179298  4277 tablet_bootstrap.cc:492] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:44.179914  4277 ts_tablet_manager.cc:1403] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.216s	user 0.013s	sys 0.029s
I20260709 12:20:44.180145  4347 consensus_queue.cc:1048] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:20:44.180586  4046 leader_election.cc:343] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.181216  4046 leader_election.cc:304] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:44.182847  4371 raft_consensus.cc:2749] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:44.183331  4277 raft_consensus.cc:359] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.184089  4277 raft_consensus.cc:385] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.184370  4277 raft_consensus.cc:740] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.185804  4295 leader_election.cc:290] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:20:44.186604  4120 leader_election.cc:343] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.188083  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3331199805b64fa8bf0d5e8847ad17b8" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:44.189842  3903 catalog_manager.cc:5697] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:44.191819  4273 raft_consensus.cc:359] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.192730  4273 raft_consensus.cc:385] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.193069  4273 raft_consensus.cc:740] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.195310  4277 consensus_queue.cc:260] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.198493  4273 consensus_queue.cc:260] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.202159  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.203665  4273 ts_tablet_manager.cc:1434] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 12:20:44.195441  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:44.211225  4273 tablet_bootstrap.cc:492] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:44.214038  4277 ts_tablet_manager.cc:1434] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 12:20:44.215538  4277 tablet_bootstrap.cc:492] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:44.203683  4079 raft_consensus.cc:2468] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
W20260709 12:20:44.224575  4120 leader_election.cc:343] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.247876  4272 tablet_bootstrap.cc:492] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:44.241009  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:44.248804  4272 ts_tablet_manager.cc:1403] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.259s	user 0.027s	sys 0.015s
W20260709 12:20:44.246630  3971 leader_election.cc:343] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.258970  4280 tablet_bootstrap.cc:492] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:44.260514  4280 ts_tablet_manager.cc:1403] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.119s	user 0.017s	sys 0.012s
I20260709 12:20:44.251925  4003 raft_consensus.cc:2468] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:44.260839  3971 leader_election.cc:304] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:44.262720  4348 raft_consensus.cc:2749] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:44.266628  4003 raft_consensus.cc:3060] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.270658  4347 raft_consensus.cc:493] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.271425  4347 raft_consensus.cc:515] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.273814  4120 leader_election.cc:304] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:44.275190  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:44.275938  4120 leader_election.cc:304] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:44.277051  4417 raft_consensus.cc:2804] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.277482  4417 raft_consensus.cc:493] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.277653  4295 raft_consensus.cc:2804] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.278267  4295 raft_consensus.cc:493] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.278644  4295 raft_consensus.cc:3060] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:44.279006  3970 leader_election.cc:343] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.280069  4347 leader_election.cc:290] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
W20260709 12:20:44.266639  4427 log.cc:927] Time spent T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59: Append to log took a long time: real 0.067s	user 0.003s	sys 0.000s
I20260709 12:20:44.277916  4417 raft_consensus.cc:3060] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.289285  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.291381  4277 tablet_bootstrap.cc:654] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.296773  4325 raft_consensus.cc:493] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.297992  4273 tablet_bootstrap.cc:654] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.297395  4325 raft_consensus.cc:515] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.303995  4306 raft_consensus.cc:493] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.305303  4325 leader_election.cc:290] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:44.310196  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26349fc0d5664798a248220d968f5269" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:44.313109  3900 catalog_manager.cc:5697] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:44.310357  4306 raft_consensus.cc:515] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:44.319519  3971 leader_election.cc:343] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.306635  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26349fc0d5664798a248220d968f5269" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:44.315435  4272 raft_consensus.cc:359] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.320889  4272 raft_consensus.cc:385] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.321225  4272 raft_consensus.cc:740] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
W20260709 12:20:44.321557  4194 leader_election.cc:343] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.322175  4272 consensus_queue.cc:260] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:44.323832  4193 leader_election.cc:343] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.324427  4193 leader_election.cc:304] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:44.324853  4280 raft_consensus.cc:359] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.325608  4280 raft_consensus.cc:385] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.325954  4280 raft_consensus.cc:740] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.324102  4306 leader_election.cc:290] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:44.326745  4280 consensus_queue.cc:260] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.329069  4280 ts_tablet_manager.cc:1434] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.068s	user 0.004s	sys 0.000s
I20260709 12:20:44.331305  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.331523  4295 raft_consensus.cc:515] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.336408  4280 tablet_bootstrap.cc:492] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:44.346545  4272 ts_tablet_manager.cc:1434] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.097s	user 0.003s	sys 0.005s
I20260709 12:20:44.348048  4272 tablet_bootstrap.cc:492] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
W20260709 12:20:44.349474  4120 leader_election.cc:343] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.338490  4325 raft_consensus.cc:2749] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:44.352682  4417 raft_consensus.cc:515] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.355436  4417 leader_election.cc:290] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:44.354887  3899 catalog_manager.cc:5697] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.333814  3971 leader_election.cc:304] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:44.358739  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3331199805b64fa8bf0d5e8847ad17b8" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:44.359431  4079 raft_consensus.cc:3060] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.360142  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:44.360713  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:44.360896  4005 raft_consensus.cc:3060] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.361420  4003 raft_consensus.cc:2468] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:44.356657  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:44.364159  4272 tablet_bootstrap.cc:654] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.363976  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3331199805b64fa8bf0d5e8847ad17b8" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:44.360253  4356 raft_consensus.cc:2749] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:44.365865  4295 leader_election.cc:290] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:20:44.367916  4120 leader_election.cc:343] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.368127  4280 tablet_bootstrap.cc:654] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.371722  4120 leader_election.cc:304] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:44.372807  4295 raft_consensus.cc:2804] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.373252  4295 raft_consensus.cc:493] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.373641  4295 raft_consensus.cc:3060] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.377488  4348 raft_consensus.cc:493] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.378141  4348 raft_consensus.cc:515] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:44.373409  4120 leader_election.cc:343] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.380501  4348 leader_election.cc:290] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:44.381397  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26349fc0d5664798a248220d968f5269" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:44.388094  4295 raft_consensus.cc:515] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.389442  4417 raft_consensus.cc:493] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.390064  4417 raft_consensus.cc:515] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.392717  4417 leader_election.cc:290] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
W20260709 12:20:44.395709  3970 leader_election.cc:343] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.384991  4005 raft_consensus.cc:2468] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:44.385473  4273 tablet_bootstrap.cc:492] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:44.398957  4273 ts_tablet_manager.cc:1403] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.194s	user 0.011s	sys 0.018s
I20260709 12:20:44.400786  4079 raft_consensus.cc:2468] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:44.397701  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e92ca9c3274410aa6ea1aaa11671569" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:44.399504  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26349fc0d5664798a248220d968f5269" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.403086  4225 raft_consensus.cc:2468] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:44.403676  4120 leader_election.cc:304] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:44.405707  4295 leader_election.cc:290] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:44.405876  4417 raft_consensus.cc:2804] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.404506  3971 leader_election.cc:304] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:44.405794  4365 raft_consensus.cc:493] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.410120  4287 raft_consensus.cc:2804] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.410631  4287 raft_consensus.cc:493] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.410997  4287 raft_consensus.cc:3060] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.408286  4417 raft_consensus.cc:697] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:44.413867  4417 consensus_queue.cc:237] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.416105  4365 raft_consensus.cc:515] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.406308  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e92ca9c3274410aa6ea1aaa11671569" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.418618  4225 raft_consensus.cc:2393] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6994df41341e478d89f7c105f550ff93 in current term 1: Already voted for candidate 25902f00fb5d41679245f23dd19654aa in this term.
I20260709 12:20:44.419989  4365 leader_election.cc:290] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:44.420737  4277 tablet_bootstrap.cc:492] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:44.421314  4277 ts_tablet_manager.cc:1403] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.206s	user 0.015s	sys 0.012s
I20260709 12:20:44.421236  4272 tablet_bootstrap.cc:492] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:44.422200  4272 ts_tablet_manager.cc:1403] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.019s
I20260709 12:20:44.425051  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ae7ddfcc1e49d4b6847e9c895edde7" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:44.424846  4277 raft_consensus.cc:359] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.426173  4277 raft_consensus.cc:385] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.426483  4277 raft_consensus.cc:740] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.413581  3902 catalog_manager.cc:5697] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:44.426529  4120 leader_election.cc:304] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:44.427238  4277 consensus_queue.cc:260] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.428305  4306 raft_consensus.cc:2804] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.428797  4306 raft_consensus.cc:697] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:44.429514  4277 ts_tablet_manager.cc:1434] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:20:44.425076  4272 raft_consensus.cc:359] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.430832  4272 raft_consensus.cc:385] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.431051  4272 raft_consensus.cc:740] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.432955  4277 tablet_bootstrap.cc:492] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:44.432829  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:44.433897  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44ae7ddfcc1e49d4b6847e9c895edde7" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:44.435592  4194 leader_election.cc:304] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:44.435928  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:44.436617  4004 raft_consensus.cc:3060] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.429693  4306 consensus_queue.cc:237] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.442132  4371 consensus_queue.cc:1048] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:44.443657  4273 raft_consensus.cc:359] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.444430  4273 raft_consensus.cc:385] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.444697  4273 raft_consensus.cc:740] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
W20260709 12:20:44.447002  4120 leader_election.cc:343] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.447407  4272 consensus_queue.cc:260] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.450248  4272 ts_tablet_manager.cc:1434] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.002s
I20260709 12:20:44.451350  4272 tablet_bootstrap.cc:492] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:44.445436  4273 consensus_queue.cc:260] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.460942  4365 raft_consensus.cc:2749] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:44.474937  4004 raft_consensus.cc:2468] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:44.478858  4273 ts_tablet_manager.cc:1434] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.000s
I20260709 12:20:44.480217  4273 tablet_bootstrap.cc:492] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:44.488262  4228 raft_consensus.cc:3060] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.488826  4280 tablet_bootstrap.cc:492] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:44.477783  4120 leader_election.cc:304] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:44.489404  4280 ts_tablet_manager.cc:1403] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.153s	user 0.018s	sys 0.001s
I20260709 12:20:44.494047  4277 tablet_bootstrap.cc:654] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.495795  4287 raft_consensus.cc:515] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.503060  4287 leader_election.cc:290] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:44.503942  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26349fc0d5664798a248220d968f5269" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:44.504707  4152 raft_consensus.cc:3060] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.508519  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26349fc0d5664798a248220d968f5269" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:44.509361  4227 raft_consensus.cc:3060] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.509588  4272 tablet_bootstrap.cc:654] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:44.518150  4194 consensus_peers.cc:597] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:44.531261  4152 raft_consensus.cc:2468] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:44.537371  4280 raft_consensus.cc:359] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.541853  4280 raft_consensus.cc:385] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.542287  4280 raft_consensus.cc:740] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.537122  4295 raft_consensus.cc:3060] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.566980  4280 consensus_queue.cc:260] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.548210  3902 catalog_manager.cc:5697] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.569612  4280 ts_tablet_manager.cc:1434] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.000s
I20260709 12:20:44.569056  3902 catalog_manager.cc:5697] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:44.571473  4280 tablet_bootstrap.cc:492] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:44.574126  4347 consensus_queue.cc:1048] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:44.583781  3970 leader_election.cc:304] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
W20260709 12:20:44.549204  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 90182.9us
I20260709 12:20:44.587616  4348 raft_consensus.cc:2804] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.589833  4348 raft_consensus.cc:697] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:44.590962  4348 consensus_queue.cc:237] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.592727  4120 leader_election.cc:304] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:44.595973  4445 raft_consensus.cc:2804] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.596560  4445 raft_consensus.cc:697] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:44.598697  4272 tablet_bootstrap.cc:492] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:44.602029  4277 tablet_bootstrap.cc:492] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:44.608573  4272 ts_tablet_manager.cc:1403] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.157s	user 0.024s	sys 0.023s
I20260709 12:20:44.608776  4277 ts_tablet_manager.cc:1403] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.178s	user 0.028s	sys 0.049s
I20260709 12:20:44.597451  4445 consensus_queue.cc:237] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.627782  4280 tablet_bootstrap.cc:654] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.628450  4273 tablet_bootstrap.cc:654] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.606227  4295 raft_consensus.cc:2749] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:44.587808  4347 consensus_queue.cc:1048] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:20:44.639611  4272 raft_consensus.cc:359] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.640280  4272 raft_consensus.cc:385] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.640462  4272 raft_consensus.cc:740] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.641052  4272 consensus_queue.cc:260] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.644335  4277 raft_consensus.cc:359] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.645187  4277 raft_consensus.cc:385] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.645566  4277 raft_consensus.cc:740] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.648527  4277 consensus_queue.cc:260] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.650144  4272 ts_tablet_manager.cc:1434] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 12:20:44.637256  4227 raft_consensus.cc:2468] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:44.651382  4272 tablet_bootstrap.cc:492] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:44.653532  4277 ts_tablet_manager.cc:1434] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:20:44.654122  4335 consensus_queue.cc:1048] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:44.661681  4272 tablet_bootstrap.cc:654] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.654644  4277 tablet_bootstrap.cc:492] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:44.671782  4277 tablet_bootstrap.cc:654] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.687314  4277 tablet_bootstrap.cc:492] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:44.687906  4277 ts_tablet_manager.cc:1403] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.005s
I20260709 12:20:44.691593  4277 raft_consensus.cc:359] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.692312  4277 raft_consensus.cc:385] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.692606  4277 raft_consensus.cc:740] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.696940  4272 tablet_bootstrap.cc:492] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:44.697566  4272 ts_tablet_manager.cc:1403] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.008s
I20260709 12:20:44.695022  4327 consensus_queue.cc:1048] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:20:44.700770  4272 raft_consensus.cc:359] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.701563  4272 raft_consensus.cc:385] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.701907  4272 raft_consensus.cc:740] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.702744  4272 consensus_queue.cc:260] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.704087  4273 tablet_bootstrap.cc:492] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:44.704730  4273 ts_tablet_manager.cc:1403] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.225s	user 0.009s	sys 0.006s
I20260709 12:20:44.705063  4272 ts_tablet_manager.cc:1434] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
W20260709 12:20:44.706012  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:44.706167  4272 tablet_bootstrap.cc:492] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:44.695050  4277 consensus_queue.cc:260] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.718072  4277 ts_tablet_manager.cc:1434] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.004s
I20260709 12:20:44.719343  4277 tablet_bootstrap.cc:492] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:44.722965  4374 consensus_queue.cc:1048] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:44.724591  3902 catalog_manager.cc:5697] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:44.726187  4277 tablet_bootstrap.cc:654] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.729422  4335 raft_consensus.cc:493] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.730152  4335 raft_consensus.cc:515] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.732757  4335 leader_election.cc:290] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:44.734966  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.738088  4277 tablet_bootstrap.cc:492] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:44.738740  4277 ts_tablet_manager.cc:1403] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.003s
I20260709 12:20:44.741956  4277 raft_consensus.cc:359] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.742807  4277 raft_consensus.cc:385] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.743161  4277 raft_consensus.cc:740] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.729453  4272 tablet_bootstrap.cc:654] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.743897  4277 consensus_queue.cc:260] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.746951  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:44.747423  4277 ts_tablet_manager.cc:1434] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:20:44.749279  4277 tablet_bootstrap.cc:492] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:44.753618  4004 raft_consensus.cc:2468] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:44.756145  4277 tablet_bootstrap.cc:654] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.768841  4277 tablet_bootstrap.cc:492] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:44.769428  4277 ts_tablet_manager.cc:1403] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:20:44.700847  3899 catalog_manager.cc:5697] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:44.707810  4273 raft_consensus.cc:359] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.770253  4273 raft_consensus.cc:385] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.770625  4273 raft_consensus.cc:740] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.771422  4273 consensus_queue.cc:260] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.772397  4120 leader_election.cc:304] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
W20260709 12:20:44.774880  4120 leader_election.cc:343] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.775449  4277 raft_consensus.cc:359] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.776254  4277 raft_consensus.cc:385] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.776360  4335 raft_consensus.cc:2804] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.777115  4335 raft_consensus.cc:493] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.777582  4335 raft_consensus.cc:3060] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.773835  4280 tablet_bootstrap.cc:492] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:44.776584  4277 raft_consensus.cc:740] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.782076  4280 ts_tablet_manager.cc:1403] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.211s	user 0.014s	sys 0.002s
I20260709 12:20:44.782527  4277 consensus_queue.cc:260] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.782405  4273 ts_tablet_manager.cc:1434] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.001s
I20260709 12:20:44.785447  4272 tablet_bootstrap.cc:492] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:44.790328  4272 ts_tablet_manager.cc:1403] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.002s
I20260709 12:20:44.786847  4273 tablet_bootstrap.cc:492] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:44.787300  4277 ts_tablet_manager.cc:1434] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:20:44.791498  4335 raft_consensus.cc:515] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.792855  4277 tablet_bootstrap.cc:492] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:44.794303  4335 leader_election.cc:290] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:44.795306  4327 raft_consensus.cc:493] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.795908  4327 raft_consensus.cc:515] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.797149  4445 raft_consensus.cc:493] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.797729  4445 raft_consensus.cc:515] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.800108  4445 leader_election.cc:290] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:44.800297  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b7f39842e94931a878f3003c1353e1" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.800737  4445 consensus_queue.cc:1048] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:20:44.802206  4046 leader_election.cc:343] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.802938  4327 leader_election.cc:290] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:44.789557  4280 raft_consensus.cc:359] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.805981  4280 raft_consensus.cc:385] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.806281  4280 raft_consensus.cc:740] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.807093  4280 consensus_queue.cc:260] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.809396  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:44.810353  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:44.811070  4004 raft_consensus.cc:3060] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.814538  4273 tablet_bootstrap.cc:654] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.818174  4004 raft_consensus.cc:2468] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:44.819540  4227 raft_consensus.cc:3060] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.821318  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd58e37cb1847738fdf9dd220c2280a" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:44.823211  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd58e37cb1847738fdf9dd220c2280a" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:44.827395  4077 raft_consensus.cc:2468] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:44.829439  4272 raft_consensus.cc:359] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.832551  4272 raft_consensus.cc:385] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.834157  4272 raft_consensus.cc:740] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.835119  4272 consensus_queue.cc:260] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.837468  4272 ts_tablet_manager.cc:1434] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.000s
I20260709 12:20:44.833503  4120 leader_election.cc:304] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:44.832098  4280 ts_tablet_manager.cc:1434] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
I20260709 12:20:44.826467  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b7f39842e94931a878f3003c1353e1" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:44.842844  4154 raft_consensus.cc:2468] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:44.844481  4280 tablet_bootstrap.cc:492] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:44.852254  4280 tablet_bootstrap.cc:654] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.853991  4445 raft_consensus.cc:2804] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.854882  4445 raft_consensus.cc:697] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:44.855824  4445 consensus_queue.cc:237] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:44.859709  4273 tablet_bootstrap.cc:492] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
W20260709 12:20:44.860221  4120 leader_election.cc:343] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.801510  4277 tablet_bootstrap.cc:654] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.860350  4273 ts_tablet_manager.cc:1403] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.005s
I20260709 12:20:44.839562  4272 tablet_bootstrap.cc:492] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:44.867724  4277 tablet_bootstrap.cc:492] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:44.868320  4277 ts_tablet_manager.cc:1403] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.000s
I20260709 12:20:44.871613  4277 raft_consensus.cc:359] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.872404  4277 raft_consensus.cc:385] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.872706  4277 raft_consensus.cc:740] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.874755  4277 consensus_queue.cc:260] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.877087  4277 ts_tablet_manager.cc:1434] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:20:44.878265  4277 tablet_bootstrap.cc:492] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
W20260709 12:20:44.879202  4120 leader_election.cc:343] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.881881  4120 leader_election.cc:304] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:44.885602  4273 raft_consensus.cc:359] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.886500  4273 raft_consensus.cc:385] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.886796  4273 raft_consensus.cc:740] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.887580  4273 consensus_queue.cc:260] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.896509  4335 raft_consensus.cc:2804] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.897634  4335 raft_consensus.cc:493] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.898036  4335 raft_consensus.cc:3060] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.899688  4277 tablet_bootstrap.cc:654] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.908427  4273 ts_tablet_manager.cc:1434] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 12:20:44.910985  4004 raft_consensus.cc:3060] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.912111  4335 raft_consensus.cc:515] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.910948  4045 leader_election.cc:304] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:44.916567  4273 tablet_bootstrap.cc:492] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:44.918069  4374 raft_consensus.cc:493] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.913335  3901 catalog_manager.cc:5697] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:44.919118  4374 raft_consensus.cc:515] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.920411  4327 raft_consensus.cc:2804] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.920845  4327 raft_consensus.cc:493] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.921200  4327 raft_consensus.cc:3060] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.921785  4374 leader_election.cc:290] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:44.914601  4272 tablet_bootstrap.cc:654] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.928265  4335 leader_election.cc:290] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:44.932519  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:44.932746  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd58e37cb1847738fdf9dd220c2280a" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:44.933318  4005 raft_consensus.cc:2468] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:44.934202  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:44.934911  4079 raft_consensus.cc:2468] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:44.935209  4194 leader_election.cc:304] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:44.942977  4374 raft_consensus.cc:2804] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.945101  4273 tablet_bootstrap.cc:654] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.947315  4277 tablet_bootstrap.cc:492] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:44.943408  4327 raft_consensus.cc:515] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.947913  4277 ts_tablet_manager.cc:1403] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.070s	user 0.011s	sys 0.014s
I20260709 12:20:44.950606  4327 leader_election.cc:290] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:44.951246  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b7f39842e94931a878f3003c1353e1" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:44.951287  4277 raft_consensus.cc:359] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.952301  4277 raft_consensus.cc:385] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.952598  4277 raft_consensus.cc:740] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
W20260709 12:20:44.952994  4046 leader_election.cc:343] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.943542  4374 raft_consensus.cc:493] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.954206  4374 raft_consensus.cc:3060] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.954699  4277 consensus_queue.cc:260] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:44.957527  4277 ts_tablet_manager.cc:1434] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 12:20:44.959419  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd58e37cb1847738fdf9dd220c2280a" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:44.960157  4079 raft_consensus.cc:3060] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.964888  4277 tablet_bootstrap.cc:492] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:44.967959  4445 raft_consensus.cc:493] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.968638  4445 raft_consensus.cc:515] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:44.973430  4445 leader_election.cc:290] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:44.974575  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b7f39842e94931a878f3003c1353e1" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:44.975306  4154 raft_consensus.cc:3060] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.976537  4277 tablet_bootstrap.cc:654] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:44.978077  4120 leader_election.cc:343] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.979086  4445 consensus_queue.cc:1048] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:20:44.980247  4120 consensus_peers.cc:597] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 -> Peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Couldn't send request to peer 72940e4e52944adb81608e286ad58a59. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:44.985971  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:44.986645  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:44.988776  4374 raft_consensus.cc:515] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.991178  4374 leader_election.cc:290] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:20:44.993341  4120 leader_election.cc:343] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.993074  4445 consensus_queue.cc:1048] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:44.994412  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:44.997442  4003 raft_consensus.cc:3060] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.010219  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:45.010574  4154 raft_consensus.cc:2468] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:45.011144  4080 raft_consensus.cc:3060] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.998358  4327 raft_consensus.cc:493] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.015105  4327 raft_consensus.cc:515] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:44.992887  4374 raft_consensus.cc:493] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.020560  4327 leader_election.cc:290] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.023252  4003 raft_consensus.cc:2468] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:45.023669  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:45.024833  4374 raft_consensus.cc:515] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.025961  4079 raft_consensus.cc:2468] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:45.031076  4273 tablet_bootstrap.cc:492] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:45.031826  4273 ts_tablet_manager.cc:1403] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.116s	user 0.007s	sys 0.020s
I20260709 12:20:45.034591  4280 tablet_bootstrap.cc:492] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:45.035326  4280 ts_tablet_manager.cc:1403] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.191s	user 0.020s	sys 0.007s
I20260709 12:20:45.035104  4045 leader_election.cc:304] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
W20260709 12:20:45.038318  4120 leader_election.cc:343] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.038802  4327 raft_consensus.cc:2804] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.038887  4120 leader_election.cc:304] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:45.039538  4327 raft_consensus.cc:697] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:45.040441  4327 consensus_queue.cc:237] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:45.041335  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.042178  4004 raft_consensus.cc:2468] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:45.042745  4445 raft_consensus.cc:2749] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.045055  4272 tablet_bootstrap.cc:492] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:45.045624  4272 ts_tablet_manager.cc:1403] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.206s	user 0.017s	sys 0.026s
I20260709 12:20:45.046396  4227 raft_consensus.cc:2468] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:45.054978  4365 raft_consensus.cc:493] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.055658  4365 raft_consensus.cc:515] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.057330  4046 leader_election.cc:304] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:45.058799  4371 raft_consensus.cc:2804] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:45.059222  4371 raft_consensus.cc:493] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.059607  4371 raft_consensus.cc:3060] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.061914  4272 raft_consensus.cc:359] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.062830  4272 raft_consensus.cc:385] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.062958  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:45.066561  4365 leader_election.cc:290] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.067962  4374 leader_election.cc:290] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:45.069273  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.070627  4194 leader_election.cc:304] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:45.073143  4080 raft_consensus.cc:2468] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:45.073832  4272 raft_consensus.cc:740] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.054206  4273 raft_consensus.cc:359] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.075517  4280 raft_consensus.cc:359] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.078964  4273 raft_consensus.cc:385] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.079445  4273 raft_consensus.cc:740] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.079334  4280 raft_consensus.cc:385] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.080302  4280 raft_consensus.cc:740] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.080412  4273 consensus_queue.cc:260] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.080281  4272 consensus_queue.cc:260] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.084975  4005 raft_consensus.cc:2393] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 6994df41341e478d89f7c105f550ff93 in this term.
I20260709 12:20:45.082508  4280 consensus_queue.cc:260] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.089478  4272 ts_tablet_manager.cc:1434] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.003s
I20260709 12:20:45.090741  4272 tablet_bootstrap.cc:492] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:45.099239  4280 ts_tablet_manager.cc:1434] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.000s
I20260709 12:20:45.100248  4194 leader_election.cc:304] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:45.100541  4280 tablet_bootstrap.cc:492] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:45.101704  4325 raft_consensus.cc:3060] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.105077  4277 tablet_bootstrap.cc:492] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:45.105919  4277 ts_tablet_manager.cc:1403] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.141s	user 0.021s	sys 0.024s
I20260709 12:20:45.082067  4374 raft_consensus.cc:2804] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.088593  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.124970  4348 consensus_queue.cc:1048] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:45.126466  4325 raft_consensus.cc:2749] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.127588  4280 tablet_bootstrap.cc:654] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.130661  4347 consensus_queue.cc:1048] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:45.135027  4003 raft_consensus.cc:2468] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:45.136588  4194 leader_election.cc:304] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:45.137504  4365 raft_consensus.cc:2804] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:45.138020  4365 raft_consensus.cc:493] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.138434  4365 raft_consensus.cc:3060] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.107774  4120 leader_election.cc:304] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:45.155184  4273 ts_tablet_manager.cc:1434] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.123s	user 0.017s	sys 0.007s
I20260709 12:20:45.155601  4445 raft_consensus.cc:2804] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.163498  4445 raft_consensus.cc:697] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:45.164599  4445 consensus_queue.cc:237] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.166545  3902 catalog_manager.cc:5697] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.162580  4348 raft_consensus.cc:493] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.168421  4348 raft_consensus.cc:515] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.170682  4348 leader_election.cc:290] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:45.172220  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:45.174214  3971 leader_election.cc:343] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.158926  4277 raft_consensus.cc:359] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:45.155781  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 66780.9us
I20260709 12:20:45.162837  4374 raft_consensus.cc:697] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:45.199730  4374 consensus_queue.cc:237] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.179327  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:45.201493  4277 raft_consensus.cc:385] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.203990  4277 raft_consensus.cc:740] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.196323  4273 tablet_bootstrap.cc:492] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:45.181819  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:45.187286  4417 raft_consensus.cc:493] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.209161  4417 raft_consensus.cc:515] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.214956  4325 raft_consensus.cc:493] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.222764  4325 raft_consensus.cc:515] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.207482  4371 raft_consensus.cc:515] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.225024  4078 raft_consensus.cc:2393] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 25902f00fb5d41679245f23dd19654aa in this term.
W20260709 12:20:45.212980  3970 leader_election.cc:343] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.210417  4277 consensus_queue.cc:260] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.231743  4277 ts_tablet_manager.cc:1434] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.125s	user 0.008s	sys 0.000s
I20260709 12:20:45.225972  3970 leader_election.cc:304] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:45.224356  4417 leader_election.cc:290] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:45.233130  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:45.211622  4080 raft_consensus.cc:3060] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.241338  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e4899b95af40fd9836e281f4c23e17" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:45.245903  4277 tablet_bootstrap.cc:492] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:45.244726  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e4899b95af40fd9836e281f4c23e17" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.245121  4348 raft_consensus.cc:2749] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.239526  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:45.252499  4280 tablet_bootstrap.cc:492] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:45.252648  4417 raft_consensus.cc:493] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.253218  4280 ts_tablet_manager.cc:1403] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.153s	user 0.027s	sys 0.027s
I20260709 12:20:45.253715  4325 leader_election.cc:290] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.254546  4371 leader_election.cc:290] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.255553  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.256153  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:45.256892  4005 raft_consensus.cc:3060] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:45.259409  4120 leader_election.cc:343] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.259804  4306 consensus_queue.cc:1048] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:45.262773  4273 tablet_bootstrap.cc:654] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.263053  4365 raft_consensus.cc:515] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.256419  4003 raft_consensus.cc:2393] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 6994df41341e478d89f7c105f550ff93 in this term.
I20260709 12:20:45.253254  4417 raft_consensus.cc:515] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.275099  4417 leader_election.cc:290] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:45.270848  4227 raft_consensus.cc:2393] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 25902f00fb5d41679245f23dd19654aa in current term 1: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:45.286700  4194 leader_election.cc:304] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:45.288344  4280 raft_consensus.cc:359] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.289182  4280 raft_consensus.cc:385] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.289462  4280 raft_consensus.cc:740] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.290231  4280 consensus_queue.cc:260] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.291635  4365 leader_election.cc:290] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:45.294445  4005 raft_consensus.cc:2468] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:45.295601  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:45.296017  4046 leader_election.cc:304] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:45.296890  4325 raft_consensus.cc:3060] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.297384  4476 raft_consensus.cc:2804] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.298267  4476 raft_consensus.cc:697] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:45.299284  4476 consensus_queue.cc:237] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:45.301642  4120 leader_election.cc:343] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.304770  4280 ts_tablet_manager.cc:1434] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.004s
I20260709 12:20:45.306551  4272 tablet_bootstrap.cc:654] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.308791  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:45.312960  4005 raft_consensus.cc:2393] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 25902f00fb5d41679245f23dd19654aa in this term.
I20260709 12:20:45.318715  4280 tablet_bootstrap.cc:492] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:45.321151  4194 leader_election.cc:304] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:45.321998  4273 tablet_bootstrap.cc:492] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:45.322602  4273 ts_tablet_manager.cc:1403] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.146s	user 0.029s	sys 0.017s
I20260709 12:20:45.323632  4482 raft_consensus.cc:2749] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.302479  4120 leader_election.cc:304] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:45.324904  4417 raft_consensus.cc:2749] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.325727  4273 raft_consensus.cc:359] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.326530  4273 raft_consensus.cc:385] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.326794  4273 raft_consensus.cc:740] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.327595  4273 consensus_queue.cc:260] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.330462  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:45.332211  4335 consensus_queue.cc:1048] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:45.334734  4306 consensus_queue.cc:1048] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:45.334724  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.337040  4273 ts_tablet_manager.cc:1434] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:20:45.338223  4273 tablet_bootstrap.cc:492] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:45.352304  4325 raft_consensus.cc:2749] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.357057  4280 tablet_bootstrap.cc:654] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.358814  4277 tablet_bootstrap.cc:654] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.361667  3901 catalog_manager.cc:5697] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.363763  3902 catalog_manager.cc:5697] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.372822  4273 tablet_bootstrap.cc:654] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.386757  4477 raft_consensus.cc:493] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 72940e4e52944adb81608e286ad58a59)
W20260709 12:20:45.374629  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 43520.4us
I20260709 12:20:45.387378  4477 raft_consensus.cc:515] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:45.399307  4272 tablet_bootstrap.cc:492] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:45.399963  4272 ts_tablet_manager.cc:1403] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.309s	user 0.007s	sys 0.015s
I20260709 12:20:45.402604  4477 leader_election.cc:290] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:45.403467  4272 raft_consensus.cc:359] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.404304  4272 raft_consensus.cc:385] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.404584  4272 raft_consensus.cc:740] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.405480  4272 consensus_queue.cc:260] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.405773  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:45.408032  4272 ts_tablet_manager.cc:1434] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:20:45.409214  4272 tablet_bootstrap.cc:492] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:45.410005  3900 catalog_manager.cc:5697] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.417150  4272 tablet_bootstrap.cc:654] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.429181  4273 tablet_bootstrap.cc:492] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:45.429929  4277 tablet_bootstrap.cc:492] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:45.430393  4273 ts_tablet_manager.cc:1403] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.092s	user 0.018s	sys 0.003s
I20260709 12:20:45.432085  4272 tablet_bootstrap.cc:492] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:45.432632  4272 ts_tablet_manager.cc:1403] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.004s
I20260709 12:20:45.434733  4273 raft_consensus.cc:359] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:45.435912  4273 raft_consensus.cc:385] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.435772  4272 raft_consensus.cc:359] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.436427  4273 raft_consensus.cc:740] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.436584  4272 raft_consensus.cc:385] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.437119  4272 raft_consensus.cc:740] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.437840  4273 consensus_queue.cc:260] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:45.438060  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:45.437979  4272 consensus_queue.cc:260] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.439811  4417 consensus_queue.cc:1048] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:45.441056  4273 ts_tablet_manager.cc:1434] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
W20260709 12:20:45.441628  4120 leader_election.cc:343] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.442878  4273 tablet_bootstrap.cc:492] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:45.443518  4272 ts_tablet_manager.cc:1434] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:20:45.444674  4272 tablet_bootstrap.cc:492] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:45.450946  4272 tablet_bootstrap.cc:654] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.464625  4273 tablet_bootstrap.cc:654] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.430560  4277 ts_tablet_manager.cc:1403] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.190s	user 0.010s	sys 0.006s
I20260709 12:20:45.475764  4277 raft_consensus.cc:359] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.476668  4277 raft_consensus.cc:385] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.476974  4277 raft_consensus.cc:740] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.478231  4273 tablet_bootstrap.cc:492] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:45.479104  4273 ts_tablet_manager.cc:1403] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.000s
I20260709 12:20:45.483003  4273 raft_consensus.cc:359] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.484021  4273 raft_consensus.cc:385] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.488739  4280 tablet_bootstrap.cc:492] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:45.489403  4280 ts_tablet_manager.cc:1403] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.171s	user 0.040s	sys 0.034s
W20260709 12:20:45.489305  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:41077, user_credentials={real_user=slave}} blocked reactor thread for 46896.5us
I20260709 12:20:45.493019  4280 raft_consensus.cc:359] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.493809  4280 raft_consensus.cc:385] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.494091  4280 raft_consensus.cc:740] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.494972  4280 consensus_queue.cc:260] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:45.496012  4120 leader_election.cc:343] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.496675  4120 leader_election.cc:304] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:45.501883  4280 ts_tablet_manager.cc:1434] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:20:45.503047  4280 tablet_bootstrap.cc:492] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:45.504388  4477 raft_consensus.cc:493] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.504968  4477 raft_consensus.cc:515] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.507201  4477 leader_election.cc:290] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.509756  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:45.510561  4227 raft_consensus.cc:2468] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:45.512676  4046 leader_election.cc:304] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:45.513813  4477 raft_consensus.cc:2804] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:45.514289  4477 raft_consensus.cc:493] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.514693  4477 raft_consensus.cc:3060] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.521091  4280 tablet_bootstrap.cc:654] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.530009  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.531028  4005 raft_consensus.cc:2468] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:45.531383  4417 raft_consensus.cc:2749] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.534972  4277 consensus_queue.cc:260] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:45.538200  4046 consensus_peers.cc:597] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa -> Peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Couldn't send request to peer 72940e4e52944adb81608e286ad58a59. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:45.539687  4045 leader_election.cc:304] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:45.543450  4476 raft_consensus.cc:2749] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:20:45.544066  4120 consensus_peers.cc:597] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 -> Peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Couldn't send request to peer 72940e4e52944adb81608e286ad58a59. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:45.537554  4277 ts_tablet_manager.cc:1434] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.000s
I20260709 12:20:45.546293  4277 tablet_bootstrap.cc:492] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:45.550716  4280 tablet_bootstrap.cc:492] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:45.551316  4280 ts_tablet_manager.cc:1403] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.049s	user 0.006s	sys 0.011s
I20260709 12:20:45.484390  4273 raft_consensus.cc:740] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.559970  4277 tablet_bootstrap.cc:654] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.579537  4485 raft_consensus.cc:493] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.580205  4485 raft_consensus.cc:515] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.580672  4280 raft_consensus.cc:359] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.581418  4280 raft_consensus.cc:385] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.582233  4273 consensus_queue.cc:260] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.585410  4485 leader_election.cc:290] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.583372  4280 raft_consensus.cc:740] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.593873  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e203dae959394f4e90051ad4746a9291" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.595568  4287 raft_consensus.cc:493] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.596272  4287 raft_consensus.cc:515] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.600445  4287 leader_election.cc:290] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:45.602177  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e4899b95af40fd9836e281f4c23e17" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:45.603080  4154 raft_consensus.cc:2468] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:45.605302  3970 leader_election.cc:304] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:45.606698  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e203dae959394f4e90051ad4746a9291" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:45.607219  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e4899b95af40fd9836e281f4c23e17" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:45.608107  4080 raft_consensus.cc:2468] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:45.611124  4348 raft_consensus.cc:2804] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:45.611680  4348 raft_consensus.cc:493] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.612030  4348 raft_consensus.cc:3060] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.613368  4445 raft_consensus.cc:493] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.613956  4445 raft_consensus.cc:515] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.616174  4445 leader_election.cc:290] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.616617  4280 consensus_queue.cc:260] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.618629  4005 raft_consensus.cc:2468] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:45.621893  4348 raft_consensus.cc:515] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.623171  4476 consensus_queue.cc:1048] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:45.625519  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e4899b95af40fd9836e281f4c23e17" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:45.626210  4080 raft_consensus.cc:3060] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.628471  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:45.629751  4280 ts_tablet_manager.cc:1434] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.078s	user 0.008s	sys 0.001s
I20260709 12:20:45.630097  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.631850  4046 leader_election.cc:304] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:45.632876  4476 raft_consensus.cc:2804] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:45.633292  4476 raft_consensus.cc:493] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.633632  4476 raft_consensus.cc:3060] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.635221  4080 raft_consensus.cc:2468] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:45.636535  3971 leader_election.cc:304] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:45.637365  4347 raft_consensus.cc:2804] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.608103  4228 raft_consensus.cc:2468] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:45.641256  4477 raft_consensus.cc:515] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.644505  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:45.645265  4228 raft_consensus.cc:3060] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.654592  4280 tablet_bootstrap.cc:492] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:45.656409  4004 raft_consensus.cc:3060] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.658525  4272 tablet_bootstrap.cc:492] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:45.659235  4272 ts_tablet_manager.cc:1403] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.215s	user 0.054s	sys 0.050s
I20260709 12:20:45.659396  4476 raft_consensus.cc:515] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.662918  4273 ts_tablet_manager.cc:1434] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.183s	user 0.008s	sys 0.001s
I20260709 12:20:45.664108  4477 leader_election.cc:290] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.664512  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e203dae959394f4e90051ad4746a9291" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:45.665270  4226 raft_consensus.cc:3060] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.665547  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e4899b95af40fd9836e281f4c23e17" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:45.666177  4154 raft_consensus.cc:3060] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.668881  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:45.669618  4005 raft_consensus.cc:3060] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.670846  4273 tablet_bootstrap.cc:492] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:45.674152  4272 raft_consensus.cc:359] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.675104  4272 raft_consensus.cc:385] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.675426  4272 raft_consensus.cc:740] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.678691  4272 consensus_queue.cc:260] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.684805  4476 leader_election.cc:290] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.672593  4228 raft_consensus.cc:2468] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:45.686807  4002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e203dae959394f4e90051ad4746a9291" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:45.687562  4002 raft_consensus.cc:3060] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.688130  4272 ts_tablet_manager.cc:1434] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:20:45.690223  4272 tablet_bootstrap.cc:492] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:45.658565  4348 leader_election.cc:290] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:45.689381  4046 leader_election.cc:304] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:45.698877  4477 raft_consensus.cc:2804] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.699429  4477 raft_consensus.cc:697] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:45.700256  4477 consensus_queue.cc:237] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.702268  4277 tablet_bootstrap.cc:492] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:45.702831  4277 ts_tablet_manager.cc:1403] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.157s	user 0.036s	sys 0.025s
I20260709 12:20:45.704535  4002 raft_consensus.cc:2468] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:45.678844  4347 raft_consensus.cc:697] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:45.714660  4277 raft_consensus.cc:359] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.715857  4277 raft_consensus.cc:385] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.716194  4277 raft_consensus.cc:740] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
W20260709 12:20:45.679665  4120 leader_election.cc:343] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.716795  4277 consensus_queue.cc:260] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:45.726178  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 48104.6us
I20260709 12:20:45.744786  4277 ts_tablet_manager.cc:1434] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.003s
I20260709 12:20:45.745707  4277 tablet_bootstrap.cc:492] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:45.736729  4046 leader_election.cc:304] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:45.746819  4485 raft_consensus.cc:2804] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.747334  4485 raft_consensus.cc:697] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:45.737098  4226 raft_consensus.cc:2468] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:45.733392  4154 raft_consensus.cc:2468] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
W20260709 12:20:45.743754  4120 leader_election.cc:343] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.748198  4485 consensus_queue.cc:237] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.756158  4120 leader_election.cc:304] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:45.758077  4445 raft_consensus.cc:2749] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.759395  4445 consensus_queue.cc:1048] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:45.760131  4347 consensus_queue.cc:237] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.764676  4476 raft_consensus.cc:493] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.760165  3899 catalog_manager.cc:5697] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.764694  4374 raft_consensus.cc:493] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.765833  4374 raft_consensus.cc:515] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.749006  4005 raft_consensus.cc:2468] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:45.767779  4374 leader_election.cc:290] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.768239  4280 tablet_bootstrap.cc:654] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.765261  4476 raft_consensus.cc:515] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:45.769133  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb6ad79685348f4a82858f9f7844c48" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:45.769860  4152 raft_consensus.cc:2468] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:45.770740  4273 tablet_bootstrap.cc:654] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.771116  4476 leader_election.cc:290] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:45.772702  4193 leader_election.cc:304] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:45.773674  4335 consensus_queue.cc:1048] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:45.774726  4325 raft_consensus.cc:2804] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:45.775153  4325 raft_consensus.cc:493] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.775477  4325 raft_consensus.cc:3060] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.798238  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:45.782385  4374 consensus_queue.cc:1048] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:20:45.818015  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:45.818794  4152 raft_consensus.cc:2468] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:45.822281  3899 catalog_manager.cc:5697] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:45.825011  4046 leader_election.cc:343] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.826098  4485 consensus_queue.cc:1048] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:45.834774  4045 leader_election.cc:304] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:45.796633  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb6ad79685348f4a82858f9f7844c48" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.836146  4476 raft_consensus.cc:2804] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:45.836539  4476 raft_consensus.cc:493] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.836885  4476 raft_consensus.cc:3060] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:45.839455  4194 leader_election.cc:343] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.843739  4272 tablet_bootstrap.cc:654] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.855739  4273 tablet_bootstrap.cc:492] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:45.856464  4273 ts_tablet_manager.cc:1403] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.186s	user 0.013s	sys 0.006s
I20260709 12:20:45.858659  3901 catalog_manager.cc:5697] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.860698  4277 tablet_bootstrap.cc:654] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.848306  4325 raft_consensus.cc:515] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.869174  4476 raft_consensus.cc:515] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:45.871507  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb6ad79685348f4a82858f9f7844c48" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:45.872133  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:45.872735  4152 raft_consensus.cc:3060] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:45.873179  4194 leader_election.cc:343] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.874205  4325 leader_election.cc:290] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:45.873798  4476 leader_election.cc:290] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:45.877413  4325 consensus_queue.cc:1048] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:45.877895  4273 raft_consensus.cc:359] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.878648  4273 raft_consensus.cc:385] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.878958  4273 raft_consensus.cc:740] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.879724  4273 consensus_queue.cc:260] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:45.881486  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb6ad79685348f4a82858f9f7844c48" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:45.882203  4153 raft_consensus.cc:3060] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.882102  4152 raft_consensus.cc:2468] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:45.883585  4045 leader_election.cc:304] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:45.884549  4476 raft_consensus.cc:2804] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.885783  4273 ts_tablet_manager.cc:1434] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:20:45.890946  4273 tablet_bootstrap.cc:492] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:45.902751  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:45.885223  4476 raft_consensus.cc:697] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
W20260709 12:20:45.905460  4508 log.cc:927] Time spent T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:20:45.928061  4295 raft_consensus.cc:493] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.928701  4295 raft_consensus.cc:515] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:45.929607  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:45.930939  4295 leader_election.cc:290] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:20:45.915313  4046 leader_election.cc:343] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.946791  4280 tablet_bootstrap.cc:492] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:45.947415  4280 ts_tablet_manager.cc:1403] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.317s	user 0.025s	sys 0.008s
I20260709 12:20:45.950852  4280 raft_consensus.cc:359] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.951555  4280 raft_consensus.cc:385] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.951829  4280 raft_consensus.cc:740] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.956781  4277 tablet_bootstrap.cc:492] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:45.957368  4277 ts_tablet_manager.cc:1403] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.212s	user 0.007s	sys 0.027s
I20260709 12:20:45.961917  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:45.951829  4153 raft_consensus.cc:2468] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:45.963543  4273 tablet_bootstrap.cc:654] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.974184  4193 leader_election.cc:304] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:45.975546  4374 raft_consensus.cc:2804] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.976428  4374 raft_consensus.cc:697] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
W20260709 12:20:45.976958  4120 leader_election.cc:343] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.977507  4374 consensus_queue.cc:237] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:45.983999  4277 raft_consensus.cc:359] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:45.985057  4277 raft_consensus.cc:385] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.985404  4277 raft_consensus.cc:740] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.986267  4277 consensus_queue.cc:260] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:45.964442  4272 tablet_bootstrap.cc:492] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:45.964890  4485 consensus_queue.cc:1048] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:45.965705  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:45.965482  4280 consensus_queue.cc:260] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:45.998919  4272 ts_tablet_manager.cc:1403] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.309s	user 0.014s	sys 0.019s
I20260709 12:20:45.999199  4280 ts_tablet_manager.cc:1434] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 12:20:46.000233  4280 tablet_bootstrap.cc:492] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:45.936097  4476 consensus_queue.cc:237] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.005527  4277 ts_tablet_manager.cc:1434] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
W20260709 12:20:46.002017  4120 leader_election.cc:343] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.006325  4120 leader_election.cc:304] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:46.005445  4272 raft_consensus.cc:359] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.008424  4272 raft_consensus.cc:385] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.009585  4272 raft_consensus.cc:740] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.009690  4489 raft_consensus.cc:2749] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.011837  4272 consensus_queue.cc:260] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.016582  4272 ts_tablet_manager.cc:1434] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:20:46.019341  3901 catalog_manager.cc:5697] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.021020  4280 tablet_bootstrap.cc:654] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.022643  4272 tablet_bootstrap.cc:492] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:46.031693  4277 tablet_bootstrap.cc:492] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
W20260709 12:20:46.035816  4046 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:41077, user_credentials={real_user=slave}} blocked reactor thread for 60543.3us
I20260709 12:20:46.044400  4272 tablet_bootstrap.cc:654] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.072511  4280 tablet_bootstrap.cc:492] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:46.073125  4280 ts_tablet_manager.cc:1403] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.073s	user 0.011s	sys 0.000s
I20260709 12:20:46.076007  4280 raft_consensus.cc:359] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.079433  4277 tablet_bootstrap.cc:654] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.079608  4280 raft_consensus.cc:385] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.083339  4280 raft_consensus.cc:740] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.087321  4280 consensus_queue.cc:260] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.092353  4280 ts_tablet_manager.cc:1434] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.004s
I20260709 12:20:46.097321  4280 tablet_bootstrap.cc:492] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:46.103482  3902 catalog_manager.cc:5697] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.115518  4273 tablet_bootstrap.cc:492] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:46.116222  4273 ts_tablet_manager.cc:1403] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.229s	user 0.015s	sys 0.017s
I20260709 12:20:46.122340  4280 tablet_bootstrap.cc:654] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.134724  4273 raft_consensus.cc:359] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.135587  4273 raft_consensus.cc:385] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.135867  4273 raft_consensus.cc:740] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.136652  4273 consensus_queue.cc:260] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.139060  4273 ts_tablet_manager.cc:1434] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:20:46.140169  4273 tablet_bootstrap.cc:492] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:46.141774  4277 tablet_bootstrap.cc:492] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:46.142340  4277 ts_tablet_manager.cc:1403] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.111s	user 0.011s	sys 0.004s
I20260709 12:20:46.145143  4277 raft_consensus.cc:359] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.155016  4277 raft_consensus.cc:385] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.155344  4277 raft_consensus.cc:740] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.156147  4523 consensus_queue.cc:1048] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:46.162221  4277 consensus_queue.cc:260] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.146008  4272 tablet_bootstrap.cc:492] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:46.146817  4273 tablet_bootstrap.cc:654] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.165179  4272 ts_tablet_manager.cc:1403] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.144s	user 0.015s	sys 0.022s
I20260709 12:20:46.168458  4272 raft_consensus.cc:359] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.169214  4272 raft_consensus.cc:385] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.169538  4272 raft_consensus.cc:740] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.170274  4272 consensus_queue.cc:260] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.172793  4272 ts_tablet_manager.cc:1434] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:46.173430  4277 ts_tablet_manager.cc:1434] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 12:20:46.173959  4272 tablet_bootstrap.cc:492] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:46.153530  4532 raft_consensus.cc:493] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.176963  4277 tablet_bootstrap.cc:492] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:46.176673  4532 raft_consensus.cc:515] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.180408  4532 leader_election.cc:290] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:46.181289  4272 tablet_bootstrap.cc:654] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.187785  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:46.189198  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:46.193567  4046 leader_election.cc:343] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.220858  4273 tablet_bootstrap.cc:492] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:46.221499  4273 ts_tablet_manager.cc:1403] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.011s
I20260709 12:20:46.221964  4272 tablet_bootstrap.cc:492] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:46.222532  4272 ts_tablet_manager.cc:1403] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.024s
I20260709 12:20:46.224861  4273 raft_consensus.cc:359] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.225315  4272 raft_consensus.cc:359] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.226085  4272 raft_consensus.cc:385] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.226370  4272 raft_consensus.cc:740] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.227080  4273 raft_consensus.cc:385] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.227622  4273 raft_consensus.cc:740] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.227159  4272 consensus_queue.cc:260] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.228073  4348 raft_consensus.cc:493] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.228420  4273 consensus_queue.cc:260] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.228850  4348 raft_consensus.cc:515] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.229773  4272 ts_tablet_manager.cc:1434] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:46.230773  4272 tablet_bootstrap.cc:492] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:46.231007  4348 leader_election.cc:290] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:46.232237  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:46.233718  4153 raft_consensus.cc:2468] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:46.235112  3970 leader_election.cc:304] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:46.232964  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:46.236064  4078 raft_consensus.cc:2468] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:46.237142  4512 raft_consensus.cc:2804] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.237538  4512 raft_consensus.cc:493] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.237877  4512 raft_consensus.cc:3060] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.241194  4273 ts_tablet_manager.cc:1434] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:20:46.246232  4512 raft_consensus.cc:515] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:46.247368  4046 leader_election.cc:343] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.247962  4046 leader_election.cc:304] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:46.249455  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:46.250118  4078 raft_consensus.cc:3060] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.251287  4523 raft_consensus.cc:2749] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.251928  4287 consensus_queue.cc:1048] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:46.254210  4273 tablet_bootstrap.cc:492] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:46.255429  4295 raft_consensus.cc:493] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.255940  4295 raft_consensus.cc:515] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.258616  4295 leader_election.cc:290] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:46.260083  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d10e7c5b29b14b4e8c90d79a5201bf26" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:46.260083  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d10e7c5b29b14b4e8c90d79a5201bf26" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:46.261488  4273 tablet_bootstrap.cc:654] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:46.262625  4120 leader_election.cc:343] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:46.263736  4120 leader_election.cc:343] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.264130  4120 leader_election.cc:304] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:46.265210  4295 raft_consensus.cc:2749] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.266400  4078 raft_consensus.cc:2468] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:46.267587  3971 leader_election.cc:304] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:46.268422  4287 raft_consensus.cc:2804] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.268884  4287 raft_consensus.cc:697] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:46.269706  4287 consensus_queue.cc:237] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.274062  4273 tablet_bootstrap.cc:492] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:46.274621  4273 ts_tablet_manager.cc:1403] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.010s
I20260709 12:20:46.276621  4277 tablet_bootstrap.cc:654] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.279004  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:46.279603  4153 raft_consensus.cc:3060] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.279955  4280 tablet_bootstrap.cc:492] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:46.280494  4280 ts_tablet_manager.cc:1403] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.187s	user 0.042s	sys 0.047s
I20260709 12:20:46.276821  4272 tablet_bootstrap.cc:654] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.277871  4512 leader_election.cc:290] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:46.283469  4280 raft_consensus.cc:359] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.284387  4280 raft_consensus.cc:385] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.284744  4280 raft_consensus.cc:740] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.277570  4273 raft_consensus.cc:359] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.285542  4280 consensus_queue.cc:260] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.286173  4273 raft_consensus.cc:385] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.286537  4273 raft_consensus.cc:740] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.287235  4273 consensus_queue.cc:260] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.287760  4153 raft_consensus.cc:2468] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:46.288146  4280 ts_tablet_manager.cc:1434] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:46.290297  4273 ts_tablet_manager.cc:1434] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:20:46.291416  4273 tablet_bootstrap.cc:492] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:46.292354  4280 tablet_bootstrap.cc:492] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:46.294181  4523 consensus_queue.cc:1048] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:46.299950  4280 tablet_bootstrap.cc:654] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.322238  4512 consensus_queue.cc:1048] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:46.322746  4523 raft_consensus.cc:493] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.323542  4523 raft_consensus.cc:515] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.326177  4523 leader_election.cc:290] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:46.326978  3899 catalog_manager.cc:5697] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.348656  4523 consensus_queue.cc:1048] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:46.353782  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08446fff0954c05b5311d3e76153e58" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:46.358712  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08446fff0954c05b5311d3e76153e58" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:46.360567  4280 tablet_bootstrap.cc:492] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:46.361279  4280 ts_tablet_manager.cc:1403] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.069s	user 0.015s	sys 0.021s
I20260709 12:20:46.362984  4004 raft_consensus.cc:2468] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:46.364583  4280 raft_consensus.cc:359] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.365157  4272 tablet_bootstrap.cc:492] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:46.365546  4280 raft_consensus.cc:385] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.365850  4272 ts_tablet_manager.cc:1403] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.135s	user 0.015s	sys 0.022s
I20260709 12:20:46.366009  4280 raft_consensus.cc:740] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.367013  4280 consensus_queue.cc:260] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.369405  4280 ts_tablet_manager.cc:1434] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:20:46.369065  4272 raft_consensus.cc:359] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.370011  4272 raft_consensus.cc:385] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.370321  4272 raft_consensus.cc:740] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.370811  4280 tablet_bootstrap.cc:492] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:46.371083  4272 consensus_queue.cc:260] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.375969  4277 tablet_bootstrap.cc:492] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:46.376595  4277 ts_tablet_manager.cc:1403] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.200s	user 0.019s	sys 0.023s
I20260709 12:20:46.378778  4280 tablet_bootstrap.cc:654] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.379683  4277 raft_consensus.cc:359] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.380455  4277 raft_consensus.cc:385] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.380733  4277 raft_consensus.cc:740] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.373528  4272 ts_tablet_manager.cc:1434] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 12:20:46.382591  4272 tablet_bootstrap.cc:492] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:46.383078  4273 tablet_bootstrap.cc:654] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:46.366979  4046 leader_election.cc:343] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.388830  4277 consensus_queue.cc:260] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.392253  4225 raft_consensus.cc:3060] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.393464  4046 leader_election.cc:304] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:46.394446  4523 raft_consensus.cc:2804] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.394850  4523 raft_consensus.cc:493] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.395206  4523 raft_consensus.cc:3060] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.401800  4365 raft_consensus.cc:493] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.403482  4272 tablet_bootstrap.cc:654] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.403908  4523 raft_consensus.cc:515] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.402514  4365 raft_consensus.cc:515] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.392225  4277 ts_tablet_manager.cc:1434] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:20:46.406755  4524 consensus_queue.cc:1048] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:46.408262  4365 leader_election.cc:290] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:46.408847  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:46.406947  4277 tablet_bootstrap.cc:492] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:46.411387  4287 raft_consensus.cc:493] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.411960  4287 raft_consensus.cc:515] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.413267  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08446fff0954c05b5311d3e76153e58" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:46.414260  4287 leader_election.cc:290] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:46.411432  4524 consensus_queue.cc:1048] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:20:46.416401  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:46.417539  4277 tablet_bootstrap.cc:654] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.418068  4194 leader_election.cc:304] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:46.419359  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:46.420025  4374 raft_consensus.cc:2749] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:20:46.415144  4046 leader_election.cc:343] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.421708  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
W20260709 12:20:46.422482  3971 leader_election.cc:343] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.422410  4079 raft_consensus.cc:2468] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:46.424891  4226 raft_consensus.cc:3060] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.426287  4523 leader_election.cc:290] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:46.424733  4272 tablet_bootstrap.cc:492] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:46.427310  4272 ts_tablet_manager.cc:1403] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.045s	user 0.005s	sys 0.007s
I20260709 12:20:46.432469  4335 raft_consensus.cc:493] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.433106  4335 raft_consensus.cc:515] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.435341  4335 leader_election.cc:290] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:46.436305  4272 raft_consensus.cc:359] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.437242  4272 raft_consensus.cc:385] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.437517  4272 raft_consensus.cc:740] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.437575  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9fe17de2e1845dd959d6b3d892cda55" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:46.437623  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9fe17de2e1845dd959d6b3d892cda55" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:46.438367  4272 consensus_queue.cc:260] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.438776  4287 raft_consensus.cc:493] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.439316  4287 raft_consensus.cc:515] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.438372  4079 raft_consensus.cc:2468] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:46.442236  4524 consensus_queue.cc:1048] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:20:46.419399  4273 tablet_bootstrap.cc:492] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
W20260709 12:20:46.443150  4120 leader_election.cc:343] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.472790  4273 ts_tablet_manager.cc:1403] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.182s	user 0.012s	sys 0.000s
I20260709 12:20:46.473006  4120 leader_election.cc:304] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:46.476159  4417 raft_consensus.cc:2804] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.475888  4273 raft_consensus.cc:359] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.476585  4417 raft_consensus.cc:493] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.477010  4417 raft_consensus.cc:3060] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.479383  4273 raft_consensus.cc:385] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.479746  4273 raft_consensus.cc:740] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.480551  4273 consensus_queue.cc:260] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.482745  4445 raft_consensus.cc:493] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.483160  4273 ts_tablet_manager.cc:1434] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:20:46.483325  4445 raft_consensus.cc:515] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.484308  4273 tablet_bootstrap.cc:492] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:46.485059  4225 raft_consensus.cc:3060] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.485621  4445 leader_election.cc:290] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:46.491818  4280 tablet_bootstrap.cc:492] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:46.492115  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f92fa80bd34e60924d5c77f79df421" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:46.492479  4280 ts_tablet_manager.cc:1403] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.122s	user 0.034s	sys 0.029s
W20260709 12:20:46.495800  4120 leader_election.cc:343] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.495910  4280 raft_consensus.cc:359] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.496680  4280 raft_consensus.cc:385] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.496963  4280 raft_consensus.cc:740] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.497620  4280 consensus_queue.cc:260] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.499948  4280 ts_tablet_manager.cc:1434] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:20:46.501137  4280 tablet_bootstrap.cc:492] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:46.504096  4273 tablet_bootstrap.cc:654] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.441556  4287 leader_election.cc:290] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:46.470031  4005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08446fff0954c05b5311d3e76153e58" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:46.513775  4005 raft_consensus.cc:3060] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.515437  4280 tablet_bootstrap.cc:654] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.442816  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:46.518724  4154 raft_consensus.cc:2468] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:46.456012  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:46.520401  4079 raft_consensus.cc:2468] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:46.456490  4277 tablet_bootstrap.cc:492] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:46.522359  4277 ts_tablet_manager.cc:1403] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.116s	user 0.010s	sys 0.004s
I20260709 12:20:46.526400  3970 leader_election.cc:304] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:46.450243  4272 ts_tablet_manager.cc:1434] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 12:20:46.528579  4272 tablet_bootstrap.cc:492] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:46.537395  4272 tablet_bootstrap.cc:654] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.525538  4277 raft_consensus.cc:359] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.538535  4277 raft_consensus.cc:385] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.538828  4277 raft_consensus.cc:740] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.540439  4277 consensus_queue.cc:260] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.543335  4348 raft_consensus.cc:2804] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.543879  4348 raft_consensus.cc:493] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.544221  4348 raft_consensus.cc:3060] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.544412  4277 ts_tablet_manager.cc:1434] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:20:46.545598  4277 tablet_bootstrap.cc:492] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:46.511077  4417 raft_consensus.cc:515] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.469959  3971 leader_election.cc:304] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 72940e4e52944adb81608e286ad58a59
W20260709 12:20:46.557014  3971 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 87160.4us
I20260709 12:20:46.565655  4417 leader_election.cc:290] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:46.566692  4005 raft_consensus.cc:2468] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:46.568702  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9fe17de2e1845dd959d6b3d892cda55" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
W20260709 12:20:46.575723  4120 leader_election.cc:343] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.578977  4374 raft_consensus.cc:493] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.579761  4374 raft_consensus.cc:515] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.581411  4277 tablet_bootstrap.cc:654] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:46.583448  4194 consensus_peers.cc:597] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:46.585819  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9fe17de2e1845dd959d6b3d892cda55" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:46.586561  4078 raft_consensus.cc:3060] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.561842  4287 raft_consensus.cc:2804] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.589922  4287 raft_consensus.cc:493] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.590333  4287 raft_consensus.cc:3060] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.601286  4374 leader_election.cc:290] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:46.609321  4046 leader_election.cc:304] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:46.610607  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:46.610988  4272 tablet_bootstrap.cc:492] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:46.628583  4272 ts_tablet_manager.cc:1403] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.100s	user 0.013s	sys 0.001s
I20260709 12:20:46.612658  4325 consensus_queue.cc:1048] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:46.616909  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f92fa80bd34e60924d5c77f79df421" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:46.630303  4374 raft_consensus.cc:493] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.633813  4525 raft_consensus.cc:2804] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.651010  4280 tablet_bootstrap.cc:492] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:46.651671  4280 ts_tablet_manager.cc:1403] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.151s	user 0.022s	sys 0.031s
W20260709 12:20:46.646956  4193 leader_election.cc:343] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.648811  4525 raft_consensus.cc:697] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:46.643625  4374 raft_consensus.cc:515] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.664981  4525 consensus_queue.cc:237] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.659703  4484 raft_consensus.cc:493] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.656572  4287 raft_consensus.cc:515] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.668723  4484 raft_consensus.cc:515] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.669612  4287 leader_election.cc:290] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:46.672848  4365 raft_consensus.cc:493] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 25902f00fb5d41679245f23dd19654aa)
I20260709 12:20:46.673206  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "724697c71b61499aa0daadd2e66af805" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:46.673782  4365 raft_consensus.cc:515] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.671280  4484 leader_election.cc:290] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
W20260709 12:20:46.675757  4120 leader_election.cc:343] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.664027  3999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:46.677990  3999 raft_consensus.cc:2468] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:46.673902  4078 raft_consensus.cc:2468] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:46.682859  4273 tablet_bootstrap.cc:492] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:46.683493  4273 ts_tablet_manager.cc:1403] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.199s	user 0.033s	sys 0.016s
I20260709 12:20:46.684124  4277 tablet_bootstrap.cc:492] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:46.676838  4280 raft_consensus.cc:359] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.684581  4277 ts_tablet_manager.cc:1403] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.139s	user 0.024s	sys 0.004s
I20260709 12:20:46.686635  4374 leader_election.cc:290] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:46.687469  4120 leader_election.cc:304] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:46.688690  4417 raft_consensus.cc:2804] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.688885  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:46.689248  4417 raft_consensus.cc:697] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:46.690562  4365 leader_election.cc:290] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:46.690287  4417 consensus_queue.cc:237] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.690661  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b7f39842e94931a878f3003c1353e1" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:46.690178  4078 raft_consensus.cc:2468] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:46.690338  4348 raft_consensus.cc:515] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.694140  4194 leader_election.cc:304] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:46.684850  4280 raft_consensus.cc:385] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.696718  4280 raft_consensus.cc:740] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.688829  4273 raft_consensus.cc:359] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.697922  4273 raft_consensus.cc:385] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.698745  4273 raft_consensus.cc:740] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.699851  4280 consensus_queue.cc:260] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.695154  4365 raft_consensus.cc:2804] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.702930  4365 raft_consensus.cc:493] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.703357  4365 raft_consensus.cc:3060] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.703590  4277 raft_consensus.cc:359] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.704365  4277 raft_consensus.cc:385] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.704645  4277 raft_consensus.cc:740] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.705081  4348 leader_election.cc:290] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:46.705418  4277 consensus_queue.cc:260] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.706434  4194 leader_election.cc:304] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 6994df41341e478d89f7c105f550ff93
I20260709 12:20:46.707809  4374 raft_consensus.cc:2804] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.708216  4277 ts_tablet_manager.cc:1434] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
W20260709 12:20:46.708242  4120 leader_election.cc:343] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.714476  4374 raft_consensus.cc:493] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.714550  4120 leader_election.cc:304] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:46.715266  4374 raft_consensus.cc:3060] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.715638  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "724697c71b61499aa0daadd2e66af805" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:46.711162  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:46.716925  4078 raft_consensus.cc:3060] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.716804  4335 raft_consensus.cc:2749] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.720233  4273 consensus_queue.cc:260] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.721277  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38b7f39842e94931a878f3003c1353e1" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:46.722847  4273 ts_tablet_manager.cc:1434] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:20:46.724154  4193 leader_election.cc:304] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
W20260709 12:20:46.726341  4120 leader_election.cc:343] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.706569  4272 raft_consensus.cc:359] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.735579  4272 raft_consensus.cc:385] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.735903  4272 raft_consensus.cc:740] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.711796  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:46.736492  4079 raft_consensus.cc:3060] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.738191  4280 ts_tablet_manager.cc:1434] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.086s	user 0.004s	sys 0.005s
I20260709 12:20:46.735916  4374 raft_consensus.cc:515] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.732187  4273 tablet_bootstrap.cc:492] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:46.738067  4120 leader_election.cc:304] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:46.738992  4272 consensus_queue.cc:260] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.734930  4277 tablet_bootstrap.cc:492] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:46.733476  4078 raft_consensus.cc:2468] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:46.762249  4280 tablet_bootstrap.cc:492] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:46.751003  4484 raft_consensus.cc:2749] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.771783  4272 ts_tablet_manager.cc:1434] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.143s	user 0.007s	sys 0.000s
I20260709 12:20:46.773084  4272 tablet_bootstrap.cc:492] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:46.753825  4325 raft_consensus.cc:2749] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:20:46.768991  4193 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 44951.3us
I20260709 12:20:46.797030  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:46.810298  4365 raft_consensus.cc:515] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.810961  4374 leader_election.cc:290] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:46.813690  3971 leader_election.cc:304] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:46.815943  4561 raft_consensus.cc:2804] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.816138  4365 leader_election.cc:290] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:46.816506  4561 raft_consensus.cc:697] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:46.818188  4226 raft_consensus.cc:2393] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 41009fde51f045c49be6461b76a27543 in current term 1: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:46.836865  4280 tablet_bootstrap.cc:654] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.837707  4561 consensus_queue.cc:237] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.840917  4287 raft_consensus.cc:493] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.841475  4287 raft_consensus.cc:515] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.845065  4287 leader_election.cc:290] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:46.848121  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:46.848930  4225 raft_consensus.cc:2468] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:46.840163  3899 catalog_manager.cc:5697] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.881989  3971 leader_election.cc:304] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:46.827622  4273 tablet_bootstrap.cc:654] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.883005  4287 raft_consensus.cc:2804] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.883554  4287 raft_consensus.cc:493] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.883908  4287 raft_consensus.cc:3060] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.831509  4277 tablet_bootstrap.cc:654] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.832682  4079 raft_consensus.cc:2468] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:46.832855  3901 catalog_manager.cc:5697] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:46.834009  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:46.892361  4153 raft_consensus.cc:3060] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.836580  4272 tablet_bootstrap.cc:654] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.907884  3971 leader_election.cc:304] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:46.908299  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:46.909536  4079 raft_consensus.cc:2393] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 41009fde51f045c49be6461b76a27543 in this term.
I20260709 12:20:46.915072  4567 raft_consensus.cc:2804] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.916237  3999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:46.919312  3999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:46.920034  3999 raft_consensus.cc:3060] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.922664  4567 raft_consensus.cc:697] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:46.923607  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:46.925699  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:46.927399  4152 raft_consensus.cc:3060] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.952678  4566 consensus_queue.cc:1048] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:20:46.956218  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:46.957086  4154 raft_consensus.cc:2468] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:46.960494  4277 tablet_bootstrap.cc:492] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:46.961130  4277 ts_tablet_manager.cc:1403] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.226s	user 0.013s	sys 0.008s
I20260709 12:20:46.964036  4272 tablet_bootstrap.cc:492] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:46.957125  4152 raft_consensus.cc:2468] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:46.964624  4272 ts_tablet_manager.cc:1403] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.192s	user 0.016s	sys 0.013s
I20260709 12:20:46.962988  4567 consensus_queue.cc:237] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:46.971485  4525 raft_consensus.cc:493] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.982980  4525 raft_consensus.cc:515] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:46.985347  4525 leader_election.cc:290] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:46.985848  4272 raft_consensus.cc:359] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:46.986691  4272 raft_consensus.cc:385] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.986505  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:46.987227  4272 raft_consensus.cc:740] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
W20260709 12:20:46.988543  4046 leader_election.cc:343] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.989545  4194 leader_election.cc:304] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:46.990329  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:46.991786  4325 raft_consensus.cc:2749] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:20:46.992348  4046 leader_election.cc:343] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.993289  4046 leader_election.cc:304] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:46.995391  4525 raft_consensus.cc:2749] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.966508  4153 raft_consensus.cc:2468] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:46.997936  3999 raft_consensus.cc:2468] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:46.999672  4287 raft_consensus.cc:515] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:46.988149  4272 consensus_queue.cc:260] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.002942  4273 tablet_bootstrap.cc:492] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:47.003499  4273 ts_tablet_manager.cc:1403] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.272s	user 0.017s	sys 0.005s
I20260709 12:20:46.998773  4193 leader_election.cc:304] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:47.012777  4277 raft_consensus.cc:359] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.013815  4277 raft_consensus.cc:385] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.014119  4277 raft_consensus.cc:740] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.011214  4280 tablet_bootstrap.cc:492] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:47.015130  4280 ts_tablet_manager.cc:1403] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.253s	user 0.014s	sys 0.015s
I20260709 12:20:47.014843  4566 consensus_queue.cc:1048] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.020687  4325 raft_consensus.cc:2804] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.021373  4325 raft_consensus.cc:697] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:47.022540  4325 consensus_queue.cc:237] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.019701  4277 consensus_queue.cc:260] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.030310  4272 ts_tablet_manager.cc:1434] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.000s
I20260709 12:20:47.040585  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:47.041647  4226 raft_consensus.cc:3060] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.040680  4287 leader_election.cc:290] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:47.034576  4272 tablet_bootstrap.cc:492] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:47.056962  4273 raft_consensus.cc:359] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.058480  4273 raft_consensus.cc:385] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.058924  4273 raft_consensus.cc:740] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.053052  4226 raft_consensus.cc:2468] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:47.070865  4277 ts_tablet_manager.cc:1434] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.109s	user 0.006s	sys 0.003s
I20260709 12:20:47.074910  4277 tablet_bootstrap.cc:492] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:47.062799  4273 consensus_queue.cc:260] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.078912  4566 raft_consensus.cc:493] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.079586  4566 raft_consensus.cc:515] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.082062  4566 leader_election.cc:290] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:47.084581  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb6ad79685348f4a82858f9f7844c48" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:47.090647  4280 raft_consensus.cc:359] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.095674  4280 raft_consensus.cc:385] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.096010  4280 raft_consensus.cc:740] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.097050  4280 consensus_queue.cc:260] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.098788  4280 ts_tablet_manager.cc:1434] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.000s
I20260709 12:20:47.079162  4273 ts_tablet_manager.cc:1434] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.075s	user 0.011s	sys 0.001s
I20260709 12:20:47.102283  4273 tablet_bootstrap.cc:492] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:47.102555  4525 raft_consensus.cc:493] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.103080  4525 raft_consensus.cc:515] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.105294  4525 leader_election.cc:290] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:47.107543  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d10e7c5b29b14b4e8c90d79a5201bf26" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:47.107995  4568 raft_consensus.cc:493] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.108525  4153 raft_consensus.cc:2468] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:47.108556  4568 raft_consensus.cc:515] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.112337  4568 leader_election.cc:290] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:47.112937  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc6c1656d03459babbca1538b0ae0ef" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:47.108008  3999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d10e7c5b29b14b4e8c90d79a5201bf26" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:47.121661  3971 leader_election.cc:304] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:47.121964  4583 consensus_queue.cc:1048] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:20:47.132477  4287 raft_consensus.cc:2804] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.133091  4287 raft_consensus.cc:697] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:47.134075  4287 consensus_queue.cc:237] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.079151  4272 tablet_bootstrap.cc:654] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.119242  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:47.135785  4150 raft_consensus.cc:3060] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.122444  4280 tablet_bootstrap.cc:492] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:47.121440  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc6c1656d03459babbca1538b0ae0ef" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:47.113315  4045 leader_election.cc:304] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:47.123983  4277 tablet_bootstrap.cc:654] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:47.142072  4193 leader_election.cc:343] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:47.142638  4194 leader_election.cc:343] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.143148  4194 leader_election.cc:304] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:47.143703  4523 raft_consensus.cc:2804] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:47.144167  4523 raft_consensus.cc:493] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.126713  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebb6ad79685348f4a82858f9f7844c48" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:47.125412  4273 tablet_bootstrap.cc:654] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.144498  4523 raft_consensus.cc:3060] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.144418  4325 raft_consensus.cc:2749] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.130257  3999 raft_consensus.cc:2468] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:47.129283  4004 raft_consensus.cc:3060] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.152098  3970 leader_election.cc:304] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:47.148319  4150 raft_consensus.cc:2468] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:47.156798  4280 tablet_bootstrap.cc:654] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.158780  4272 tablet_bootstrap.cc:492] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:47.159430  4272 ts_tablet_manager.cc:1403] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.126s	user 0.014s	sys 0.000s
W20260709 12:20:47.167618  4046 consensus_peers.cc:597] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa -> Peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Couldn't send request to peer 72940e4e52944adb81608e286ad58a59. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:47.171152  4272 raft_consensus.cc:359] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:47.173645  4273 tablet_bootstrap.cc:492] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:47.178377  4277 tablet_bootstrap.cc:492] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:47.188825  4272 raft_consensus.cc:385] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.189316  4272 raft_consensus.cc:740] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.184950  4280 tablet_bootstrap.cc:492] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:47.193397  4280 ts_tablet_manager.cc:1403] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.071s	user 0.009s	sys 0.005s
I20260709 12:20:47.189458  4277 ts_tablet_manager.cc:1403] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.115s	user 0.019s	sys 0.000s
I20260709 12:20:47.194416  4272 consensus_queue.cc:260] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:47.178885  4523 raft_consensus.cc:515] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.197491  4277 raft_consensus.cc:359] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.190053  4273 ts_tablet_manager.cc:1403] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.088s	user 0.013s	sys 0.001s
I20260709 12:20:47.199712  4561 raft_consensus.cc:2749] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.201939  4272 ts_tablet_manager.cc:1434] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:20:47.203428  4273 raft_consensus.cc:359] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.209317  4273 raft_consensus.cc:385] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.210086  4272 tablet_bootstrap.cc:492] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:47.217914  4272 tablet_bootstrap.cc:654] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.217878  3901 catalog_manager.cc:5697] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.209668  4273 raft_consensus.cc:740] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.227046  4273 consensus_queue.cc:260] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.227336  3902 catalog_manager.cc:5697] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.202026  4277 raft_consensus.cc:385] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:20:47.205886  4120 consensus_peers.cc:597] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 -> Peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Couldn't send request to peer 72940e4e52944adb81608e286ad58a59. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:47.229638  3902 catalog_manager.cc:5697] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.244451  4325 raft_consensus.cc:493] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.245344  4325 raft_consensus.cc:515] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:47.250953  3902 catalog_manager.cc:5697] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:47.253137  4325 leader_election.cc:290] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:47.253916  4273 ts_tablet_manager.cc:1434] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 12:20:47.254210  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08446fff0954c05b5311d3e76153e58" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:47.255234  4273 tablet_bootstrap.cc:492] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:47.258198  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d10e7c5b29b14b4e8c90d79a5201bf26" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:47.258913  4150 raft_consensus.cc:3060] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:47.260485  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:47.261344  4597 consensus_queue.cc:1048] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.265498  4581 raft_consensus.cc:493] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.266057  4581 raft_consensus.cc:515] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.268414  4581 leader_election.cc:290] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:47.270450  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:47.273181  4120 leader_election.cc:343] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.273546  3998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
W20260709 12:20:47.275225  4120 leader_election.cc:343] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.275382  4150 raft_consensus.cc:2468] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:47.277426  4120 leader_election.cc:304] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:47.278316  4581 raft_consensus.cc:2749] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.279369  4277 raft_consensus.cc:740] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.279975  4045 leader_election.cc:304] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:47.280256  4277 consensus_queue.cc:260] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.281962  4597 raft_consensus.cc:2804] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.282487  4597 raft_consensus.cc:697] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:47.205551  4280 raft_consensus.cc:359] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.207657  4523 leader_election.cc:290] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:47.283391  4597 consensus_queue.cc:237] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.208379  3999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d10e7c5b29b14b4e8c90d79a5201bf26" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:47.284593  3999 raft_consensus.cc:3060] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.290946  4273 tablet_bootstrap.cc:654] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.294974  3999 raft_consensus.cc:2468] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:47.296083  4272 tablet_bootstrap.cc:492] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:47.296162  4280 raft_consensus.cc:385] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.296602  4280 raft_consensus.cc:740] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.296686  4272 ts_tablet_manager.cc:1403] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.087s	user 0.024s	sys 0.023s
I20260709 12:20:47.300729  4002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08446fff0954c05b5311d3e76153e58" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:47.301957  4272 raft_consensus.cc:359] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.302817  4272 raft_consensus.cc:385] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.303133  4272 raft_consensus.cc:740] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.304242  4277 ts_tablet_manager.cc:1434] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.109s	user 0.005s	sys 0.001s
I20260709 12:20:47.304071  4272 consensus_queue.cc:260] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.305454  4277 tablet_bootstrap.cc:492] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:47.309245  4194 leader_election.cc:304] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:47.310909  4568 raft_consensus.cc:2749] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.308017  4280 consensus_queue.cc:260] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.313573  4280 ts_tablet_manager.cc:1434] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.120s	user 0.006s	sys 0.000s
I20260709 12:20:47.314574  4593 raft_consensus.cc:493] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.316758  4272 ts_tablet_manager.cc:1434] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:20:47.318426  4593 raft_consensus.cc:515] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.318027  4277 tablet_bootstrap.cc:654] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.319140  4272 tablet_bootstrap.cc:492] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:47.322403  4593 leader_election.cc:290] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:47.324172  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:47.329146  3999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:47.339897  4280 tablet_bootstrap.cc:492] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:47.341610  3999 raft_consensus.cc:2468] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:47.343178  4120 leader_election.cc:304] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:47.339975  4079 raft_consensus.cc:2468] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:47.343624  4277 tablet_bootstrap.cc:492] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:47.344244  4277 ts_tablet_manager.cc:1403] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.001s
I20260709 12:20:47.344892  4581 raft_consensus.cc:2804] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:47.345292  4581 raft_consensus.cc:493] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.345628  4581 raft_consensus.cc:3060] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.347263  4277 raft_consensus.cc:359] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.348090  4277 raft_consensus.cc:385] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.348382  4277 raft_consensus.cc:740] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.351006  4277 consensus_queue.cc:260] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.336258  4272 tablet_bootstrap.cc:654] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.354748  4277 ts_tablet_manager.cc:1434] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:20:47.355859  4277 tablet_bootstrap.cc:492] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:47.335804  4273 tablet_bootstrap.cc:492] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:47.357321  4273 ts_tablet_manager.cc:1403] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.102s	user 0.019s	sys 0.000s
I20260709 12:20:47.362612  4277 tablet_bootstrap.cc:654] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.364681  4273 raft_consensus.cc:359] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.365521  4273 raft_consensus.cc:385] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.365830  4273 raft_consensus.cc:740] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.374190  4287 consensus_queue.cc:1048] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.384001  4277 tablet_bootstrap.cc:492] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:47.384604  4277 ts_tablet_manager.cc:1403] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.009s
I20260709 12:20:47.387696  4277 raft_consensus.cc:359] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.361377  4581 raft_consensus.cc:515] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.388516  4277 raft_consensus.cc:385] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.392066  4277 raft_consensus.cc:740] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.392844  4277 consensus_queue.cc:260] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.394548  4277 ts_tablet_manager.cc:1434] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:20:47.395550  4277 tablet_bootstrap.cc:492] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:47.401842  4273 consensus_queue.cc:260] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.403475  4277 tablet_bootstrap.cc:654] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.404650  4273 ts_tablet_manager.cc:1434] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:20:47.405928  4348 consensus_queue.cc:1048] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:47.408226  3899 catalog_manager.cc:5697] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.414666  4272 tablet_bootstrap.cc:492] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:47.415282  4272 ts_tablet_manager.cc:1403] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.096s	user 0.010s	sys 0.005s
I20260709 12:20:47.418293  3999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:47.418392  4272 raft_consensus.cc:359] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.419291  4272 raft_consensus.cc:385] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.419276  3999 raft_consensus.cc:3060] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.419749  4272 raft_consensus.cc:740] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.424360  4348 consensus_queue.cc:1048] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.424979  4272 consensus_queue.cc:260] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.426388  4581 leader_election.cc:290] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:47.427693  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:47.428409  4079 raft_consensus.cc:3060] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.429338  4280 tablet_bootstrap.cc:654] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.437949  4079 raft_consensus.cc:2468] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:47.439400  4120 leader_election.cc:304] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:47.444927  4581 raft_consensus.cc:2804] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.445851  4273 tablet_bootstrap.cc:492] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:47.446441  4581 raft_consensus.cc:697] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:47.447517  4581 consensus_queue.cc:237] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.448729  4272 ts_tablet_manager.cc:1434] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:20:47.451179  4272 tablet_bootstrap.cc:492] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:47.467432  4273 tablet_bootstrap.cc:654] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.468722  4272 tablet_bootstrap.cc:654] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.516872  3902 catalog_manager.cc:5697] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:47.529333  4597 raft_consensus.cc:493] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.534498  4597 raft_consensus.cc:515] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.536844  4597 leader_election.cc:290] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:47.538146  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "724697c71b61499aa0daadd2e66af805" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:47.538857  4228 raft_consensus.cc:2468] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:47.540227  4046 leader_election.cc:304] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:47.541326  4597 raft_consensus.cc:2804] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:47.541795  4597 raft_consensus.cc:493] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.542205  4597 raft_consensus.cc:3060] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.547677  4150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "724697c71b61499aa0daadd2e66af805" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:47.548422  4150 raft_consensus.cc:2468] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:47.550699  4597 raft_consensus.cc:515] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.474296  4568 consensus_queue.cc:1048] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.077s
I20260709 12:20:47.556046  4280 tablet_bootstrap.cc:492] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:47.556634  4280 ts_tablet_manager.cc:1403] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.218s	user 0.016s	sys 0.006s
I20260709 12:20:47.557444  4325 consensus_queue.cc:1048] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:47.559617  4280 raft_consensus.cc:359] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.560329  4280 raft_consensus.cc:385] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.560606  4280 raft_consensus.cc:740] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.563607  4273 tablet_bootstrap.cc:492] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:47.564330  4273 ts_tablet_manager.cc:1403] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.159s	user 0.012s	sys 0.000s
I20260709 12:20:47.566500  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "724697c71b61499aa0daadd2e66af805" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:47.567232  4153 raft_consensus.cc:3060] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.567250  4273 raft_consensus.cc:359] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:47.568658  4273 raft_consensus.cc:385] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.568938  4273 raft_consensus.cc:740] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.575369  4228 raft_consensus.cc:3060] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.577111  4153 raft_consensus.cc:2468] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:47.578577  4045 leader_election.cc:304] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:47.579514  4532 raft_consensus.cc:2804] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.580046  4532 raft_consensus.cc:697] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:47.580932  4532 consensus_queue.cc:237] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.569862  4280 consensus_queue.cc:260] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.594218  4287 raft_consensus.cc:493] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.594892  4287 raft_consensus.cc:515] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.600005  4561 consensus_queue.cc:1048] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.604488  4280 ts_tablet_manager.cc:1434] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 12:20:47.607565  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:47.610806  4151 raft_consensus.cc:2468] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:47.612452  3970 leader_election.cc:304] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:47.613336  4561 raft_consensus.cc:2804] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:47.613785  4561 raft_consensus.cc:493] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.614281  4561 raft_consensus.cc:3060] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.617274  4226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "724697c71b61499aa0daadd2e66af805" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:47.617957  4226 raft_consensus.cc:3060] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.618625  4597 leader_election.cc:290] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:47.606076  4287 leader_election.cc:290] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:47.622757  4561 raft_consensus.cc:515] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.592509  4273 consensus_queue.cc:260] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:47.617204  4612 raft_consensus.cc:493] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.626048  4612 raft_consensus.cc:515] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.626876  4280 tablet_bootstrap.cc:492] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:47.625496  4273 ts_tablet_manager.cc:1434] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 12:20:47.606148  4272 tablet_bootstrap.cc:492] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:47.624253  3899 catalog_manager.cc:5697] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.628356  4273 tablet_bootstrap.cc:492] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:47.628319  4272 ts_tablet_manager.cc:1403] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.177s	user 0.019s	sys 0.031s
I20260709 12:20:47.631631  4272 raft_consensus.cc:359] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:47.632261  4272 raft_consensus.cc:385] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.632511  4272 raft_consensus.cc:740] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.639515  4273 tablet_bootstrap.cc:654] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.643595  4593 raft_consensus.cc:493] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.644258  4593 raft_consensus.cc:515] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.646477  4612 leader_election.cc:290] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:47.647434  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9fe17de2e1845dd959d6b3d892cda55" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:47.647679  4593 leader_election.cc:290] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:47.649304  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc6c1656d03459babbca1538b0ae0ef" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:47.650699  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:47.651410  3999 raft_consensus.cc:2468] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:47.655828  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9fe17de2e1845dd959d6b3d892cda55" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
W20260709 12:20:47.656889  4120 leader_election.cc:343] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.634532  4272 consensus_queue.cc:260] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:47.671221  3971 leader_election.cc:343] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.670145  4226 raft_consensus.cc:2468] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:47.674072  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc6c1656d03459babbca1538b0ae0ef" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:47.674921  4224 raft_consensus.cc:2468] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:47.680148  4120 leader_election.cc:304] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa
I20260709 12:20:47.685292  4581 raft_consensus.cc:2804] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:47.685770  4581 raft_consensus.cc:493] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.686126  4581 raft_consensus.cc:3060] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.693112  4272 ts_tablet_manager.cc:1434] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.064s	user 0.005s	sys 0.002s
I20260709 12:20:47.698750  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:47.699438  4153 raft_consensus.cc:3060] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.718245  4194 leader_election.cc:304] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:47.721540  4568 raft_consensus.cc:2749] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.726178  4280 tablet_bootstrap.cc:654] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.729588  4567 consensus_queue.cc:1048] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.734257  4561 leader_election.cc:290] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:47.734560  4228 raft_consensus.cc:3060] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.737895  4567 consensus_queue.cc:1048] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:47.746748  4272 tablet_bootstrap.cc:492] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:47.757022  4525 consensus_queue.cc:1048] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.767953  4581 raft_consensus.cc:515] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.782766  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:47.791442  4525 consensus_queue.cc:1048] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.793145  4272 tablet_bootstrap.cc:654] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:47.827296  3971 leader_election.cc:343] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.835839  4445 raft_consensus.cc:493] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.846587  4445 raft_consensus.cc:515] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.848899  4445 leader_election.cc:290] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:47.852267  3999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fda924a0568433fa70ab72d4a0c3968" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:47.853483  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fda924a0568433fa70ab72d4a0c3968" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:47.855013  4153 raft_consensus.cc:2468] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
W20260709 12:20:47.857172  4120 leader_election.cc:343] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.857316  3970 leader_election.cc:304] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:47.858266  4287 raft_consensus.cc:2804] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.858733  4287 raft_consensus.cc:697] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:47.859508  4287 consensus_queue.cc:237] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:47.860749  4120 leader_election.cc:343] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.861189  4120 leader_election.cc:304] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:47.863309  4581 leader_election.cc:290] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:47.863863  4581 raft_consensus.cc:2749] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.878119  4581 consensus_queue.cc:1048] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:20:47.891374  4280 tablet_bootstrap.cc:492] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:47.895320  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc6c1656d03459babbca1538b0ae0ef" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:47.897231  4228 raft_consensus.cc:3060] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.921072  4277 tablet_bootstrap.cc:492] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:47.921693  4228 raft_consensus.cc:2468] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:47.921919  4277 ts_tablet_manager.cc:1403] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.527s	user 0.142s	sys 0.142s
I20260709 12:20:47.930197  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc6c1656d03459babbca1538b0ae0ef" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:47.932703  4277 raft_consensus.cc:359] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.933497  4277 raft_consensus.cc:385] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.933871  4277 raft_consensus.cc:740] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.934763  4277 consensus_queue.cc:260] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.936941  4280 ts_tablet_manager.cc:1403] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.310s	user 0.020s	sys 0.006s
W20260709 12:20:47.944100  4120 leader_election.cc:343] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.945451  4445 consensus_queue.cc:1048] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.946053  4120 leader_election.cc:304] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa
I20260709 12:20:47.947060  4445 raft_consensus.cc:2804] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.947571  4445 raft_consensus.cc:697] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:47.952804  4273 tablet_bootstrap.cc:492] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:47.953400  4273 ts_tablet_manager.cc:1403] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.325s	user 0.022s	sys 0.003s
I20260709 12:20:47.956346  4273 raft_consensus.cc:359] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.962373  4273 raft_consensus.cc:385] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.962667  4273 raft_consensus.cc:740] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.963541  4277 ts_tablet_manager.cc:1434] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.002s
I20260709 12:20:47.965999  4272 tablet_bootstrap.cc:492] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:47.954612  4445 consensus_queue.cc:237] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:47.983222  4272 ts_tablet_manager.cc:1403] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.237s	user 0.007s	sys 0.017s
I20260709 12:20:47.982530  4273 consensus_queue.cc:260] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:47.985942  4273 ts_tablet_manager.cc:1434] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:20:47.986953  4273 tablet_bootstrap.cc:492] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:47.988052  4277 tablet_bootstrap.cc:492] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:47.990474  4272 raft_consensus.cc:359] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:47.991276  4272 raft_consensus.cc:385] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.996735  4273 tablet_bootstrap.cc:654] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.981001  4280 raft_consensus.cc:359] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:47.998500  4280 raft_consensus.cc:385] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.998811  4280 raft_consensus.cc:740] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.001555  4280 consensus_queue.cc:260] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.003801  4280 ts_tablet_manager.cc:1434] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.067s	user 0.005s	sys 0.001s
I20260709 12:20:48.004851  4280 tablet_bootstrap.cc:492] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:47.991554  4272 raft_consensus.cc:740] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.008816  4272 consensus_queue.cc:260] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:48.012408  4272 ts_tablet_manager.cc:1434] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.004s
I20260709 12:20:48.014351  4277 tablet_bootstrap.cc:654] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.014770  4272 tablet_bootstrap.cc:492] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:48.026201  4523 consensus_queue.cc:1048] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:48.039865  4273 tablet_bootstrap.cc:492] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:48.040481  4273 ts_tablet_manager.cc:1403] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.054s	user 0.019s	sys 0.008s
I20260709 12:20:48.023298  3901 catalog_manager.cc:5697] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:48.047178  4273 raft_consensus.cc:359] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.047937  4273 raft_consensus.cc:385] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.048223  4273 raft_consensus.cc:740] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.051203  4273 consensus_queue.cc:260] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.052139  4597 raft_consensus.cc:493] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.052691  4597 raft_consensus.cc:515] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.050709  3903 catalog_manager.cc:5697] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.056012  4272 tablet_bootstrap.cc:654] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.056643  4273 ts_tablet_manager.cc:1434] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 12:20:48.058696  4273 tablet_bootstrap.cc:492] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:48.061753  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389645cf278940c6803751b614ba1dfa" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:48.078125  4280 tablet_bootstrap.cc:654] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.069480  4223 raft_consensus.cc:2468] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:48.094333  3998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389645cf278940c6803751b614ba1dfa" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:48.101717  3998 raft_consensus.cc:2468] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:48.091813  4597 leader_election.cc:290] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:48.104080  4273 tablet_bootstrap.cc:654] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.100857  4046 leader_election.cc:304] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:48.111622  4525 raft_consensus.cc:2804] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.112046  4525 raft_consensus.cc:493] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.112360  4525 raft_consensus.cc:3060] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.124720  4597 consensus_queue.cc:1048] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:20:48.134493  4277 tablet_bootstrap.cc:492] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:48.135134  4277 ts_tablet_manager.cc:1403] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.153s	user 0.026s	sys 0.009s
I20260709 12:20:48.135902  4273 tablet_bootstrap.cc:492] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:48.136431  4273 ts_tablet_manager.cc:1403] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.078s	user 0.013s	sys 0.005s
I20260709 12:20:48.137974  4277 raft_consensus.cc:359] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.138758  4277 raft_consensus.cc:385] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.139016  4277 raft_consensus.cc:740] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.139187  4273 raft_consensus.cc:359] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.139880  4273 raft_consensus.cc:385] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.140856  4280 tablet_bootstrap.cc:492] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:48.140158  4273 raft_consensus.cc:740] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.141360  4280 ts_tablet_manager.cc:1403] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.137s	user 0.016s	sys 0.006s
I20260709 12:20:48.142006  4273 consensus_queue.cc:260] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.144289  4273 ts_tablet_manager.cc:1434] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:20:48.144085  4280 raft_consensus.cc:359] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.144779  4280 raft_consensus.cc:385] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.145066  4280 raft_consensus.cc:740] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.145920  4280 consensus_queue.cc:260] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.147603  4280 ts_tablet_manager.cc:1434] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:20:48.148372  4280 tablet_bootstrap.cc:492] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:48.150391  4273 tablet_bootstrap.cc:492] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:48.155717  4280 tablet_bootstrap.cc:654] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.156471  4277 consensus_queue.cc:260] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.160405  4277 ts_tablet_manager.cc:1434] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:20:48.161446  4277 tablet_bootstrap.cc:492] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:48.158527  4273 tablet_bootstrap.cc:654] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.168147  4272 tablet_bootstrap.cc:492] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:48.170778  4525 raft_consensus.cc:515] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.174266  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389645cf278940c6803751b614ba1dfa" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:48.174954  4223 raft_consensus.cc:3060] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.170722  4272 ts_tablet_manager.cc:1403] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.156s	user 0.008s	sys 0.012s
I20260709 12:20:48.178442  4525 leader_election.cc:290] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:48.179937  4277 tablet_bootstrap.cc:654] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.184229  3998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389645cf278940c6803751b614ba1dfa" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:48.184926  3998 raft_consensus.cc:3060] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.194494  4272 raft_consensus.cc:359] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:48.195114  3998 raft_consensus.cc:2468] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:48.196012  4223 raft_consensus.cc:2468] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:48.198958  4273 tablet_bootstrap.cc:492] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:48.199378  4280 tablet_bootstrap.cc:492] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:48.200851  4277 tablet_bootstrap.cc:492] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:48.201982  4272 raft_consensus.cc:385] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.202759  4273 ts_tablet_manager.cc:1403] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.053s	user 0.007s	sys 0.014s
I20260709 12:20:48.202773  4280 ts_tablet_manager.cc:1403] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.003s
I20260709 12:20:48.202833  4277 ts_tablet_manager.cc:1403] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.042s	user 0.006s	sys 0.007s
I20260709 12:20:48.207284  4046 leader_election.cc:304] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:48.207682  4280 raft_consensus.cc:359] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.208379  4525 raft_consensus.cc:2804] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:48.208421  4273 raft_consensus.cc:359] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.209232  4525 raft_consensus.cc:697] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:48.209424  4273 raft_consensus.cc:385] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.209815  4277 raft_consensus.cc:359] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.209781  4273 raft_consensus.cc:740] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.210729  4277 raft_consensus.cc:385] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.211107  4277 raft_consensus.cc:740] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.210399  4597 raft_consensus.cc:493] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.211537  4273 consensus_queue.cc:260] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.211714  4597 raft_consensus.cc:515] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.211861  4277 consensus_queue.cc:260] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.213997  4273 ts_tablet_manager.cc:1434] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:20:48.214471  4277 ts_tablet_manager.cc:1434] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:20:48.215384  4273 tablet_bootstrap.cc:492] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:48.208403  4280 raft_consensus.cc:385] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.219345  4280 raft_consensus.cc:740] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.220568  4532 raft_consensus.cc:493] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.221253  4532 raft_consensus.cc:515] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.220634  4280 consensus_queue.cc:260] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.223452  4532 leader_election.cc:290] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:48.216089  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:48.228253  4045 leader_election.cc:343] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.203226  4272 raft_consensus.cc:740] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.211261  4525 consensus_queue.cc:237] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.229450  4272 consensus_queue.cc:260] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:48.217106  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:48.214382  4597 leader_election.cc:290] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:48.233470  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435b4d42ac35468f8a95973a14a9d6e3" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:48.236385  4046 leader_election.cc:343] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.236984  4046 leader_election.cc:304] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
W20260709 12:20:48.238318  4046 leader_election.cc:343] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.240302  4629 raft_consensus.cc:493] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.240862  4629 raft_consensus.cc:515] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.243109  4629 leader_election.cc:290] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:48.225015  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435b4d42ac35468f8a95973a14a9d6e3" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:48.246207  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778cd7f085494ba1842e5f758d7f9828" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:48.239382  4597 raft_consensus.cc:2749] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.246167  4153 raft_consensus.cc:2468] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:48.245357  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778cd7f085494ba1842e5f758d7f9828" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:48.250130  4568 raft_consensus.cc:493] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.250635  4568 raft_consensus.cc:515] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:48.253312  4568 leader_election.cc:290] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:48.254688  4272 ts_tablet_manager.cc:1434] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.076s	user 0.006s	sys 0.000s
I20260709 12:20:48.256026  4272 tablet_bootstrap.cc:492] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:48.256837  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:48.257674  3971 leader_election.cc:343] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:48.258543  3971 leader_election.cc:343] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.257753  4153 raft_consensus.cc:2393] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 41009fde51f045c49be6461b76a27543 in this term.
I20260709 12:20:48.259159  3971 leader_election.cc:304] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:48.250809  4045 leader_election.cc:304] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:48.261619  4597 raft_consensus.cc:2804] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.262055  4597 raft_consensus.cc:493] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.262221  4629 raft_consensus.cc:2749] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.262436  4597 raft_consensus.cc:3060] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.238804  4280 ts_tablet_manager.cc:1434] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.002s
I20260709 12:20:48.264535  4280 tablet_bootstrap.cc:492] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:48.254565  4002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:48.215508  4277 tablet_bootstrap.cc:492] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:48.274726  4194 leader_election.cc:304] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:48.276090  4568 raft_consensus.cc:3060] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.288822  4277 tablet_bootstrap.cc:654] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.272171  4280 tablet_bootstrap.cc:654] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.308617  4525 raft_consensus.cc:493] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.310644  4525 raft_consensus.cc:515] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.313066  4525 leader_election.cc:290] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:48.313966  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc6c1656d03459babbca1538b0ae0ef" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:48.317684  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc6c1656d03459babbca1538b0ae0ef" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:48.319200  4227 raft_consensus.cc:2393] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 25902f00fb5d41679245f23dd19654aa in current term 1: Already voted for candidate 6994df41341e478d89f7c105f550ff93 in this term.
I20260709 12:20:48.320693  4046 leader_election.cc:304] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:48.322214  4525 raft_consensus.cc:3060] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.324213  4568 raft_consensus.cc:2749] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.333916  4272 tablet_bootstrap.cc:654] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.345155  4280 tablet_bootstrap.cc:492] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:48.346882  4583 raft_consensus.cc:493] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.347851  4277 tablet_bootstrap.cc:492] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:48.347503  4583 raft_consensus.cc:515] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:48.349893  4583 leader_election.cc:290] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:48.350391  4273 tablet_bootstrap.cc:654] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.350768  4277 ts_tablet_manager.cc:1403] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.135s	user 0.013s	sys 0.011s
I20260709 12:20:48.352697  4002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:48.353328  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:48.353526  4280 ts_tablet_manager.cc:1403] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.089s	user 0.028s	sys 0.013s
I20260709 12:20:48.354714  4597 raft_consensus.cc:515] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
W20260709 12:20:48.355844  4120 leader_election.cc:343] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:48.356913  4120 leader_election.cc:343] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.357717  4597 leader_election.cc:290] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:48.357828  4280 raft_consensus.cc:359] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.358522  4525 raft_consensus.cc:2749] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.358831  4280 raft_consensus.cc:385] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.358904  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435b4d42ac35468f8a95973a14a9d6e3" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:48.359334  4280 raft_consensus.cc:740] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
W20260709 12:20:48.360493  4046 leader_election.cc:343] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.364578  4272 tablet_bootstrap.cc:492] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:48.361086  4280 consensus_queue.cc:260] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.361455  4277 raft_consensus.cc:359] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.372674  4273 tablet_bootstrap.cc:492] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:48.361544  4120 leader_election.cc:304] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:48.374511  4272 ts_tablet_manager.cc:1403] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.119s	user 0.011s	sys 0.022s
I20260709 12:20:48.384243  4272 raft_consensus.cc:359] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.385003  4272 raft_consensus.cc:385] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.385298  4272 raft_consensus.cc:740] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.376294  4277 raft_consensus.cc:385] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.386155  4272 consensus_queue.cc:260] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.376684  4273 ts_tablet_manager.cc:1403] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.162s	user 0.013s	sys 0.001s
I20260709 12:20:48.388005  4525 raft_consensus.cc:493] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.388818  4272 ts_tablet_manager.cc:1434] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:20:48.388568  4525 raft_consensus.cc:515] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.390321  4273 raft_consensus.cc:359] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.378026  4593 raft_consensus.cc:2749] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.390902  4525 leader_election.cc:290] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:48.386263  4277 raft_consensus.cc:740] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.393198  4151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f92fa80bd34e60924d5c77f79df421" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:48.393971  4151 raft_consensus.cc:2468] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:48.393749  4277 consensus_queue.cc:260] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.395583  4277 ts_tablet_manager.cc:1434] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.001s
I20260709 12:20:48.396613  4348 raft_consensus.cc:493] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.397408  4045 leader_election.cc:304] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:48.376087  4280 ts_tablet_manager.cc:1434] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:20:48.398612  4272 tablet_bootstrap.cc:492] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:48.378818  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435b4d42ac35468f8a95973a14a9d6e3" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:48.400086  4532 raft_consensus.cc:2804] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.400224  4153 raft_consensus.cc:3060] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.400540  4532 raft_consensus.cc:493] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.397192  4348 raft_consensus.cc:515] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:48.400852  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:48.402825  4348 leader_election.cc:290] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:48.391106  4273 raft_consensus.cc:385] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.374436  3903 catalog_manager.cc:5697] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:48.404264  4273 raft_consensus.cc:740] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.396662  4277 tablet_bootstrap.cc:492] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:48.405366  4002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f92fa80bd34e60924d5c77f79df421" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:48.406172  4002 raft_consensus.cc:2468] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:48.407503  4273 consensus_queue.cc:260] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.410948  4272 tablet_bootstrap.cc:654] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.433871  4532 raft_consensus.cc:3060] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.412644  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:48.441938  4272 tablet_bootstrap.cc:492] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:48.413802  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:48.442770  4223 raft_consensus.cc:2468] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:48.443598  4561 consensus_queue.cc:1048] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:48.444257  4272 ts_tablet_manager.cc:1403] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.014s
I20260709 12:20:48.448192  4272 raft_consensus.cc:359] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.448974  4272 raft_consensus.cc:385] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.449290  4272 raft_consensus.cc:740] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.450146  4078 raft_consensus.cc:2468] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:48.411247  4153 raft_consensus.cc:2468] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:48.450138  4272 consensus_queue.cc:260] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.453126  4045 leader_election.cc:304] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:48.454945  3971 leader_election.cc:304] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:48.455844  4525 raft_consensus.cc:2804] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:48.457834  4561 raft_consensus.cc:2804] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.458729  4561 raft_consensus.cc:493] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.459229  4561 raft_consensus.cc:3060] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.455278  4272 ts_tablet_manager.cc:1434] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:20:48.457351  4525 raft_consensus.cc:697] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:48.462782  4277 tablet_bootstrap.cc:654] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.462908  4525 consensus_queue.cc:237] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.466109  4532 raft_consensus.cc:515] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.468437  4532 leader_election.cc:290] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:48.470930  4273 ts_tablet_manager.cc:1434] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.001s
I20260709 12:20:48.472908  4273 tablet_bootstrap.cc:492] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:48.480412  4272 tablet_bootstrap.cc:492] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:48.480569  4280 tablet_bootstrap.cc:492] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:48.488194  4272 tablet_bootstrap.cc:654] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.494930  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f92fa80bd34e60924d5c77f79df421" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:48.495738  4148 raft_consensus.cc:3060] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.519176  4277 tablet_bootstrap.cc:492] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:48.519850  4277 ts_tablet_manager.cc:1403] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.123s	user 0.013s	sys 0.012s
I20260709 12:20:48.527114  4148 raft_consensus.cc:2468] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:48.514425  4561 raft_consensus.cc:515] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.508985  4629 consensus_queue.cc:1048] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:20:48.530581  4002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35f92fa80bd34e60924d5c77f79df421" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:48.531497  4002 raft_consensus.cc:3060] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.532884  4045 leader_election.cc:304] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
W20260709 12:20:48.546079  3970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 44063.4us
I20260709 12:20:48.548619  3903 catalog_manager.cc:5697] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.545681  4593 raft_consensus.cc:493] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.555650  4593 raft_consensus.cc:515] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.558678  4593 leader_election.cc:290] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:48.559149  4561 leader_election.cc:290] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:48.560007  4277 raft_consensus.cc:359] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.560772  4277 raft_consensus.cc:385] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.561093  4280 tablet_bootstrap.cc:654] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.561225  4277 raft_consensus.cc:740] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.562626  4277 consensus_queue.cc:260] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.565136  4277 ts_tablet_manager.cc:1434] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.002s
I20260709 12:20:48.550211  4523 raft_consensus.cc:2804] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:48.568243  4523 raft_consensus.cc:697] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:48.569199  4523 consensus_queue.cc:237] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.570958  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2adcdb890e0483095cb9923dce850a1" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:48.575874  4277 tablet_bootstrap.cc:492] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
W20260709 12:20:48.576731  4120 leader_election.cc:343] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.583999  3998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2adcdb890e0483095cb9923dce850a1" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:48.591020  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:48.591938  4079 raft_consensus.cc:3060] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:48.594547  4120 leader_election.cc:343] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.595201  4120 leader_election.cc:304] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:48.600397  4272 tablet_bootstrap.cc:492] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:48.601250  4272 ts_tablet_manager.cc:1403] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.121s	user 0.017s	sys 0.000s
I20260709 12:20:48.550114  4581 consensus_queue.cc:1048] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 12:20:48.615262  4079 raft_consensus.cc:2468] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:48.602130  4445 raft_consensus.cc:2749] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.619735  4273 tablet_bootstrap.cc:654] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.618088  3971 leader_election.cc:304] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:48.632205  4567 raft_consensus.cc:2804] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:48.624315  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:48.625654  4568 raft_consensus.cc:493] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.657527  4568 raft_consensus.cc:515] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.660238  4568 leader_election.cc:290] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:48.660928  4227 raft_consensus.cc:3060] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.664547  4581 raft_consensus.cc:493] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.665278  4581 raft_consensus.cc:515] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.666344  4567 raft_consensus.cc:697] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:48.667809  4581 leader_election.cc:290] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:48.667553  4567 consensus_queue.cc:237] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:48.649442  4277 tablet_bootstrap.cc:654] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.639771  4656 raft_consensus.cc:493] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.683020  4656 raft_consensus.cc:515] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:48.691733  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:41077, user_credentials={real_user=slave}} blocked reactor thread for 94746.1us
I20260709 12:20:48.685698  4656 leader_election.cc:290] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:48.732959  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fda924a0568433fa70ab72d4a0c3968" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:48.733815  4153 raft_consensus.cc:2468] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:48.733942  3998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778cd7f085494ba1842e5f758d7f9828" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:48.734666  3998 raft_consensus.cc:2468] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:48.741178  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:48.743867  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778cd7f085494ba1842e5f758d7f9828" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:48.744503  4078 raft_consensus.cc:2468] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:48.744554  4272 raft_consensus.cc:359] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.752308  4272 raft_consensus.cc:385] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:20:48.754083  4120 leader_election.cc:343] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.752712  4272 raft_consensus.cc:740] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.762643  4272 consensus_queue.cc:260] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.731123  4581 consensus_queue.cc:1048] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:20:48.787551  4272 ts_tablet_manager.cc:1434] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.186s	user 0.007s	sys 0.002s
I20260709 12:20:48.816088  4193 leader_election.cc:304] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:48.817884  4272 tablet_bootstrap.cc:492] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:48.840576  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:48.841346  4277 tablet_bootstrap.cc:492] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:48.842006  4277 ts_tablet_manager.cc:1403] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.267s	user 0.033s	sys 0.011s
I20260709 12:20:48.844432  4227 raft_consensus.cc:2468] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:48.822197  4657 raft_consensus.cc:2804] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.869215  4657 raft_consensus.cc:493] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.869803  4657 raft_consensus.cc:3060] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.880226  4277 raft_consensus.cc:359] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:48.880919  4277 raft_consensus.cc:385] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.825124  4002 raft_consensus.cc:2468] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:48.888314  4273 tablet_bootstrap.cc:492] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:48.889117  4273 ts_tablet_manager.cc:1403] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.417s	user 0.019s	sys 0.015s
I20260709 12:20:48.889474  4194 leader_election.cc:304] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:48.890641  4659 raft_consensus.cc:493] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.891222  4659 raft_consensus.cc:515] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:48.892589  4655 raft_consensus.cc:2804] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.893046  4655 raft_consensus.cc:493] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.893391  4655 raft_consensus.cc:3060] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.881138  4277 raft_consensus.cc:740] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.867904  4524 consensus_queue.cc:1048] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:20:48.900300  4280 tablet_bootstrap.cc:492] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:48.900950  4280 ts_tablet_manager.cc:1403] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.421s	user 0.042s	sys 0.019s
I20260709 12:20:48.903995  4277 consensus_queue.cc:260] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:48.914019  4277 ts_tablet_manager.cc:1434] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.002s
I20260709 12:20:48.916429  4277 tablet_bootstrap.cc:492] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:48.899223  4659 leader_election.cc:290] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:48.921533  4003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fda924a0568433fa70ab72d4a0c3968" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:48.931207  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:48.932103  4080 raft_consensus.cc:2468] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 0.
I20260709 12:20:48.934870  4277 tablet_bootstrap.cc:654] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.935060  4120 leader_election.cc:304] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:48.936060  4445 raft_consensus.cc:2804] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.936482  4445 raft_consensus.cc:493] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.936941  4445 raft_consensus.cc:3060] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.938436  4273 raft_consensus.cc:359] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.939308  4273 raft_consensus.cc:385] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.939647  4273 raft_consensus.cc:740] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.940439  4273 consensus_queue.cc:260] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.943045  4273 ts_tablet_manager.cc:1434] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.054s	user 0.004s	sys 0.004s
I20260709 12:20:48.949124  4280 raft_consensus.cc:359] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.959271  4280 raft_consensus.cc:385] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.959647  4280 raft_consensus.cc:740] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.960361  4280 consensus_queue.cc:260] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:48.957343  3899 catalog_manager.cc:5697] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.967964  4273 tablet_bootstrap.cc:492] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
W20260709 12:20:48.970386  4120 leader_election.cc:343] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.970963  4120 leader_election.cc:304] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:48.971763  4280 ts_tablet_manager.cc:1434] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.070s	user 0.004s	sys 0.004s
I20260709 12:20:48.972949  4280 tablet_bootstrap.cc:492] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:48.980791  3902 catalog_manager.cc:5697] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.978080  4581 raft_consensus.cc:2749] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:49.004454  4524 consensus_queue.cc:1048] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:20:49.027674  4655 raft_consensus.cc:515] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:49.039332  4657 raft_consensus.cc:515] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.044544  4670 raft_consensus.cc:493] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.045110  4670 raft_consensus.cc:515] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:49.047533  4670 leader_election.cc:290] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:49.044416  4272 tablet_bootstrap.cc:654] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:49.051141  4046 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 78231us
I20260709 12:20:49.052738  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2adcdb890e0483095cb9923dce850a1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:49.053506  4079 raft_consensus.cc:2468] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:49.056919  3971 leader_election.cc:304] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:49.093583  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2adcdb890e0483095cb9923dce850a1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:49.094491  4154 raft_consensus.cc:2468] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:49.095716  4280 tablet_bootstrap.cc:654] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.096853  4655 leader_election.cc:290] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:49.097689  4657 leader_election.cc:290] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:49.106611  4653 raft_consensus.cc:2804] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:49.107194  4653 raft_consensus.cc:493] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.107550  4653 raft_consensus.cc:3060] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.029323  4445 raft_consensus.cc:515] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:49.054961  4222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:49.111930  4445 leader_election.cc:290] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:49.128980  4680 consensus_queue.cc:1048] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
W20260709 12:20:49.134371  4120 leader_election.cc:343] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.143208  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:49.144019  4080 raft_consensus.cc:3060] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.148701  4273 tablet_bootstrap.cc:654] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.154371  4228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:49.152792  4080 raft_consensus.cc:2468] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:49.163825  4277 tablet_bootstrap.cc:492] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:49.165511  4668 raft_consensus.cc:493] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:49.154225  4673 log.cc:927] Time spent T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 12:20:49.178153  4120 leader_election.cc:343] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.198575  4120 leader_election.cc:304] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:49.195896  4277 ts_tablet_manager.cc:1403] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.280s	user 0.024s	sys 0.002s
I20260709 12:20:49.231127  4001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778cd7f085494ba1842e5f758d7f9828" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:49.232106  4001 raft_consensus.cc:3060] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.195875  4668 raft_consensus.cc:515] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:49.244546  4668 leader_election.cc:290] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:49.251230  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fda924a0568433fa70ab72d4a0c3968" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:49.252172  4148 raft_consensus.cc:3060] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.205612  4445 raft_consensus.cc:2804] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:49.255692  4445 raft_consensus.cc:697] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
W20260709 12:20:49.259109  4194 leader_election.cc:343] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.260129  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778cd7f085494ba1842e5f758d7f9828" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:49.260926  4078 raft_consensus.cc:3060] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.271518  4277 raft_consensus.cc:359] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:49.272464  4277 raft_consensus.cc:385] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.272823  4277 raft_consensus.cc:740] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.256945  4445 consensus_queue.cc:237] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
W20260709 12:20:49.276080  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 53891.4us
I20260709 12:20:49.281373  4078 raft_consensus.cc:2468] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:49.277951  4277 consensus_queue.cc:260] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:49.292361  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435b4d42ac35468f8a95973a14a9d6e3" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:49.260099  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fda924a0568433fa70ab72d4a0c3968" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:49.293999  4272 tablet_bootstrap.cc:492] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:49.294713  4272 ts_tablet_manager.cc:1403] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.506s	user 0.026s	sys 0.031s
I20260709 12:20:49.283911  4567 consensus_queue.cc:1048] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:20:49.309250  4228 raft_consensus.cc:3060] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.310948  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435b4d42ac35468f8a95973a14a9d6e3" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:49.315814  4277 ts_tablet_manager.cc:1434] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.115s	user 0.009s	sys 0.000s
I20260709 12:20:49.325289  4661 consensus_queue.cc:1048] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:20:49.338639  4148 raft_consensus.cc:2468] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:49.322840  4583 raft_consensus.cc:493] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.311161  4597 consensus_queue.cc:1048] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:20:49.346771  4583 raft_consensus.cc:515] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.348120  4678 consensus_queue.cc:1048] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:49.351665  4679 consensus_queue.cc:1048] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:49.345418  4193 leader_election.cc:304] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
W20260709 12:20:49.393714  4193 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 48409.1us
I20260709 12:20:49.394238  4568 raft_consensus.cc:2804] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:49.394869  4568 raft_consensus.cc:697] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:49.356441  4658 raft_consensus.cc:493] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.395802  4568 consensus_queue.cc:237] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.396135  4658 raft_consensus.cc:515] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:49.398448  4653 raft_consensus.cc:515] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:49.414992  4280 tablet_bootstrap.cc:492] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:49.415858  4280 ts_tablet_manager.cc:1403] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.443s	user 0.020s	sys 0.017s
I20260709 12:20:49.398711  4658 leader_election.cc:290] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:49.435714  4277 tablet_bootstrap.cc:492] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93: Bootstrap starting.
I20260709 12:20:49.447206  4272 raft_consensus.cc:359] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.454309  4272 raft_consensus.cc:385] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.454633  4272 raft_consensus.cc:740] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.455330  4272 consensus_queue.cc:260] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.426877  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:49.484931  4001 raft_consensus.cc:2468] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
W20260709 12:20:49.491170  4686 log.cc:927] Time spent T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Append to log took a long time: real 0.085s	user 0.003s	sys 0.000s
I20260709 12:20:49.492105  4653 leader_election.cc:290] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:49.494052  4078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
W20260709 12:20:49.474278  4194 leader_election.cc:343] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.496608  4194 leader_election.cc:304] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:49.498066  4684 raft_consensus.cc:2804] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:49.498638  4684 raft_consensus.cc:697] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:49.502048  4272 ts_tablet_manager.cc:1434] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.207s	user 0.010s	sys 0.001s
I20260709 12:20:49.511824  4684 consensus_queue.cc:237] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:49.514592  4280 raft_consensus.cc:359] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.515413  4280 raft_consensus.cc:385] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.515679  4280 raft_consensus.cc:740] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.516347  4280 consensus_queue.cc:260] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.524940  4583 leader_election.cc:290] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:49.565085  4280 ts_tablet_manager.cc:1434] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.149s	user 0.004s	sys 0.004s
I20260709 12:20:49.566531  4280 tablet_bootstrap.cc:492] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:49.567356  4194 leader_election.cc:304] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
W20260709 12:20:49.635221  4194 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 125477us
I20260709 12:20:49.636232  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2adcdb890e0483095cb9923dce850a1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:49.637012  4153 raft_consensus.cc:3060] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.649585  4277 tablet_bootstrap.cc:654] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.683495  4280 tablet_bootstrap.cc:654] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.675590  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:49.700414  4120 leader_election.cc:343] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:49.704107  4120 leader_election.cc:343] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.708511  4668 raft_consensus.cc:2749] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:49.715329  4273 tablet_bootstrap.cc:492] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:49.715914  4273 ts_tablet_manager.cc:1403] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.748s	user 0.019s	sys 0.028s
I20260709 12:20:49.718147  4272 tablet_bootstrap.cc:492] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:49.732640  4280 tablet_bootstrap.cc:492] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:49.741097  4280 ts_tablet_manager.cc:1403] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.009s
I20260709 12:20:49.740921  4273 raft_consensus.cc:359] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:49.740198  4120 leader_election.cc:343] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.741791  4273 raft_consensus.cc:385] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.742117  4273 raft_consensus.cc:740] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.742159  4120 leader_election.cc:304] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:49.747763  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2adcdb890e0483095cb9923dce850a1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:49.748544  4080 raft_consensus.cc:3060] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.745281  4669 raft_consensus.cc:2749] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:49.768359  4273 consensus_queue.cc:260] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:49.787766  4273 ts_tablet_manager.cc:1434] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.072s	user 0.002s	sys 0.006s
I20260709 12:20:49.788784  4273 tablet_bootstrap.cc:492] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:49.767534  4153 raft_consensus.cc:2468] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:49.802800  4273 tablet_bootstrap.cc:654] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.787194  4001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:49.800468  4675 consensus_queue.cc:1048] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:20:49.805227  3970 leader_election.cc:304] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:49.809777  4653 raft_consensus.cc:2804] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:49.824149  4653 raft_consensus.cc:697] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:49.825122  4653 consensus_queue.cc:237] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:49.826408  4280 raft_consensus.cc:359] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.827528  4280 raft_consensus.cc:385] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.827894  4280 raft_consensus.cc:740] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.828840  4280 consensus_queue.cc:260] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.831786  4080 raft_consensus.cc:2468] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:49.832050  4677 raft_consensus.cc:493] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.835012  4680 raft_consensus.cc:493] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.835100  4677 raft_consensus.cc:515] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:49.835701  4680 raft_consensus.cc:515] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:49.872040  4709 log.cc:927] Time spent T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:20:49.876220  4272 tablet_bootstrap.cc:654] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.892503  3999 raft_consensus.cc:3060] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.933357  4277 tablet_bootstrap.cc:492] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93: No bootstrap required, opened a new log
I20260709 12:20:49.934070  4277 ts_tablet_manager.cc:1403] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93: Time spent bootstrapping tablet: real 0.499s	user 0.014s	sys 0.023s
I20260709 12:20:49.932826  4656 raft_consensus.cc:493] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.946432  4656 raft_consensus.cc:515] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:49.952056  4120 consensus_peers.cc:597] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 -> Peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Couldn't send request to peer 72940e4e52944adb81608e286ad58a59. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:49.964977  4280 ts_tablet_manager.cc:1434] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.223s	user 0.015s	sys 0.006s
I20260709 12:20:49.966815  4280 tablet_bootstrap.cc:492] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
W20260709 12:20:50.073946  3970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 92340.4us
I20260709 12:20:50.083556  4677 leader_election.cc:290] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:50.100831  4656 leader_election.cc:290] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:50.102072  4277 raft_consensus.cc:359] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:50.103181  4277 raft_consensus.cc:385] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.103580  4277 raft_consensus.cc:740] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.104411  4277 consensus_queue.cc:260] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:50.116928  3903 catalog_manager.cc:5697] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:50.122752  3903 catalog_manager.cc:5697] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:50.097921  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9233488a2b6745f8a012d9edc348333a" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:50.139750  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:50.149825  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:50.150681  4154 raft_consensus.cc:2468] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:50.123075  4684 consensus_queue.cc:1048] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:20:50.160806  4568 consensus_queue.cc:1048] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:50.166505  4715 log.cc:927] Time spent T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Append to log took a long time: real 0.064s	user 0.003s	sys 0.000s
I20260709 12:20:50.168176  4680 leader_election.cc:290] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:50.170257  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:50.171105  4148 raft_consensus.cc:2468] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:50.162062  4193 leader_election.cc:304] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:50.143481  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
W20260709 12:20:50.183868  3971 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 394344us
I20260709 12:20:50.195890  3970 leader_election.cc:304] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
W20260709 12:20:50.199870  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 56378.3us
I20260709 12:20:50.182569  4656 raft_consensus.cc:2804] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:50.204527  4656 raft_consensus.cc:493] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:50.211858  4277 ts_tablet_manager.cc:1434] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93: Time spent starting tablet: real 0.277s	user 0.011s	sys 0.008s
I20260709 12:20:50.195955  4004 raft_consensus.cc:2468] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:50.205039  4656 raft_consensus.cc:3060] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.213589  4272 tablet_bootstrap.cc:492] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:50.217039  4272 ts_tablet_manager.cc:1403] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.499s	user 0.014s	sys 0.017s
I20260709 12:20:50.231716  4280 tablet_bootstrap.cc:654] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:50.249656  4272 raft_consensus.cc:359] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:50.250777  4272 raft_consensus.cc:385] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.251137  4272 raft_consensus.cc:740] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.228497  4706 raft_consensus.cc:2804] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:20:50.253662  4193 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 70100.7us
I20260709 12:20:50.217106  4682 consensus_queue.cc:1048] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 12:20:50.258991  4273 tablet_bootstrap.cc:492] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:50.259656  4273 ts_tablet_manager.cc:1403] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.471s	user 0.035s	sys 0.036s
I20260709 12:20:50.262923  4273 raft_consensus.cc:359] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
W20260709 12:20:50.255247  4046 leader_election.cc:343] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.264159  4273 raft_consensus.cc:385] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.264514  4273 raft_consensus.cc:740] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.268133  4273 consensus_queue.cc:260] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:50.276971  4273 ts_tablet_manager.cc:1434] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:20:50.278157  4273 tablet_bootstrap.cc:492] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:50.285046  4273 tablet_bootstrap.cc:654] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:50.290874  4668 consensus_queue.cc:1048] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:50.293426  4706 raft_consensus.cc:493] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:50.294142  4706 raft_consensus.cc:3060] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.328567  3998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9233488a2b6745f8a012d9edc348333a" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:50.344918  4272 consensus_queue.cc:260] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:50.345993  4656 raft_consensus.cc:515] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:50.378489  4272 ts_tablet_manager.cc:1434] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.161s	user 0.012s	sys 0.000s
I20260709 12:20:50.389060  4272 tablet_bootstrap.cc:492] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:50.456403  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:50.470360  4272 tablet_bootstrap.cc:654] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:50.444175  4656 leader_election.cc:290] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
W20260709 12:20:50.506004  4046 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:41077, user_credentials={real_user=slave}} blocked reactor thread for 69022.2us
W20260709 12:20:50.515352  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 86858.5us
I20260709 12:20:50.442322  4561 consensus_queue.cc:1048] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
W20260709 12:20:50.530129  4046 leader_election.cc:343] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.533123  4046 leader_election.cc:304] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:50.542862  4153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
W20260709 12:20:50.540432  4120 leader_election.cc:343] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.543653  4153 raft_consensus.cc:3060] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.544104  4120 leader_election.cc:304] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:50.549845  4593 raft_consensus.cc:2749] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:50.567754  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:50.568908  4224 raft_consensus.cc:2393] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41009fde51f045c49be6461b76a27543 in current term 1: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:50.576356  4280 tablet_bootstrap.cc:492] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:50.577000  4280 ts_tablet_manager.cc:1403] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.610s	user 0.009s	sys 0.020s
W20260709 12:20:50.586705  4732 log.cc:927] Time spent T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Append to log took a long time: real 0.068s	user 0.001s	sys 0.003s
I20260709 12:20:50.590062  4153 raft_consensus.cc:2468] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:50.607666  4273 tablet_bootstrap.cc:492] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:50.608532  4273 ts_tablet_manager.cc:1403] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.331s	user 0.016s	sys 0.019s
I20260709 12:20:50.620287  4675 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.621004  4675 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:50.631181  4280 raft_consensus.cc:359] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:50.632179  4280 raft_consensus.cc:385] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.632501  4280 raft_consensus.cc:740] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.625789  4743 raft_consensus.cc:2749] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:50.682366  4675 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:50.634877  4682 raft_consensus.cc:493] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.683059  4273 raft_consensus.cc:359] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:50.687280  4273 raft_consensus.cc:385] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.683713  4280 consensus_queue.cc:260] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:50.687960  4273 raft_consensus.cc:740] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.683578  4682 raft_consensus.cc:515] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:50.696530  4682 leader_election.cc:290] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:50.706192  4001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:50.701553  4273 consensus_queue.cc:260] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:50.710112  4120 leader_election.cc:343] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.720734  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:50.746503  4193 leader_election.cc:304] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:50.747486  4682 raft_consensus.cc:2804] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:50.748039  4682 raft_consensus.cc:697] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:50.750204  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:50.751000  4077 raft_consensus.cc:2468] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:50.749022  4682 consensus_queue.cc:237] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:50.752429  4120 leader_election.cc:343] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.756857  4273 ts_tablet_manager.cc:1434] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.148s	user 0.009s	sys 0.000s
I20260709 12:20:50.760401  4120 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:50.761230  4273 tablet_bootstrap.cc:492] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa: Bootstrap starting.
I20260709 12:20:50.755190  4280 ts_tablet_manager.cc:1434] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.178s	user 0.010s	sys 0.003s
I20260709 12:20:50.763821  4280 tablet_bootstrap.cc:492] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:50.752447  4706 raft_consensus.cc:515] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:50.766832  4004 raft_consensus.cc:2393] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 41009fde51f045c49be6461b76a27543 in this term.
I20260709 12:20:50.767763  4706 leader_election.cc:290] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:50.752125  4659 raft_consensus.cc:493] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.768745  4659 raft_consensus.cc:515] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:50.770283  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:50.771066  4659 leader_election.cc:290] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:50.759783  4532 raft_consensus.cc:493] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.778278  4532 raft_consensus.cc:515] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:50.778954  4748 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.779553  4748 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:50.780859  4532 leader_election.cc:290] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:50.773171  4146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:50.782055  4146 raft_consensus.cc:2468] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:50.790143  4581 raft_consensus.cc:2749] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:50.792981  4744 raft_consensus.cc:493] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:50.815548  4194 leader_election.cc:304] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:50.825124  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:50.825997  4147 raft_consensus.cc:2468] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:50.827951  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:50.828266  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:50.838099  4272 tablet_bootstrap.cc:492] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:50.839942  4272 ts_tablet_manager.cc:1403] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.451s	user 0.022s	sys 0.018s
I20260709 12:20:50.838912  4045 leader_election.cc:304] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:50.828812  4225 raft_consensus.cc:2468] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:50.862356  4076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:50.869153  4280 tablet_bootstrap.cc:654] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:50.847599  4524 raft_consensus.cc:2804] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:50.813889  4744 raft_consensus.cc:515] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:50.872591  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:50.879846  4079 raft_consensus.cc:2468] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:50.854629  4748 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:50.883785  3971 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:20:50.874305  4524 raft_consensus.cc:493] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:50.890172  4524 raft_consensus.cc:3060] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.858955  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:50.867640  4684 raft_consensus.cc:2804] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:50.925779  4273 tablet_bootstrap.cc:654] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:50.960655  4721 raft_consensus.cc:2804] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:50.961290  4721 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:50.961771  4721 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:50.969444  3971 leader_election.cc:343] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:50.955600  4757 log.cc:927] Time spent T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:20:50.976938  4581 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:50.981375  4744 leader_election.cc:290] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:50.997970  4272 raft_consensus.cc:359] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
W20260709 12:20:51.007426  4193 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 185115us
I20260709 12:20:51.001668  4272 raft_consensus.cc:385] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.008424  4272 raft_consensus.cc:740] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.009393  4272 consensus_queue.cc:260] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:51.011492  4272 ts_tablet_manager.cc:1434] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.170s	user 0.000s	sys 0.007s
I20260709 12:20:51.012557  4272 tablet_bootstrap.cc:492] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:50.993970  4581 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:51.016060  4581 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:51.021944  4273 tablet_bootstrap.cc:492] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa: No bootstrap required, opened a new log
I20260709 12:20:51.022604  4273 ts_tablet_manager.cc:1403] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa: Time spent bootstrapping tablet: real 0.262s	user 0.017s	sys 0.007s
I20260709 12:20:51.023128  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1c0613280b411f98f4807034b0a1cb" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:51.024127  4154 raft_consensus.cc:2393] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 41009fde51f045c49be6461b76a27543 in current term 1: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:51.019811  4721 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.028182  4684 raft_consensus.cc:493] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:51.028769  4684 raft_consensus.cc:3060] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.030203  4524 raft_consensus.cc:515] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:51.037362  4076 raft_consensus.cc:2393] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6994df41341e478d89f7c105f550ff93 in current term 1: Already voted for candidate 25902f00fb5d41679245f23dd19654aa in this term.
I20260709 12:20:51.038362  4524 leader_election.cc:290] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:51.032985  4273 raft_consensus.cc:359] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:51.039287  4273 raft_consensus.cc:385] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.039628  4273 raft_consensus.cc:740] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.040073  4280 tablet_bootstrap.cc:492] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:51.040825  4280 ts_tablet_manager.cc:1403] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.277s	user 0.018s	sys 0.011s
I20260709 12:20:51.043203  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:51.030094  4721 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:51.044606  4280 raft_consensus.cc:359] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.045305  4280 raft_consensus.cc:385] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.045646  4280 raft_consensus.cc:740] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.044304  4684 raft_consensus.cc:515] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:51.046460  4280 consensus_queue.cc:260] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.035004  4272 tablet_bootstrap.cc:654] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:51.048545  4280 ts_tablet_manager.cc:1434] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:20:51.049407  4280 tablet_bootstrap.cc:492] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:51.049484  4684 leader_election.cc:290] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:51.043192  4220 raft_consensus.cc:3060] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.055270  4280 tablet_bootstrap.cc:654] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:51.056867  4272 tablet_bootstrap.cc:492] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:51.040333  4273 consensus_queue.cc:260] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:51.058529  4272 ts_tablet_manager.cc:1403] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.002s
I20260709 12:20:51.059643  4273 ts_tablet_manager.cc:1434] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:20:51.062785  4272 raft_consensus.cc:359] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.127781  3996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:51.130429  3998 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:51.162959  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:51.164506  4148 raft_consensus.cc:3060] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.165700  4120 leader_election.cc:304] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:51.170421  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:51.063650  4272 raft_consensus.cc:385] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.186038  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:51.187685  4221 raft_consensus.cc:2393] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 25902f00fb5d41679245f23dd19654aa in current term 1: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:51.189549  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:51.186332  4272 raft_consensus.cc:740] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.198939  4272 consensus_queue.cc:260] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.207094  4076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:51.208534  3903 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48782 (request call id 88) took 1313 ms. Trace:
I20260709 12:20:51.209261  4076 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.214483  4272 ts_tablet_manager.cc:1434] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.156s	user 0.008s	sys 0.002s
I20260709 12:20:51.217427  4272 tablet_bootstrap.cc:492] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:51.230530  4073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:51.238317  3902 catalog_manager.cc:5697] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 1 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:51.241351  4073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:51.243040  4073 raft_consensus.cc:2393] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 25902f00fb5d41679245f23dd19654aa in this term.
I20260709 12:20:51.285533  4280 tablet_bootstrap.cc:492] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:51.286617  4280 ts_tablet_manager.cc:1403] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.237s	user 0.012s	sys 0.004s
I20260709 12:20:51.290678  4045 leader_election.cc:304] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:51.313017  4745 raft_consensus.cc:2749] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:51.209177  3903 rpcz_store.cc:276] 0709 12:20:49.895200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:49.895390 (+   190us) service_pool.cc:224] Handling call
0709 12:20:51.208202 (+1312812us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:50.282270 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:50.282858 (+   588us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:50.282901 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:20:50.282936 (+    35us) tablet.cc:662] Decoding operations
0709 12:20:50.283372 (+   436us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:50.283443 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:50.283481 (+    38us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:20:50.283831 (+   350us) tablet.cc:701] Row locks acquired
0709 12:20:50.283851 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:20:50.284002 (+   151us) write_op.cc:270] Start()
0709 12:20:50.284118 (+   116us) write_op.cc:276] Timestamp: P: 1783599650283958 usec, L: 0
0709 12:20:50.284141 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:20:50.284912 (+   771us) log.cc:844] Serialized 922 byte log entry
0709 12:20:50.620465 (+335553us) op_driver.cc:464] REPLICATION: finished
0709 12:20:50.700703 (+ 80238us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b4400022b4c. stack: 00007f85bb768343 00007f85bb768240 00007f85bb0f71d4 00007f85c4e35304 00007f85c4e35299 00007f85bb622db1 00007f85bb6c0c85 00007f85bb6c16a2 00007f85be4331db 00007f85be42eaaf 00007f85bb77d0ac 00007f85c45d79a9 00007f85bb79969b 00007f85bb79793f 00007f85bb799c02 00007f85c22bdef0
0709 12:20:50.828604 (+127901us) write_op.cc:301] APPLY: starting
0709 12:20:50.828694 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:50.828813 (+   119us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:50.828837 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:50.829203 (+   366us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:50.829376 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:20:50.829423 (+    47us) write_op.cc:312] APPLY: finished
0709 12:20:50.829635 (+   212us) log.cc:844] Serialized 28 byte log entry
0709 12:20:50.839195 (+  9560us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:50.842317 (+  3122us) write_op.cc:454] Released schema lock
0709 12:20:50.846640 (+  4323us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:50.872873 (+ 26233us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:50.878400 (+  5527us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":45647,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":206542,"apply.run_cpu_time_us":2012,"apply.run_wall_time_us":31075,"num_ops":2,"prepare.queue_time_us":100899,"prepare.run_cpu_time_us":3007,"prepare.run_wall_time_us":4803,"replication_time_us":336175,"spinlock_wait_cycles":8069504,"thread_start_us":229006,"threads_started":2}]]}
W20260709 12:20:51.323212  3971 leader_election.cc:343] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:51.319706  4682 consensus_queue.cc:1048] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:20:51.337426  4280 raft_consensus.cc:359] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.338675  4280 raft_consensus.cc:385] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.339036  4280 raft_consensus.cc:740] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.339946  4280 consensus_queue.cc:260] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.355186  4148 raft_consensus.cc:2468] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
W20260709 12:20:51.355638  3970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 153523us
I20260709 12:20:51.380816  4445 raft_consensus.cc:2749] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:51.370225  4280 ts_tablet_manager.cc:1434] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.083s	user 0.010s	sys 0.000s
I20260709 12:20:51.383236  4280 tablet_bootstrap.cc:492] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:51.373898  4272 tablet_bootstrap.cc:654] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:51.387543  4193 leader_election.cc:304] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa
I20260709 12:20:51.394924  3970 leader_election.cc:304] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:51.401106  4561 raft_consensus.cc:2749] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:51.380697  4120 leader_election.cc:304] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:51.402149  4561 consensus_queue.cc:1048] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:51.404827  4445 raft_consensus.cc:2749] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:51.406819  4120 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:51.410135  4659 raft_consensus.cc:2749] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:51.413174  4076 raft_consensus.cc:2468] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:51.416150  4682 raft_consensus.cc:2804] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:51.422258  4682 raft_consensus.cc:697] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:51.422448  4280 tablet_bootstrap.cc:654] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:51.432770  4682 consensus_queue.cc:237] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:20:51.434204  4706 raft_consensus.cc:493] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.434942  4706 raft_consensus.cc:515] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.440539  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:51.450250  3971 leader_election.cc:343] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:51.452787  4532 raft_consensus.cc:493] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.453624  4532 raft_consensus.cc:515] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:51.459940  4656 consensus_queue.cc:1048] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:51.437722  4706 leader_election.cc:290] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:51.466077  3996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
W20260709 12:20:51.468591  4046 leader_election.cc:343] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:51.465309  4532 leader_election.cc:290] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:51.476199  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:51.477377  4148 raft_consensus.cc:2468] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:51.478843  3970 leader_election.cc:304] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:51.480219  4706 raft_consensus.cc:2804] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:51.484337  4706 raft_consensus.cc:493] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:51.484797  4706 raft_consensus.cc:3060] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.521291  4668 raft_consensus.cc:493] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.527750  4668 raft_consensus.cc:515] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.529896  3971 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 72940e4e52944adb81608e286ad58a59
I20260709 12:20:51.531786  4561 raft_consensus.cc:2804] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:51.532407  4561 raft_consensus.cc:697] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:51.533569  4561 consensus_queue.cc:237] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.538842  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:51.539883  4154 raft_consensus.cc:2468] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 0.
I20260709 12:20:51.547504  4045 leader_election.cc:304] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:51.549321  4668 leader_election.cc:290] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:51.550879  3996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9233488a2b6745f8a012d9edc348333a" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:51.552636  3996 raft_consensus.cc:2468] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:51.569336  4074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9233488a2b6745f8a012d9edc348333a" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:51.570304  4074 raft_consensus.cc:2468] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:51.571854  4761 raft_consensus.cc:2804] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:51.572371  4761 raft_consensus.cc:493] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:51.572736  4761 raft_consensus.cc:3060] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.579912  4194 leader_election.cc:304] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:51.581375  4668 raft_consensus.cc:2804] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:51.581883  4668 raft_consensus.cc:493] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:51.581962  4272 tablet_bootstrap.cc:492] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:51.582809  4272 ts_tablet_manager.cc:1403] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.366s	user 0.033s	sys 0.010s
I20260709 12:20:51.582921  4668 raft_consensus.cc:3060] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.590409  4706 raft_consensus.cc:515] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.608820  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:51.610744  4152 raft_consensus.cc:3060] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.627893  4280 tablet_bootstrap.cc:492] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:51.628624  4280 ts_tablet_manager.cc:1403] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.246s	user 0.014s	sys 0.016s
I20260709 12:20:51.636742  4706 leader_election.cc:290] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:51.641372  4280 raft_consensus.cc:359] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:51.642436  4280 raft_consensus.cc:385] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.642745  4280 raft_consensus.cc:740] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.643426  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:51.649729  4272 raft_consensus.cc:359] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.650525  4272 raft_consensus.cc:385] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.650816  4272 raft_consensus.cc:740] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.651752  4272 consensus_queue.cc:260] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.649536  4280 consensus_queue.cc:260] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:51.654276  4761 raft_consensus.cc:515] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:51.656854  4761 leader_election.cc:290] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:51.664718  4152 raft_consensus.cc:2468] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
W20260709 12:20:51.673862  3951 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:51.676348  3970 leader_election.cc:304] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:51.679471  4677 raft_consensus.cc:2804] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:51.680181  4677 raft_consensus.cc:697] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:51.682288  4677 consensus_queue.cc:237] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:51.684944  3971 leader_election.cc:343] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:51.687463  4272 ts_tablet_manager.cc:1434] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.101s	user 0.008s	sys 0.000s
I20260709 12:20:51.688971  4272 tablet_bootstrap.cc:492] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:51.691798  4001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:51.655107  4280 ts_tablet_manager.cc:1434] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:20:51.693543  4280 tablet_bootstrap.cc:492] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:51.696085  4668 raft_consensus.cc:515] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.698236  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:51.699162  4152 raft_consensus.cc:3060] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.699584  4001 raft_consensus.cc:3060] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.703277  3902 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48766 (request call id 85) took 1543 ms. Trace:
I20260709 12:20:51.714804  4152 raft_consensus.cc:2468] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:51.721383  4045 leader_election.cc:304] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:51.723150  4668 leader_election.cc:290] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:51.724049  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9233488a2b6745f8a012d9edc348333a" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:51.724972  4004 raft_consensus.cc:3060] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.726855  4272 tablet_bootstrap.cc:654] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:51.729925  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9233488a2b6745f8a012d9edc348333a" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:51.730665  4077 raft_consensus.cc:3060] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.733268  4525 raft_consensus.cc:2804] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:51.738065  4525 raft_consensus.cc:697] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:51.739154  4525 consensus_queue.cc:237] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:51.703734  3902 rpcz_store.cc:276] 0709 12:20:50.160138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:50.160309 (+   171us) service_pool.cc:224] Handling call
0709 12:20:51.703204 (+1542895us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:51.488208 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:51.488959 (+   751us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:51.489034 (+    75us) write_op.cc:435] Acquired schema lock
0709 12:20:51.489053 (+    19us) tablet.cc:662] Decoding operations
0709 12:20:51.489517 (+   464us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:51.489633 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:51.489668 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:51.490865 (+  1197us) tablet.cc:701] Row locks acquired
0709 12:20:51.490916 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:20:51.496490 (+  5574us) write_op.cc:270] Start()
0709 12:20:51.523413 (+ 26923us) write_op.cc:276] Timestamp: P: 1783599651491075 usec, L: 0
0709 12:20:51.526728 (+  3315us) op_driver.cc:348] REPLICATION: starting
0709 12:20:51.545105 (+ 18377us) log.cc:844] Serialized 537 byte log entry
0709 12:20:51.604568 (+ 59463us) op_driver.cc:464] REPLICATION: finished
0709 12:20:51.642407 (+ 37839us) write_op.cc:301] APPLY: starting
0709 12:20:51.642516 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:51.650611 (+  8095us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:51.650867 (+   256us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:51.651212 (+   345us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:51.651466 (+   254us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:20:51.651528 (+    62us) write_op.cc:312] APPLY: finished
0709 12:20:51.651803 (+   275us) log.cc:844] Serialized 22 byte log entry
0709 12:20:51.652006 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:51.652094 (+    88us) write_op.cc:454] Released schema lock
0709 12:20:51.652151 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:51.662203 (+ 10052us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:51.662240 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":74468,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":37517,"apply.run_cpu_time_us":2164,"apply.run_wall_time_us":19735,"num_ops":1,"prepare.queue_time_us":133270,"prepare.run_cpu_time_us":4093,"prepare.run_wall_time_us":60246,"replication_time_us":77662,"thread_start_us":88708,"threads_started":2}]]}
I20260709 12:20:51.746631  4004 raft_consensus.cc:2468] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:51.748620  4272 tablet_bootstrap.cc:492] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:51.754752  4272 ts_tablet_manager.cc:1403] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.010s
I20260709 12:20:51.754964  4194 leader_election.cc:304] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:51.755847  4668 raft_consensus.cc:2804] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:51.756379  4668 raft_consensus.cc:697] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:51.757346  4668 consensus_queue.cc:237] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.752526  4001 raft_consensus.cc:2468] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:51.768222  4272 raft_consensus.cc:359] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.770927  4272 raft_consensus.cc:385] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.771277  4272 raft_consensus.cc:740] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.792047  4280 tablet_bootstrap.cc:654] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:51.795450  4077 raft_consensus.cc:2468] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:51.793403  3903 catalog_manager.cc:5697] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:51.799868  3903 catalog_manager.cc:5697] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:51.812398  4272 consensus_queue.cc:260] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.820680  4272 ts_tablet_manager.cc:1434] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.065s	user 0.004s	sys 0.003s
I20260709 12:20:51.821983  4272 tablet_bootstrap.cc:492] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59: Bootstrap starting.
I20260709 12:20:51.835623  4272 tablet_bootstrap.cc:654] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:51.841838  3899 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48754 (request call id 87) took 1628 ms. Trace:
I20260709 12:20:51.842336  3899 rpcz_store.cc:276] 0709 12:20:50.221374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:50.224390 (+  3016us) service_pool.cc:224] Handling call
0709 12:20:51.841573 (+1617183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5930,"reported_tablets":15}
I20260709 12:20:51.842818  4668 consensus_queue.cc:1048] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:51.854725  3901 catalog_manager.cc:5697] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:51.880538  4280 tablet_bootstrap.cc:492] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: No bootstrap required, opened a new log
I20260709 12:20:51.881278  4280 ts_tablet_manager.cc:1403] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.188s	user 0.005s	sys 0.009s
I20260709 12:20:51.900341  4730 consensus_queue.cc:1048] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:51.905946  4280 raft_consensus.cc:359] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.906936  4280 raft_consensus.cc:385] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.908077  4280 raft_consensus.cc:740] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.908944  4280 consensus_queue.cc:260] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.918324  4668 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.931015  4280 ts_tablet_manager.cc:1434] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.002s
I20260709 12:20:51.919041  4668 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.933670  4668 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:51.934551  4677 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.935179  4677 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.947227  4272 tablet_bootstrap.cc:492] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59: No bootstrap required, opened a new log
I20260709 12:20:51.947916  4272 ts_tablet_manager.cc:1403] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59: Time spent bootstrapping tablet: real 0.126s	user 0.033s	sys 0.025s
I20260709 12:20:51.951054  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:51.952054  4148 raft_consensus.cc:2468] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 0.
I20260709 12:20:51.955171  4193 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:51.959290  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:51.960145  4147 raft_consensus.cc:2468] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 0.
I20260709 12:20:51.961483  4730 raft_consensus.cc:2804] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:51.961987  4730 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:51.962312  4730 raft_consensus.cc:3060] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.961666  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:51.963035  3970 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:51.963833  4561 raft_consensus.cc:2804] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:51.964215  4561 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:51.964562  4561 raft_consensus.cc:3060] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.971798  3996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:51.949292  4677 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:51.978749  4561 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:51.980262  3996 raft_consensus.cc:2393] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 41009fde51f045c49be6461b76a27543 in this term.
I20260709 12:20:51.985176  4561 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:51.988513  4272 raft_consensus.cc:359] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.990304  4272 raft_consensus.cc:385] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:51.990592  4272 raft_consensus.cc:740] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Initialized, Role: FOLLOWER
I20260709 12:20:51.991355  4272 consensus_queue.cc:260] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:51.993403  4272 ts_tablet_manager.cc:1434] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:20:51.995627  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:51.996367  4148 raft_consensus.cc:3060] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:51.998787  3903 catalog_manager.cc:5697] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 1 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:52.003139  4730 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:52.004565  4221 raft_consensus.cc:2393] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41009fde51f045c49be6461b76a27543 in current term 1: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:52.011286  4221 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:52.011561  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:52.012755  4004 raft_consensus.cc:2393] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 41009fde51f045c49be6461b76a27543 in this term.
I20260709 12:20:52.009593  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:52.018982  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:52.020041  4224 raft_consensus.cc:2393] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 41009fde51f045c49be6461b76a27543 in current term 1: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:52.022531  4148 raft_consensus.cc:2468] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 1.
I20260709 12:20:52.023185  4748 consensus_queue.cc:1048] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:52.023934  3970 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:52.025357  4661 raft_consensus.cc:2804] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:52.025946  4661 raft_consensus.cc:697] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:20:52.026865  4661 consensus_queue.cc:237] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:52.032490  4193 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:52.010424  4730 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:52.042971  3899 catalog_manager.cc:5697] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 1 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:52.042691  4797 raft_consensus.cc:2749] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:52.067334  4748 consensus_queue.cc:1048] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:52.081987  4730 raft_consensus.cc:493] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:52.082625  4730 raft_consensus.cc:515] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:52.084733  4730 leader_election.cc:290] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:52.085454  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:52.086467  4148 raft_consensus.cc:2393] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 1: Already voted for candidate 41009fde51f045c49be6461b76a27543 in this term.
I20260709 12:20:52.087179  4004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:52.089879  4194 leader_election.cc:304] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:52.091320  4797 raft_consensus.cc:3060] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:52.104156  4797 raft_consensus.cc:2749] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:52.114449  3901 catalog_manager.cc:5697] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:52.124713  3901 catalog_manager.cc:5697] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:52.126868  3901 catalog_manager.cc:5697] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:52.129715  4680 consensus_queue.cc:1048] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:52.155493  4797 consensus_queue.cc:1048] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:52.156986  4525 consensus_queue.cc:1048] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:52.184180  4748 consensus_queue.cc:1048] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:52.190105  4684 consensus_queue.cc:1048] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:52.231839  4661 consensus_queue.cc:1048] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:52.589870  4661 consensus_queue.cc:1048] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:52.612968  4661 consensus_queue.cc:1048] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:53.187984 27116 tablet_server.cc:179] TabletServer@127.26.123.1:0 shutting down...
W20260709 12:20:53.210564  4046 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 386 similar messages]
W20260709 12:20:53.219965  4046 consensus_peers.cc:597] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.224063  4120 consensus_peers.cc:597] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.242915  4046 consensus_peers.cc:597] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.248971 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:53.250168 27116 tablet_replica.cc:333] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.251194 27116 raft_consensus.cc:2243] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:20:53.251542  4120 consensus_peers.cc:597] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.252177 27116 raft_consensus.cc:2272] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.254887 27116 tablet_replica.cc:333] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.255640 27116 raft_consensus.cc:2243] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.256500 27116 raft_consensus.cc:2272] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.258738 27116 tablet_replica.cc:333] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.259459 27116 raft_consensus.cc:2243] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.259930 27116 raft_consensus.cc:2272] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.262900 27116 tablet_replica.cc:333] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.264091 27116 raft_consensus.cc:2243] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.264698 27116 raft_consensus.cc:2272] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.268481 27116 tablet_replica.cc:333] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.269513 27116 raft_consensus.cc:2243] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.270794 27116 raft_consensus.cc:2272] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:53.271382  4194 consensus_peers.cc:597] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.274135 27116 tablet_replica.cc:333] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.275041 27116 raft_consensus.cc:2243] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.275530 27116 raft_consensus.cc:2272] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.278415 27116 tablet_replica.cc:333] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.279181 27116 raft_consensus.cc:2243] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.280277 27116 raft_consensus.cc:2272] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.283268 27116 tablet_replica.cc:333] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.283926 27116 raft_consensus.cc:2243] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.284438 27116 raft_consensus.cc:2272] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.287561 27116 tablet_replica.cc:333] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.288570 27116 raft_consensus.cc:2243] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.289630 27116 raft_consensus.cc:2272] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.292364 27116 tablet_replica.cc:333] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.293226 27116 raft_consensus.cc:2243] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.294449 27116 raft_consensus.cc:2272] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.298976 27116 tablet_replica.cc:333] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.299958 27116 raft_consensus.cc:2243] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.300693 27116 raft_consensus.cc:2272] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.304342 27116 tablet_replica.cc:333] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.305161 27116 raft_consensus.cc:2243] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.306377 27116 raft_consensus.cc:2272] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.310051 27116 tablet_replica.cc:333] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.311038 27116 raft_consensus.cc:2243] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.312405 27116 raft_consensus.cc:2272] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.315867 27116 tablet_replica.cc:333] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: stopping tablet replica
W20260709 12:20:53.316246  4194 consensus_peers.cc:597] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.317063 27116 raft_consensus.cc:2243] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.318310 27116 raft_consensus.cc:2272] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.320966 27116 tablet_replica.cc:333] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.321789 27116 raft_consensus.cc:2243] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.322283 27116 raft_consensus.cc:2272] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.325176 27116 tablet_replica.cc:333] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.326066 27116 raft_consensus.cc:2243] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.327099 27116 raft_consensus.cc:2272] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.330413 27116 tablet_replica.cc:333] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
W20260709 12:20:53.331167  4194 consensus_peers.cc:597] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.331498 27116 raft_consensus.cc:2243] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.332785 27116 raft_consensus.cc:2272] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.335760 27116 tablet_replica.cc:333] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.336740 27116 raft_consensus.cc:2243] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.337409 27116 raft_consensus.cc:2272] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.340930 27116 tablet_replica.cc:333] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.342075 27116 raft_consensus.cc:2243] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.342765 27116 raft_consensus.cc:2272] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.345441 27116 tablet_replica.cc:333] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.346191 27116 raft_consensus.cc:2243] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.346760 27116 raft_consensus.cc:2272] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.349207 27116 tablet_replica.cc:333] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.349884 27116 raft_consensus.cc:2243] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.350888 27116 raft_consensus.cc:2272] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.353395 27116 tablet_replica.cc:333] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.354112 27116 raft_consensus.cc:2243] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.354624 27116 raft_consensus.cc:2272] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.356980 27116 tablet_replica.cc:333] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.357640 27116 raft_consensus.cc:2243] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.358615 27116 raft_consensus.cc:2272] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.360905 27116 tablet_replica.cc:333] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.361538 27116 raft_consensus.cc:2243] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.362059 27116 raft_consensus.cc:2272] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.364627 27116 tablet_replica.cc:333] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.365242 27116 raft_consensus.cc:2243] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.365700 27116 raft_consensus.cc:2272] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.368686 27116 tablet_replica.cc:333] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.369716 27116 raft_consensus.cc:2243] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.371044 27116 raft_consensus.cc:2272] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:53.373128  4120 consensus_peers.cc:597] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.374406 27116 tablet_replica.cc:333] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.375237 27116 raft_consensus.cc:2243] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.375787 27116 raft_consensus.cc:2272] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.378959 27116 tablet_replica.cc:333] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.379845 27116 raft_consensus.cc:2243] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.380486 27116 raft_consensus.cc:2272] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.383721 27116 tablet_replica.cc:333] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.384608 27116 raft_consensus.cc:2243] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.385164 27116 raft_consensus.cc:2272] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.388115 27116 tablet_replica.cc:333] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.389084 27116 raft_consensus.cc:2243] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.389767 27116 raft_consensus.cc:2272] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.393272 27116 tablet_replica.cc:333] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.394273 27116 raft_consensus.cc:2243] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.394896 27116 raft_consensus.cc:2272] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.398074 27116 tablet_replica.cc:333] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.398748 27116 raft_consensus.cc:2243] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.399742 27116 raft_consensus.cc:2272] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.402172 27116 tablet_replica.cc:333] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.402789 27116 raft_consensus.cc:2243] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.403265 27116 raft_consensus.cc:2272] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.406003 27116 tablet_replica.cc:333] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.406863 27116 raft_consensus.cc:2243] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.407459 27116 raft_consensus.cc:2272] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.409578 27116 tablet_replica.cc:333] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.410271 27116 raft_consensus.cc:2243] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.410797 27116 raft_consensus.cc:2272] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.413336 27116 tablet_replica.cc:333] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.414292 27116 raft_consensus.cc:2243] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.415185 27116 raft_consensus.cc:2272] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.417814 27116 tablet_replica.cc:333] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.418552 27116 raft_consensus.cc:2243] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.419144 27116 raft_consensus.cc:2272] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.421985 27116 tablet_replica.cc:333] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.422773 27116 raft_consensus.cc:2243] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.423341 27116 raft_consensus.cc:2272] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.426326 27116 tablet_replica.cc:333] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.427060 27116 raft_consensus.cc:2243] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.428017 27116 raft_consensus.cc:2272] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.431260 27116 tablet_replica.cc:333] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.432103 27116 raft_consensus.cc:2243] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:53.433137 27116 raft_consensus.cc:2272] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:53.434058  4046 consensus_peers.cc:597] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.435935 27116 tablet_replica.cc:333] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.436617 27116 raft_consensus.cc:2243] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.437052 27116 raft_consensus.cc:2272] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.439438 27116 tablet_replica.cc:333] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.440006 27116 raft_consensus.cc:2243] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.440446 27116 raft_consensus.cc:2272] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.443141 27116 tablet_replica.cc:333] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.443900 27116 raft_consensus.cc:2243] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.444471 27116 raft_consensus.cc:2272] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:53.447216 27116 tablet_replica.cc:333] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.447767 27116 raft_consensus.cc:2243] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:53.448239 27116 raft_consensus.cc:2272] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:53.449685  4046 consensus_peers.cc:597] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.451459 27116 tablet_replica.cc:333] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: stopping tablet replica
I20260709 12:20:53.452394 27116 raft_consensus.cc:2243] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:20:53.453267  4194 consensus_peers.cc:597] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.453632 27116 raft_consensus.cc:2272] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:53.457593  4046 consensus_peers.cc:597] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.483341  4194 consensus_peers.cc:597] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.488799  4120 consensus_peers.cc:597] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.494820  4120 consensus_peers.cc:597] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.518208  4046 consensus_peers.cc:597] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.534276 27116 tablet_server.cc:196] TabletServer@127.26.123.1:0 shutdown complete.
W20260709 12:20:53.548215  4194 consensus_peers.cc:597] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.553157  4120 consensus_peers.cc:597] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.563478  4194 consensus_peers.cc:597] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.574949  4120 consensus_peers.cc:597] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.576130  4194 consensus_peers.cc:597] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.581238  4120 consensus_peers.cc:597] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.593824  4046 consensus_peers.cc:597] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.627310  4046 consensus_peers.cc:597] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.640192 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:53.648437  4046 consensus_peers.cc:597] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:53.649101  4819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:53.648345  4818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:53.654603 27116 server_base.cc:1061] running on GCE node
W20260709 12:20:53.656335  4822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:53.657100  4046 consensus_peers.cc:597] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.659996 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:53.660368 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:53.660633 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599653660606 us; error 0 us; skew 500 ppm
I20260709 12:20:53.661303 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:53.664453 27116 webserver.cc:533] Webserver started at http://127.26.123.1:42587/ using document root <none> and password file <none>
I20260709 12:20:53.665015 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:53.665205 27116 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:20:53.669842 27116 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.000s
W20260709 12:20:53.674816  4194 consensus_peers.cc:597] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:53.680914  4826 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:53.682010 27116 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.000s
I20260709 12:20:53.682364 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "41009fde51f045c49be6461b76a27543"
format_stamp: "Formatted at 2026-07-09 12:20:39 on dist-test-slave-9q7n"
I20260709 12:20:53.682777 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:53.721848 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:53.723151 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:53.724820 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:53.730052  4833 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:20:54.111774 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:20:54.112176 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.383s	user 0.006s	sys 0.004s
I20260709 12:20:54.112588 27116 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:20:54.122169  4833 tablet_bootstrap.cc:492] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.142912  4833 tablet_bootstrap.cc:492] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.144416  4833 tablet_bootstrap.cc:492] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.145234  4833 ts_tablet_manager.cc:1403] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.023s	user 0.021s	sys 0.000s
I20260709 12:20:54.148203  4833 raft_consensus.cc:359] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.149070  4833 raft_consensus.cc:740] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.149950  4833 consensus_queue.cc:260] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.155655  4833 ts_tablet_manager.cc:1434] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.007s
I20260709 12:20:54.156737  4833 tablet_bootstrap.cc:492] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.170495  4833 tablet_bootstrap.cc:492] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.171298  4833 tablet_bootstrap.cc:492] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.171917  4833 ts_tablet_manager.cc:1403] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.002s
I20260709 12:20:54.173965  4833 raft_consensus.cc:359] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.174564  4833 raft_consensus.cc:740] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.175148  4833 consensus_queue.cc:260] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.176635  4833 ts_tablet_manager.cc:1434] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:20:54.177474  4833 tablet_bootstrap.cc:492] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.205492  4833 tablet_bootstrap.cc:492] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.206660  4833 tablet_bootstrap.cc:492] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.207365  4833 ts_tablet_manager.cc:1403] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.001s
I20260709 12:20:54.210186  4833 raft_consensus.cc:359] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.210920  4833 raft_consensus.cc:740] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.212112  4833 consensus_queue.cc:260] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.214393  4833 ts_tablet_manager.cc:1434] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:54.215235  4833 tablet_bootstrap.cc:492] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.231518  4833 tablet_bootstrap.cc:492] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.232509  4833 tablet_bootstrap.cc:492] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.233198  4833 ts_tablet_manager.cc:1403] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 12:20:54.235883  4833 raft_consensus.cc:359] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.236676  4833 raft_consensus.cc:740] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.237859  4833 consensus_queue.cc:260] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.239989  4833 ts_tablet_manager.cc:1434] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:54.240852  4833 tablet_bootstrap.cc:492] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.270129  4833 tablet_bootstrap.cc:492] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.270961  4833 tablet_bootstrap.cc:492] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.271541  4833 ts_tablet_manager.cc:1403] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.024s
I20260709 12:20:54.273525  4833 raft_consensus.cc:359] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.274120  4833 raft_consensus.cc:740] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.274653  4833 consensus_queue.cc:260] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.276592  4833 ts_tablet_manager.cc:1434] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:20:54.277333  4833 tablet_bootstrap.cc:492] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.291271  4833 tablet_bootstrap.cc:492] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.292198  4833 tablet_bootstrap.cc:492] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.292881  4833 ts_tablet_manager.cc:1403] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 12:20:54.295403  4833 raft_consensus.cc:359] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.296063  4833 raft_consensus.cc:740] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.296617  4659 raft_consensus.cc:493] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.296823  4833 consensus_queue.cc:260] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.297241  4659 raft_consensus.cc:515] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.299183  4833 ts_tablet_manager.cc:1434] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:20:54.299422  4659 leader_election.cc:290] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.300168  4833 tablet_bootstrap.cc:492] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.302464  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.309016  4668 raft_consensus.cc:493] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
W20260709 12:20:54.309413  4120 leader_election.cc:336] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.310164  4120 leader_election.cc:304] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.311053  4659 raft_consensus.cc:2749] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.309702  4668 raft_consensus.cc:515] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.313335  4668 leader_election.cc:290] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.318476  4833 tablet_bootstrap.cc:492] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.319727  4833 tablet_bootstrap.cc:492] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.319497  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.320624  4833 ts_tablet_manager.cc:1403] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
W20260709 12:20:54.323223  4194 leader_election.cc:336] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.323837  4833 raft_consensus.cc:359] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.324648  4833 raft_consensus.cc:740] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.325500  4833 consensus_queue.cc:260] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.326665  4193 leader_election.cc:304] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:54.327834  4833 ts_tablet_manager.cc:1434] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:54.330098  4835 raft_consensus.cc:2749] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.328979  4833 tablet_bootstrap.cc:492] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.340651  4659 raft_consensus.cc:493] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.341346  4659 raft_consensus.cc:515] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.343628  4659 leader_election.cc:290] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.345093  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
W20260709 12:20:54.352694  4120 leader_election.cc:336] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.353215  4120 leader_election.cc:304] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:54.354028  4659 raft_consensus.cc:2749] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.360189  4525 raft_consensus.cc:493] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.360805  4525 raft_consensus.cc:515] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.362334  4833 tablet_bootstrap.cc:492] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.362867  4525 leader_election.cc:290] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.363374  4833 tablet_bootstrap.cc:492] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.364131  4833 ts_tablet_manager.cc:1403] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.004s
I20260709 12:20:54.364811  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2adcdb890e0483095cb9923dce850a1" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.367084  4833 raft_consensus.cc:359] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.367817  4833 raft_consensus.cc:740] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
W20260709 12:20:54.367955  4046 leader_election.cc:336] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.368444  4046 leader_election.cc:304] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:54.368637  4833 consensus_queue.cc:260] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.370206  4525 raft_consensus.cc:2749] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.371052  4833 ts_tablet_manager.cc:1434] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:20:54.371783  4833 tablet_bootstrap.cc:492] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.379599  4668 raft_consensus.cc:493] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.380190  4668 raft_consensus.cc:515] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.382298  4668 leader_election.cc:290] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.386742  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:54.388033  4194 leader_election.cc:336] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.388665  4193 leader_election.cc:304] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:54.389812  4668 raft_consensus.cc:2749] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.392442  4833 tablet_bootstrap.cc:492] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.393883  4833 tablet_bootstrap.cc:492] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.394654  4833 ts_tablet_manager.cc:1403] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.006s
I20260709 12:20:54.396270  4668 raft_consensus.cc:493] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.396859  4668 raft_consensus.cc:515] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.398866  4668 leader_election.cc:290] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.397720  4833 raft_consensus.cc:359] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.399569  4659 raft_consensus.cc:493] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.400156  4659 raft_consensus.cc:515] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.402312  4659 leader_election.cc:290] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.399603  4833 raft_consensus.cc:740] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.405778  4833 consensus_queue.cc:260] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.406247  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2adcdb890e0483095cb9923dce850a1" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.406960  4077 raft_consensus.cc:2468] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:54.408392  4833 ts_tablet_manager.cc:1434] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:20:54.409407  4833 tablet_bootstrap.cc:492] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.411167  4120 leader_election.cc:304] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:54.414986  4834 raft_consensus.cc:493] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.415756  4834 raft_consensus.cc:515] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.418339  4810 raft_consensus.cc:2804] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.418825  4810 raft_consensus.cc:493] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.419234  4810 raft_consensus.cc:3060] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.413714  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
W20260709 12:20:54.419423  4120 leader_election.cc:336] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.429507  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.421998  4834 leader_election.cc:290] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.431857 27116 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:20:54.432237 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.320s	user 0.296s	sys 0.004s
W20260709 12:20:54.433233  4194 leader_election.cc:336] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.436995  4810 raft_consensus.cc:515] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.440709  4525 raft_consensus.cc:493] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.444876  4525 raft_consensus.cc:515] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.444242  4838 raft_consensus.cc:493] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.446826  4659 raft_consensus.cc:493] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
W20260709 12:20:54.447045  4194 leader_election.cc:336] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.445955  4838 raft_consensus.cc:515] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.447505  4659 raft_consensus.cc:515] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.448688  4833 tablet_bootstrap.cc:492] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.442382  4810 leader_election.cc:290] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.447579  4194 leader_election.cc:304] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.449759  4838 leader_election.cc:290] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.450700  4834 raft_consensus.cc:2749] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.449677  4833 tablet_bootstrap.cc:492] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.452210  4833 ts_tablet_manager.cc:1403] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.010s
I20260709 12:20:54.449846  4659 leader_election.cc:290] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.455235  4833 raft_consensus.cc:359] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.456089  4833 raft_consensus.cc:740] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.458580  4525 leader_election.cc:290] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.444419  4193 leader_election.cc:304] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:54.461457  4834 raft_consensus.cc:2749] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:54.466431  4833 consensus_queue.cc:260] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.469177  4833 ts_tablet_manager.cc:1434] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:20:54.470247  4833 tablet_bootstrap.cc:492] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
W20260709 12:20:54.475317  4046 leader_election.cc:336] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.476389  4525 raft_consensus.cc:493] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.477087  4525 raft_consensus.cc:515] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.481006  4525 leader_election.cc:290] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.490486  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:54.491338  4220 raft_consensus.cc:2468] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
W20260709 12:20:54.493005  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 49727us
I20260709 12:20:54.495155  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2adcdb890e0483095cb9923dce850a1" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
W20260709 12:20:54.495255  4046 leader_election.cc:336] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.495759  4077 raft_consensus.cc:3060] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.496426  4046 leader_election.cc:304] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:54.499197  4525 raft_consensus.cc:2804] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.499619  4525 raft_consensus.cc:493] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.499960  4525 raft_consensus.cc:3060] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.501919  4074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb8a7e4c51b446578c698dcf9a253001" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
W20260709 12:20:54.509588  4046 leader_election.cc:336] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.511139  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e4899b95af40fd9836e281f4c23e17" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.513032  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.513707  4152 raft_consensus.cc:2468] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:54.519234  4833 tablet_bootstrap.cc:492] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.519424  4077 raft_consensus.cc:2468] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 2.
I20260709 12:20:54.520531  4833 tablet_bootstrap.cc:492] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.521006  4525 raft_consensus.cc:515] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.521431  4833 ts_tablet_manager.cc:1403] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.007s
I20260709 12:20:54.522083  4838 raft_consensus.cc:493] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.522625  4838 raft_consensus.cc:515] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.523746  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:54.524552  4224 raft_consensus.cc:3060] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.524508  4045 leader_election.cc:304] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:54.526018  4680 raft_consensus.cc:2749] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.524746  4838 leader_election.cc:290] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.527241  4045 leader_election.cc:304] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.527840  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:54.528669  4680 raft_consensus.cc:2804] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.529075  4680 raft_consensus.cc:493] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.529441  4680 raft_consensus.cc:3060] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.534255  4224 raft_consensus.cc:2468] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:20:54.534952  4659 raft_consensus.cc:493] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.536439  4833 raft_consensus.cc:359] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.543769  4833 raft_consensus.cc:740] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.543709  4659 raft_consensus.cc:515] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.544952  4833 consensus_queue.cc:260] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.546288  4659 leader_election.cc:290] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
W20260709 12:20:54.534654  4046 leader_election.cc:336] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.547735  4833 ts_tablet_manager.cc:1434] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 12:20:54.547323  4046 leader_election.cc:304] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:54.548588  4680 raft_consensus.cc:515] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.548803  4833 tablet_bootstrap.cc:492] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.549515  4838 raft_consensus.cc:2749] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.552901  4120 leader_election.cc:304] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:54.554347  4817 raft_consensus.cc:2804] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:54.554981  4817 raft_consensus.cc:697] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:54.555893  4817 consensus_queue.cc:237] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:54.562835  4120 leader_election.cc:336] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:54.563720  4120 leader_election.cc:336] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.564122  4120 leader_election.cc:304] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.554322  4046 leader_election.cc:304] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:54.568105  4525 leader_election.cc:290] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.568744  4525 raft_consensus.cc:2804] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:54.569176  4525 raft_consensus.cc:697] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:54.565809  4680 leader_election.cc:290] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.565382  4659 raft_consensus.cc:2749] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.575425  4525 consensus_queue.cc:237] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:54.577322  4046 leader_election.cc:336] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.577957  4833 tablet_bootstrap.cc:492] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.587823  4833 tablet_bootstrap.cc:492] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.588719  4833 ts_tablet_manager.cc:1403] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.008s
I20260709 12:20:54.579507  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:54.580312  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:54.590097  4152 raft_consensus.cc:3060] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.590137  4224 raft_consensus.cc:2468] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:54.584944  4864 raft_consensus.cc:493] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
W20260709 12:20:54.586249  4120 leader_election.cc:336] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:54.591534  4046 leader_election.cc:336] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.594355  4817 raft_consensus.cc:493] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.594921  4817 raft_consensus.cc:515] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.597065  4817 leader_election.cc:290] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.597433  3903 catalog_manager.cc:5697] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.599741  4833 raft_consensus.cc:359] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.600474  4833 raft_consensus.cc:740] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.594805  4120 leader_election.cc:304] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.605767  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:54.601459  4833 consensus_queue.cc:260] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.608539  4152 raft_consensus.cc:2468] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:20:54.609041  4224 raft_consensus.cc:2468] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 2.
I20260709 12:20:54.610847  4833 ts_tablet_manager.cc:1434] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:20:54.611179  4120 leader_election.cc:304] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:54.601871  4864 raft_consensus.cc:515] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
W20260709 12:20:54.613453  4120 leader_election.cc:336] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.611727  4833 tablet_bootstrap.cc:492] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.615471  4659 raft_consensus.cc:2804] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:20:54.615999  4659 raft_consensus.cc:493] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.616397  4659 raft_consensus.cc:3060] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:54.617192  4680 raft_consensus.cc:493] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.618638  4680 raft_consensus.cc:515] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.617350  4045 leader_election.cc:304] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.624377  4838 raft_consensus.cc:2804] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:54.624558  4680 leader_election.cc:290] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.624922  4838 raft_consensus.cc:697] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:54.615342  4817 raft_consensus.cc:2804] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.625684  4817 raft_consensus.cc:493] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.627236  4834 raft_consensus.cc:493] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.626104  4838 consensus_queue.cc:237] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.627934  4834 raft_consensus.cc:515] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.632301  4834 leader_election.cc:290] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.626212  4817 raft_consensus.cc:3060] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.632890  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.633700  4152 raft_consensus.cc:2468] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:54.637470  4193 leader_election.cc:304] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
W20260709 12:20:54.642714  4194 leader_election.cc:336] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.642740  4834 raft_consensus.cc:2804] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:20:54.644133  4046 leader_election.cc:336] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.645040  4852 raft_consensus.cc:493] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.648777  4852 raft_consensus.cc:515] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.651147  4852 leader_election.cc:290] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.643603  4834 raft_consensus.cc:493] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
W20260709 12:20:54.667028  4045 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.3:39469, user_credentials={real_user=slave}} blocked reactor thread for 49761.9us
I20260709 12:20:54.673280  4852 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.674072  4852 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.674997  4864 leader_election.cc:290] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.676100  4852 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.677134  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01529dc6d8944b9697b9a87c4e4bf450" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:54.673259  4668 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.681133  4668 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.669610  4834 raft_consensus.cc:3060] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.705138  4668 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.709349  4876 raft_consensus.cc:493] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.704804  4817 raft_consensus.cc:515] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.709986  4876 raft_consensus.cc:515] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.710410  4659 raft_consensus.cc:515] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.711516  4817 leader_election.cc:290] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.712242  4876 leader_election.cc:290] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.712633  4659 leader_election.cc:290] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.712882  4876 raft_consensus.cc:493] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
W20260709 12:20:54.716655  4046 leader_election.cc:336] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.717235  4046 leader_election.cc:304] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:54.704000  4874 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.725296  4874 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.722476  4680 raft_consensus.cc:2749] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.733196  4852 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.733950  4835 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.733937  4852 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.729015  4864 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.735033  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.734865  4835 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.713464  4876 raft_consensus.cc:515] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.737241  4835 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.737995  4852 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.735033  4864 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.738994  4077 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:54.742095  4864 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.742906  4864 raft_consensus.cc:493] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.743413  4864 raft_consensus.cc:515] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:54.744316  4046 leader_election.cc:336] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.745994  4864 leader_election.cc:290] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.748072  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb8a7e4c51b446578c698dcf9a253001" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.748800  4152 raft_consensus.cc:2468] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:54.749547  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.734287  4874 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.751425  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.752219  4154 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:54.754796  4834 raft_consensus.cc:515] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:54.755611  4046 leader_election.cc:336] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:54.757966  4194 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 78433.5us
I20260709 12:20:54.760092  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.760782  4077 raft_consensus.cc:2468] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:54.763365  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.764120  4154 raft_consensus.cc:2468] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:54.765103  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.765792  4147 raft_consensus.cc:2468] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:54.768473  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26349fc0d5664798a248220d968f5269" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.774322  4872 raft_consensus.cc:493] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.774966  4872 raft_consensus.cc:515] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:54.764956  4046 leader_election.cc:336] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.776309  3899 catalog_manager.cc:5697] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 2 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.777614  4872 leader_election.cc:290] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.774923  4833 tablet_bootstrap.cc:492] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.779709  4833 tablet_bootstrap.cc:492] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.780712  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:54.780933  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48a9a743ab6141218e529314c318f16b" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:54.781462  4227 raft_consensus.cc:3060] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.781966  4876 leader_election.cc:290] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.782094  4864 raft_consensus.cc:493] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.781211  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:54.783023  4220 raft_consensus.cc:2393] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6994df41341e478d89f7c105f550ff93 in current term 2: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:54.782212  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:54.784303  4224 raft_consensus.cc:2468] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:54.781598  4221 raft_consensus.cc:3060] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:54.782675  4864 raft_consensus.cc:515] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.792248  4872 raft_consensus.cc:493] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.792871  4872 raft_consensus.cc:515] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.794939  4872 leader_election.cc:290] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.791245  4120 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:54.796020  4872 raft_consensus.cc:2804] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.796398  4872 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.780453  4833 ts_tablet_manager.cc:1403] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.169s	user 0.022s	sys 0.013s
I20260709 12:20:54.796763  4872 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.796696  4864 leader_election.cc:290] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.799567  4876 raft_consensus.cc:493] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.800133  4876 raft_consensus.cc:515] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.800438  4833 raft_consensus.cc:359] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.801167  4833 raft_consensus.cc:740] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:54.801983  4833 consensus_queue.cc:260] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.802408  4876 leader_election.cc:290] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.803896  4221 raft_consensus.cc:2468] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 3.
I20260709 12:20:54.804580  4833 ts_tablet_manager.cc:1434] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:20:54.805063  4227 raft_consensus.cc:2468] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 2.
I20260709 12:20:54.805238  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:54.805588  4833 tablet_bootstrap.cc:492] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.806236  4225 raft_consensus.cc:2468] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:54.807488  4659 raft_consensus.cc:493] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.808115  4659 raft_consensus.cc:515] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.810386  4659 leader_election.cc:290] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.811380  4045 leader_election.cc:304] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.812606  4864 raft_consensus.cc:2804] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.812999  4864 raft_consensus.cc:493] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.813403  4864 raft_consensus.cc:3060] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:54.817823  4194 leader_election.cc:336] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:54.818637  4194 leader_election.cc:336] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.823093  4073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.823316  4194 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.824080  4073 raft_consensus.cc:2468] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
I20260709 12:20:54.813266  4834 leader_election.cc:290] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.825161  4880 raft_consensus.cc:2804] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.825623  4880 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.825717  4864 raft_consensus.cc:515] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.832693  4833 tablet_bootstrap.cc:492] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.832825  4872 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.833918  4833 tablet_bootstrap.cc:492] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:54.834728  4833 ts_tablet_manager.cc:1403] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.004s
I20260709 12:20:54.822528  4193 leader_election.cc:304] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.835179  4872 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.829357  4045 leader_election.cc:304] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:54.836275  4668 raft_consensus.cc:2804] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.837181  4668 raft_consensus.cc:493] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.837355  4864 leader_election.cc:290] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.837543  4668 raft_consensus.cc:3060] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.838068  4864 raft_consensus.cc:2749] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.838564  4045 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.837756  4833 raft_consensus.cc:359] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.839206  4833 raft_consensus.cc:740] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
W20260709 12:20:54.831241  4120 leader_election.cc:336] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.839962  4833 consensus_queue.cc:260] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:54.840655  4120 leader_election.cc:336] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.827194  4194 leader_election.cc:304] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
W20260709 12:20:54.841436  4120 leader_election.cc:336] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:54.841995  4120 leader_election.cc:336] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:54.842420  4194 leader_election.cc:336] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.842658  4833 ts_tablet_manager.cc:1434] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:54.843605  4833 tablet_bootstrap.cc:492] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:54.826099  4880 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.846271  4668 raft_consensus.cc:515] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.829713  4876 raft_consensus.cc:493] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.848592  4876 raft_consensus.cc:515] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.848791  4073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.849812  4073 raft_consensus.cc:2468] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:54.850984  4876 leader_election.cc:290] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.848274  4074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e4899b95af40fd9836e281f4c23e17" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.851666  4834 raft_consensus.cc:2804] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.851917  4074 raft_consensus.cc:2468] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:54.854176  4817 raft_consensus.cc:493] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.856570  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb8a7e4c51b446578c698dcf9a253001" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:54.857313  4147 raft_consensus.cc:3060] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.857561  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75948129b8264d75972f4660893bd024" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:54.856384  4817 raft_consensus.cc:515] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.860323  4817 leader_election.cc:290] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.860845  4074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:54.861122  4149 raft_consensus.cc:2468] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 1.
I20260709 12:20:54.839821  4864 raft_consensus.cc:2804] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.862130  4074 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.853392  4834 raft_consensus.cc:493] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.863920  4834 raft_consensus.cc:3060] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.866906  4668 leader_election.cc:290] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.855569  4193 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.872540  4074 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 2.
W20260709 12:20:54.842989  4194 leader_election.cc:336] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.876538  4864 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:54.877089  4864 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:54.877184  4876 raft_consensus.cc:2804] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.878861  4876 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.880873  4876 raft_consensus.cc:3060] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:54.882751  4120 leader_election.cc:336] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:54.886133  4194 leader_election.cc:336] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.887476  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01529dc6d8944b9697b9a87c4e4bf450" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.889252  4077 raft_consensus.cc:2468] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 1.
W20260709 12:20:54.890944  4194 leader_election.cc:336] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.904079  4193 leader_election.cc:304] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
W20260709 12:20:54.888449  4046 leader_election.cc:336] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.909322  4668 raft_consensus.cc:2749] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:54.918143  4120 leader_election.cc:304] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:54.919585  4120 leader_election.cc:304] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:54.921756  4852 raft_consensus.cc:2804] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.922344  4852 raft_consensus.cc:493] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.922792  4852 raft_consensus.cc:3060] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.923548  3903 catalog_manager.cc:5697] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 2 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:54.920727  4817 raft_consensus.cc:493] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.930505  4817 raft_consensus.cc:515] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.922117  4872 raft_consensus.cc:2804] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.938267  4872 raft_consensus.cc:493] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.932722  4817 leader_election.cc:290] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.938736  4872 raft_consensus.cc:3060] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.925042  4120 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:54.945052  4833 tablet_bootstrap.cc:492] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:54.947335  4817 raft_consensus.cc:2804] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:54.949847  4817 raft_consensus.cc:697] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:54.950911  4817 consensus_queue.cc:237] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.952682  4194 leader_election.cc:304] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:54.953786  4668 raft_consensus.cc:2804] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.954183  4668 raft_consensus.cc:493] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.954522  4668 raft_consensus.cc:3060] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.957703  4045 leader_election.cc:304] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:54.959683  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:54.960417  4154 raft_consensus.cc:3060] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.961205  4074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75948129b8264d75972f4660893bd024" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:54.961946  4074 raft_consensus.cc:2468] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:54.962384  4838 raft_consensus.cc:2804] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:54.962739  4838 raft_consensus.cc:493] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:54.963114  4838 raft_consensus.cc:3060] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.970620  4147 raft_consensus.cc:2468] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:20:54.974591  4864 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.977139  4864 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 3 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:54.973384  4876 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:54.979493  4876 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:54.985385  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18dadd11589447aea90415c88b77e312" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:54.986150  4147 raft_consensus.cc:3060] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.990700  4225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:54.985849  4668 raft_consensus.cc:515] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:54.992607  4225 raft_consensus.cc:2468] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:54.987170  4120 leader_election.cc:304] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:54.994990  4120 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
W20260709 12:20:54.993455  4194 leader_election.cc:336] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:54.996833  4872 raft_consensus.cc:515] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:54.999691  4668 leader_election.cc:290] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:54.999939  4810 raft_consensus.cc:2804] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:55.001765  4810 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:55.002224  4810 raft_consensus.cc:3060] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:54.867182  4880 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.005977  4880 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:55.006458  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26349fc0d5664798a248220d968f5269" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:55.007077  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:55.008131  4833 tablet_bootstrap.cc:492] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:55.000568  4147 raft_consensus.cc:2468] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 2.
I20260709 12:20:55.008980  4833 ts_tablet_manager.cc:1403] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.166s	user 0.031s	sys 0.036s
I20260709 12:20:55.007261  4224 raft_consensus.cc:2468] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:54.989339  4852 raft_consensus.cc:515] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.016513  4852 leader_election.cc:290] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:55.016674  4810 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.020242  4834 raft_consensus.cc:515] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.020885  4148 raft_consensus.cc:2393] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 2: Already voted for candidate 6994df41341e478d89f7c105f550ff93 in this term.
W20260709 12:20:55.022477  4046 leader_election.cc:336] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.022678  4834 leader_election.cc:290] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:55.023484  4872 leader_election.cc:290] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:55.019558  4154 raft_consensus.cc:2468] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 2.
I20260709 12:20:55.001335  4838 raft_consensus.cc:515] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.025091  4659 raft_consensus.cc:2749] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.026356  4810 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
W20260709 12:20:54.998453  4194 leader_election.cc:336] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.024187  4833 raft_consensus.cc:359] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.029592  4193 leader_election.cc:304] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.030938  4834 raft_consensus.cc:2804] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.031494  4834 raft_consensus.cc:697] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:55.032639  4046 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.032377  4834 consensus_queue.cc:237] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.033522  4864 raft_consensus.cc:2804] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:55.033947  4864 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:55.034459  4864 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.037950  4193 leader_election.cc:304] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.038105  4224 raft_consensus.cc:1275] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.039662  4879 raft_consensus.cc:2804] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.040258  4879 raft_consensus.cc:697] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:55.041342  4879 consensus_queue.cc:237] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:55.045079  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 44241.2us
I20260709 12:20:55.046748  4120 leader_election.cc:304] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.045831  3903 catalog_manager.cc:5697] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:55.029839  4833 raft_consensus.cc:740] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:55.052362  4852 raft_consensus.cc:2804] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:55.052940  4852 raft_consensus.cc:697] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 3 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:55.052580  4833 consensus_queue.cc:260] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.053946  4874 consensus_queue.cc:1048] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:55.053953  4852 consensus_queue.cc:237] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.056427  4120 leader_election.cc:304] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.028756  4838 leader_election.cc:290] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:55.061815  4893 raft_consensus.cc:2804] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.062388  4893 raft_consensus.cc:697] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:55.063243  4073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:55.063259  4893 consensus_queue.cc:237] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.061302  4074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01529dc6d8944b9697b9a87c4e4bf450" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:55.064524  4074 raft_consensus.cc:3060] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.061374  4076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14347d13af8448bb12300afb6a90639" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:55.067198  4076 raft_consensus.cc:3060] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.068300  4833 ts_tablet_manager.cc:1434] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.000s
I20260709 12:20:55.069271  4833 tablet_bootstrap.cc:492] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:55.071003  4120 leader_election.cc:304] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:55.076290  4659 raft_consensus.cc:2804] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:55.076707  4659 raft_consensus.cc:493] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:55.077015  4659 raft_consensus.cc:3060] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.089150  4659 raft_consensus.cc:515] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:55.092326  4046 leader_election.cc:336] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.095139  4659 leader_election.cc:290] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:55.095002  4076 raft_consensus.cc:2468] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 2.
I20260709 12:20:55.098125  4120 leader_election.cc:304] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:55.103144  4872 raft_consensus.cc:2804] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:55.103595  4872 raft_consensus.cc:493] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:20:55.103961  4872 raft_consensus.cc:3060] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.088402  4864 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.119637  4073 raft_consensus.cc:2393] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 72940e4e52944adb81608e286ad58a59 in current term 2: Already voted for candidate 25902f00fb5d41679245f23dd19654aa in this term.
W20260709 12:20:55.120446  4194 leader_election.cc:336] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.121096  4194 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:55.117723  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
W20260709 12:20:55.123474  4194 leader_election.cc:336] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:55.129841  4194 leader_election.cc:336] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:55.127602  4046 consensus_peers.cc:597] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.091398  4074 raft_consensus.cc:2468] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 2.
W20260709 12:20:55.133458  4194 leader_election.cc:336] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.135394  4147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75948129b8264d75972f4660893bd024" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:55.136129  4147 raft_consensus.cc:3060] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:55.136968  4046 leader_election.cc:336] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:55.138579  4046 consensus_peers.cc:597] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.138660  4864 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:55.129108  4835 raft_consensus.cc:2749] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:20:55.141395  4120 leader_election.cc:336] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:55.143419  4120 leader_election.cc:336] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.145035  4194 leader_election.cc:304] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.145320  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:55.146214  4224 raft_consensus.cc:2393] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 25902f00fb5d41679245f23dd19654aa in current term 2: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:55.152315  3903 catalog_manager.cc:5697] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:55.145064  4120 leader_election.cc:336] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:55.154728  4120 leader_election.cc:336] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.154892  3903 catalog_manager.cc:5697] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:55.160959  4045 leader_election.cc:304] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.162358  4838 raft_consensus.cc:2804] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.162914  4838 raft_consensus.cc:697] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:55.162618  4872 raft_consensus.cc:515] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.163829  4838 consensus_queue.cc:237] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.164981  4872 leader_election.cc:290] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:55.154567  4194 leader_election.cc:304] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.159593  3899 catalog_manager.cc:5697] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 2 to 3, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 3 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:55.155222  4835 raft_consensus.cc:2804] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.169454  4835 raft_consensus.cc:697] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:55.169971  4194 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.170547  4835 consensus_queue.cc:237] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.171105  4876 raft_consensus.cc:2804] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.171773  4876 raft_consensus.cc:697] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:55.172686  4876 consensus_queue.cc:237] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.174118  4668 raft_consensus.cc:2749] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.179353  4833 tablet_bootstrap.cc:492] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:55.180553  4833 tablet_bootstrap.cc:492] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:55.181298  4833 ts_tablet_manager.cc:1403] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.112s	user 0.019s	sys 0.019s
I20260709 12:20:55.188457  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:55.189353  4227 raft_consensus.cc:2393] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6994df41341e478d89f7c105f550ff93 in current term 2: Already voted for candidate 72940e4e52944adb81608e286ad58a59 in this term.
I20260709 12:20:55.189502  4076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e4899b95af40fd9836e281f4c23e17" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:55.190269  4076 raft_consensus.cc:3060] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.191279  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:55.191983  4077 raft_consensus.cc:3060] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.194032  4833 raft_consensus.cc:359] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.194756  4833 raft_consensus.cc:740] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:55.196709  4833 consensus_queue.cc:260] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.197070  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e9cdd46c01a43e3a92c6065a0a15e12" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:55.197712  4227 raft_consensus.cc:3060] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.200801  4079 raft_consensus.cc:1275] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.207810  3903 catalog_manager.cc:5697] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 2 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:55.205220  3899 catalog_manager.cc:5697] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:55.214829  4045 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:55.216599  4874 raft_consensus.cc:2749] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:20:55.220068  4077 raft_consensus.cc:2468] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 2.
I20260709 12:20:55.222748  4833 ts_tablet_manager.cc:1434] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 12:20:55.223995  4833 tablet_bootstrap.cc:492] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:55.224152  4227 raft_consensus.cc:2468] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 2.
I20260709 12:20:55.226384  4120 leader_election.cc:304] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:55.219419  4147 raft_consensus.cc:2468] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:20:55.227104  4893 raft_consensus.cc:2764] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:20:55.229924  4893 consensus_queue.cc:1048] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:20:55.241997  4046 leader_election.cc:336] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.249125  4046 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:55.250236  4874 raft_consensus.cc:2749] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.235941  4120 leader_election.cc:304] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
W20260709 12:20:55.256162  4120 leader_election.cc:336] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:55.256982  4120 leader_election.cc:336] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.257946  4852 raft_consensus.cc:2804] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.258610  4852 raft_consensus.cc:697] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:55.259537  4852 consensus_queue.cc:237] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.262063  4076 raft_consensus.cc:2468] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 2.
I20260709 12:20:55.284186  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26349fc0d5664798a248220d968f5269" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:55.285099  4224 raft_consensus.cc:3060] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.308972  4120 leader_election.cc:304] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:20:55.311609  4659 raft_consensus.cc:2804] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.312160  4659 raft_consensus.cc:697] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:55.308226  4148 raft_consensus.cc:1275] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.322336  4659 consensus_queue.cc:237] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.323104  4045 leader_election.cc:304] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.326093  4838 raft_consensus.cc:2804] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.326712  4838 raft_consensus.cc:697] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:55.327626  4838 consensus_queue.cc:237] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
W20260709 12:20:55.346760  4120 leader_election.cc:336] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:55.347457  4120 leader_election.cc:336] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:20:55.348233  4120 consensus_peers.cc:597] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.340458  3899 catalog_manager.cc:5697] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:55.348650  4120 leader_election.cc:336] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.349704  4864 consensus_queue.cc:1048] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:55.349043  4120 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:55.350963  4893 raft_consensus.cc:2749] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.349558  3899 catalog_manager.cc:5697] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:55.352847  4120 leader_election.cc:336] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.358389  4840 raft_consensus.cc:493] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.359155  4840 raft_consensus.cc:515] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.358314  4120 leader_election.cc:304] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.362030  4893 raft_consensus.cc:2804] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.360675  4224 raft_consensus.cc:2468] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 2.
I20260709 12:20:55.362649  4893 raft_consensus.cc:697] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:55.385716  4893 consensus_queue.cc:237] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.389657  4840 leader_election.cc:290] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:55.390405  4840 raft_consensus.cc:493] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.390898  4840 raft_consensus.cc:515] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.393303  4840 leader_election.cc:290] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:55.404443  3903 catalog_manager.cc:5697] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:55.433717  4120 leader_election.cc:336] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:20:55.469329  4833 tablet_bootstrap.cc:492] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:55.470647  4833 tablet_bootstrap.cc:492] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:55.472990  4833 ts_tablet_manager.cc:1403] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.249s	user 0.028s	sys 0.005s
I20260709 12:20:55.476285  4833 raft_consensus.cc:359] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.478343  4148 raft_consensus.cc:1275] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.480263  4879 consensus_queue.cc:1048] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:55.489044  4833 raft_consensus.cc:740] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:55.489274  4154 raft_consensus.cc:1275] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.491384  4879 consensus_queue.cc:1048] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:55.499442  3903 catalog_manager.cc:5697] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 2 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:55.502413  4833 consensus_queue.cc:260] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:20:55.517642  4194 consensus_peers.cc:597] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:55.521378  4925 raft_consensus.cc:493] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.522104  4925 raft_consensus.cc:515] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.524739  4833 ts_tablet_manager.cc:1434] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 12:20:55.527467  4833 tablet_bootstrap.cc:492] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:55.530587  3903 catalog_manager.cc:5697] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:55.538208  4194 consensus_peers.cc:597] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.538952  3903 catalog_manager.cc:5697] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:55.547224  4840 raft_consensus.cc:493] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.548080  4840 raft_consensus.cc:515] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.550590  4840 leader_election.cc:290] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:55.555605  4925 leader_election.cc:290] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:55.558720  4224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:55.559415  4227 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778cd7f085494ba1842e5f758d7f9828" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:55.560361  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08446fff0954c05b5311d3e76153e58" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:55.565295  4120 leader_election.cc:304] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:55.566901  4076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778cd7f085494ba1842e5f758d7f9828" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:55.567880  4893 raft_consensus.cc:2804] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:55.568792  4893 raft_consensus.cc:697] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:55.569419  4830 leader_election.cc:304] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:55.570590  4925 raft_consensus.cc:2749] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.569857  4893 consensus_queue.cc:237] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.571880  4076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b08446fff0954c05b5311d3e76153e58" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:55.574070  4830 leader_election.cc:304] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:55.575776  4925 raft_consensus.cc:2749] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.584986  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd58e37cb1847738fdf9dd220c2280a" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
W20260709 12:20:55.603998  4194 consensus_peers.cc:597] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.628496  4220 raft_consensus.cc:1275] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:55.642217  4076 raft_consensus.cc:1240] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Rejecting Update request from peer 6994df41341e478d89f7c105f550ff93 for earlier term 2. Current term is 3. Ops: []
I20260709 12:20:55.665783  4079 raft_consensus.cc:1275] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:20:55.678318  4194 consensus_peers.cc:597] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.681871  4194 consensus_peers.cc:597] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:55.684122  4834 consensus_queue.cc:1048] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:55.694444  3899 catalog_manager.cc:5697] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 1 to 2, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 6994df41341e478d89f7c105f550ff93 (127.26.123.3). New cstate: current_term: 2 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:55.697284  4046 consensus_peers.cc:597] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.702804  4833 tablet_bootstrap.cc:492] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:55.703970  4833 tablet_bootstrap.cc:492] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:55.705641  4076 raft_consensus.cc:1275] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.704787  4833 ts_tablet_manager.cc:1403] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.179s	user 0.029s	sys 0.037s
I20260709 12:20:55.710649  4833 raft_consensus.cc:359] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.711416  4833 raft_consensus.cc:740] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:55.711864  4148 raft_consensus.cc:1275] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.712182  4833 consensus_queue.cc:260] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.712955  4834 consensus_queue.cc:1048] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:55.713840  4872 consensus_queue.cc:1048] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:55.715473  4833 ts_tablet_manager.cc:1434] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:20:55.723066  4833 tablet_bootstrap.cc:492] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:55.745765  4925 raft_consensus.cc:493] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.746398  4925 raft_consensus.cc:515] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.748564  4925 leader_election.cc:290] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:55.751564  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:55.755580  4225 raft_consensus.cc:1275] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:20:55.763757  4194 consensus_peers.cc:597] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:55.767108  4194 consensus_peers.cc:597] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.769387  4893 consensus_queue.cc:1059] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:55.770712  4872 raft_consensus.cc:3055] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:55.771134  4872 raft_consensus.cc:740] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:55.772086  4872 consensus_queue.cc:260] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.773432  4872 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:55.774891  4925 raft_consensus.cc:493] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.775460  4925 raft_consensus.cc:515] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.777599  4925 leader_election.cc:290] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:55.778985  4864 consensus_queue.cc:1048] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:55.779548  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:55.779500  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:55.783483  4830 leader_election.cc:304] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:55.790517  4840 raft_consensus.cc:2749] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.791344  4840 raft_consensus.cc:493] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.791862  4840 raft_consensus.cc:515] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.805626  4073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:55.806715  4073 raft_consensus.cc:2393] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 41009fde51f045c49be6461b76a27543 in current term 2: Already voted for candidate 6994df41341e478d89f7c105f550ff93 in this term.
I20260709 12:20:55.812201  4840 leader_election.cc:290] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
W20260709 12:20:55.814188  4046 consensus_peers.cc:597] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:55.825745  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 56578.2us
W20260709 12:20:55.830533  4194 consensus_peers.cc:597] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:55.843008  4833 tablet_bootstrap.cc:492] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:55.844015  4833 tablet_bootstrap.cc:492] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:55.844674  4833 ts_tablet_manager.cc:1403] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.122s	user 0.011s	sys 0.016s
I20260709 12:20:55.847298  4833 raft_consensus.cc:359] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.847985  4833 raft_consensus.cc:740] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:55.848719  4833 consensus_queue.cc:260] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.850898  4833 ts_tablet_manager.cc:1434] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:20:55.851706  4833 tablet_bootstrap.cc:492] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:55.853796  4840 raft_consensus.cc:493] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.854468  4840 raft_consensus.cc:515] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:55.858662  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:55.860620  4840 leader_election.cc:290] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
W20260709 12:20:55.862614  4046 consensus_peers.cc:597] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:55.872812  4146 raft_consensus.cc:1275] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.881632  4833 tablet_bootstrap.cc:492] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:55.882736  4833 tablet_bootstrap.cc:492] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:55.883697  4833 ts_tablet_manager.cc:1403] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.008s
W20260709 12:20:55.886411  4194 consensus_peers.cc:597] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:55.893085  4120 consensus_peers.cc:597] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.886510  4833 raft_consensus.cc:359] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.903863  4833 raft_consensus.cc:740] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:55.905077  4864 consensus_queue.cc:1048] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:55.904968  4833 consensus_queue.cc:260] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:55.906682  4080 raft_consensus.cc:1275] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.908563  4833 ts_tablet_manager.cc:1434] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:20:55.908576  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:55.910192  4833 tablet_bootstrap.cc:492] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:55.910620  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dd58e37cb1847738fdf9dd220c2280a" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:55.913722  4227 raft_consensus.cc:1275] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:55.915303  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "525557ea38a44858b86a97b0c444ecc4" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:55.916783  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07edbbf4ef64ddbbad929d20ce78c4e" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:55.918332  4148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64bc045a9f78405ebd413d12e6236f61" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:55.919566  4872 consensus_queue.cc:1048] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:20:55.887655  4046 consensus_peers.cc:597] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:55.929917  4046 consensus_peers.cc:597] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.931842 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:41077
I20260709 12:20:55.931962  4969 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:41077 every 8 connection(s)
I20260709 12:20:55.938750  4810 consensus_queue.cc:1048] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:55.943360  4079 raft_consensus.cc:1275] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:20:55.959117  4120 consensus_peers.cc:597] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:55.959867  4120 consensus_peers.cc:597] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:55.960425  4120 consensus_peers.cc:597] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:55.960860  4120 consensus_peers.cc:597] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:55.961629  4120 consensus_peers.cc:597] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:55.963582  4829 leader_election.cc:304] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:55.964869  4840 raft_consensus.cc:2749] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.966118  4829 leader_election.cc:304] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:55.966391  4833 tablet_bootstrap.cc:492] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:55.967419  4840 raft_consensus.cc:2749] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.967990  4833 tablet_bootstrap.cc:492] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:55.968820  4833 ts_tablet_manager.cc:1403] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.059s	user 0.023s	sys 0.028s
I20260709 12:20:55.968688  4829 leader_election.cc:304] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:55.969563  4840 raft_consensus.cc:2749] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.970227  4829 leader_election.cc:304] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:55.971585  4829 leader_election.cc:304] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:55.972400  4925 raft_consensus.cc:3060] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.973541  4840 raft_consensus.cc:2749] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.977648  4919 raft_consensus.cc:3060] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:55.984730  4925 raft_consensus.cc:2749] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:55.993876  4833 raft_consensus.cc:359] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.994647  4833 raft_consensus.cc:740] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:55.997372  4833 consensus_queue.cc:260] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:55.999603  4833 ts_tablet_manager.cc:1434] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:20:56.000466  4833 tablet_bootstrap.cc:492] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.001468  4810 consensus_queue.cc:1048] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:20:56.010221  4840 raft_consensus.cc:493] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:56.010807  4925 raft_consensus.cc:493] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:56.010900  4840 raft_consensus.cc:515] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.011539  4925 raft_consensus.cc:515] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.014699  4909 raft_consensus.cc:3060] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:56.015189  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
W20260709 12:20:56.016088  4046 consensus_peers.cc:597] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:56.013690  4925 leader_election.cc:290] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
W20260709 12:20:56.018505  4194 consensus_peers.cc:597] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:56.020460  4872 consensus_queue.cc:1048] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:20:56.043687  4840 leader_election.cc:290] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:56.045521  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fda924a0568433fa70ab72d4a0c3968" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:56.049629  4970 heartbeater.cc:344] Connected to a master server at 127.26.123.62:34017
I20260709 12:20:56.050140  4970 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:56.051209  4970 heartbeater.cc:507] Master 127.26.123.62:34017 requested a full tablet report, sending...
I20260709 12:20:56.068037  4833 tablet_bootstrap.cc:492] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.069168  4833 tablet_bootstrap.cc:492] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.069927  4833 ts_tablet_manager.cc:1403] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.070s	user 0.026s	sys 0.003s
I20260709 12:20:56.071828  4919 raft_consensus.cc:3060] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:56.081809  4146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:56.081624  4833 raft_consensus.cc:359] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.082594  4833 raft_consensus.cc:740] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.083317  4146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fda924a0568433fa70ab72d4a0c3968" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:56.083434  4833 consensus_queue.cc:260] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.086470  4833 ts_tablet_manager.cc:1434] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.004s
I20260709 12:20:56.087393  4833 tablet_bootstrap.cc:492] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.089147  4914 raft_consensus.cc:3060] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:56.095991  4046 consensus_peers.cc:597] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:56.115962  4046 consensus_peers.cc:597] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:56.120040  4925 raft_consensus.cc:493] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 25902f00fb5d41679245f23dd19654aa)
I20260709 12:20:56.120716  4925 raft_consensus.cc:515] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.138300  4829 leader_election.cc:304] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:56.140407  4840 raft_consensus.cc:2749] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:56.142722  4829 leader_election.cc:304] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:56.156401  4840 raft_consensus.cc:2749] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:56.184217  4925 leader_election.cc:290] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:20:56.187803  4221 raft_consensus.cc:1275] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:56.196764  4906 raft_consensus.cc:1275] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:56.214653  4907 raft_consensus.cc:3060] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:56.261023  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
W20260709 12:20:56.263411  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.4:42389, user_credentials={real_user=slave}} blocked reactor thread for 59125.9us
I20260709 12:20:56.249358  4810 consensus_queue.cc:1048] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:20:56.275473  4840 raft_consensus.cc:493] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:56.276170  4840 raft_consensus.cc:515] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.282133  4903 raft_consensus.cc:3060] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:56.290521  4194 consensus_peers.cc:597] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:20:56.293783  4194 consensus_peers.cc:597] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:56.303186  4840 leader_election.cc:290] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:20:56.304812  4833 tablet_bootstrap.cc:492] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.305393  4907 raft_consensus.cc:1275] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:56.304713  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:56.330485  4833 tablet_bootstrap.cc:492] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.332803  4833 ts_tablet_manager.cc:1403] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.246s	user 0.037s	sys 0.020s
I20260709 12:20:56.356809  4903 raft_consensus.cc:1275] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:20:56.359118  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 79701.2us
I20260709 12:20:56.363404  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a252087aa0c9426cb3067f2f9004dd29" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:56.363690  4146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:56.393167  4810 consensus_queue.cc:1048] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:56.398627  4996 consensus_queue.cc:1048] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:56.400545  4996 consensus_queue.cc:1048] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:20:56.403291  4120 consensus_peers.cc:597] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:20:56.406987  4829 leader_election.cc:304] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:20:56.408003  4988 raft_consensus.cc:2749] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:20:56.408543  4829 leader_election.cc:304] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41009fde51f045c49be6461b76a27543; no voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59
I20260709 12:20:56.409467  4988 raft_consensus.cc:2749] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:56.417323  4907 raft_consensus.cc:3060] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:56.422595  3903 ts_manager.cc:194] Re-registered known tserver with Master: 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:56.414314  4833 raft_consensus.cc:359] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.425031  4833 raft_consensus.cc:740] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.426048  4833 consensus_queue.cc:260] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.428525  4833 ts_tablet_manager.cc:1434] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.091s	user 0.001s	sys 0.003s
I20260709 12:20:56.429462  4833 tablet_bootstrap.cc:492] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
W20260709 12:20:56.469946  4120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:43667, user_credentials={real_user=slave}} blocked reactor thread for 47312.8us
I20260709 12:20:56.477641  4833 tablet_bootstrap.cc:492] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.480001  4833 tablet_bootstrap.cc:492] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.480751  4833 ts_tablet_manager.cc:1403] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.017s
I20260709 12:20:56.493945  4833 raft_consensus.cc:359] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.494837  4833 raft_consensus.cc:740] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.495632  4833 consensus_queue.cc:260] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.513521  4833 ts_tablet_manager.cc:1434] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:20:56.514626  4833 tablet_bootstrap.cc:492] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.534716  4833 tablet_bootstrap.cc:492] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.535737  4833 tablet_bootstrap.cc:492] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.536442  4833 ts_tablet_manager.cc:1403] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.000s
I20260709 12:20:56.539420  4833 raft_consensus.cc:359] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.540135  4833 raft_consensus.cc:740] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.540831  4833 consensus_queue.cc:260] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.543066  4833 ts_tablet_manager.cc:1434] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:20:56.544035  4833 tablet_bootstrap.cc:492] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.560179  3903 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40880
I20260709 12:20:56.561161  4833 tablet_bootstrap.cc:492] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.562374  4833 tablet_bootstrap.cc:492] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.563057  4833 ts_tablet_manager.cc:1403] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.001s
I20260709 12:20:56.566356  4970 heartbeater.cc:499] Master 127.26.123.62:34017 was elected leader, sending a full tablet report...
I20260709 12:20:56.566215  4833 raft_consensus.cc:359] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.566926  4833 raft_consensus.cc:740] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.567701  4833 consensus_queue.cc:260] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.569912  4833 ts_tablet_manager.cc:1434] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:56.570798  4833 tablet_bootstrap.cc:492] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.572039  4903 raft_consensus.cc:3060] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:56.594419  4833 tablet_bootstrap.cc:492] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.595266  4833 tablet_bootstrap.cc:492] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.595911  4833 ts_tablet_manager.cc:1403] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.011s
I20260709 12:20:56.597949  4833 raft_consensus.cc:359] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.598464  4833 raft_consensus.cc:740] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.599045  4833 consensus_queue.cc:260] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.600704  4833 ts_tablet_manager.cc:1434] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:20:56.601421  4833 tablet_bootstrap.cc:492] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.630589  4833 tablet_bootstrap.cc:492] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.636216  4833 tablet_bootstrap.cc:492] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.636981  4833 ts_tablet_manager.cc:1403] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.007s
I20260709 12:20:56.639578  4833 raft_consensus.cc:359] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.640193  4833 raft_consensus.cc:740] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.640910  4833 consensus_queue.cc:260] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.642632  4833 ts_tablet_manager.cc:1434] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:20:56.643321  4833 tablet_bootstrap.cc:492] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.658198  4833 tablet_bootstrap.cc:492] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.659150  4833 tablet_bootstrap.cc:492] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.659857  4833 ts_tablet_manager.cc:1403] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.003s
I20260709 12:20:56.665112  4833 raft_consensus.cc:359] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:56.665967  4833 raft_consensus.cc:740] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.666659  4833 consensus_queue.cc:260] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:56.668707  4833 ts_tablet_manager.cc:1434] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.003s
I20260709 12:20:56.669984  4833 tablet_bootstrap.cc:492] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.688416  4833 tablet_bootstrap.cc:492] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.689345  4833 tablet_bootstrap.cc:492] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.690356  4833 ts_tablet_manager.cc:1403] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.011s
I20260709 12:20:56.692780  4833 raft_consensus.cc:359] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.693491  4833 raft_consensus.cc:740] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.694906  4833 consensus_queue.cc:260] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.698172  4833 ts_tablet_manager.cc:1434] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:20:56.699102  4833 tablet_bootstrap.cc:492] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.715126  4833 tablet_bootstrap.cc:492] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.716048  4833 tablet_bootstrap.cc:492] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.716647  4833 ts_tablet_manager.cc:1403] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 12:20:56.718752  4833 raft_consensus.cc:359] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:56.719426  4833 raft_consensus.cc:740] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.719954  4833 consensus_queue.cc:260] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:56.721812  4833 ts_tablet_manager.cc:1434] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:20:56.722672  4833 tablet_bootstrap.cc:492] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.738569  4833 tablet_bootstrap.cc:492] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.739638  4833 tablet_bootstrap.cc:492] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.740523  4833 ts_tablet_manager.cc:1403] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.006s
I20260709 12:20:56.744805  4833 raft_consensus.cc:359] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.746084  4833 raft_consensus.cc:740] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.747224  4833 consensus_queue.cc:260] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.749536  4833 ts_tablet_manager.cc:1434] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:20:56.751866  4833 tablet_bootstrap.cc:492] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.769716  4833 tablet_bootstrap.cc:492] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.770852  4833 tablet_bootstrap.cc:492] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.771636  4833 ts_tablet_manager.cc:1403] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:20:56.774524  4833 raft_consensus.cc:359] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:56.775424  4833 raft_consensus.cc:740] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.776165  4833 consensus_queue.cc:260] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:56.778719  4833 ts_tablet_manager.cc:1434] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:56.779690  4833 tablet_bootstrap.cc:492] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.808437  4833 tablet_bootstrap.cc:492] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.810338  4833 tablet_bootstrap.cc:492] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.811161  4833 ts_tablet_manager.cc:1403] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.007s
I20260709 12:20:56.865964  4833 raft_consensus.cc:359] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:56.866921  4833 raft_consensus.cc:740] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:56.867769  4833 consensus_queue.cc:260] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:56.870868  4833 ts_tablet_manager.cc:1434] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.004s
I20260709 12:20:56.872160  4833 tablet_bootstrap.cc:492] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:56.876151  4903 raft_consensus.cc:3060] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:56.902133  4904 raft_consensus.cc:3060] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:56.934237  5016 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:56.934877  5016 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.938305  4919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
W20260709 12:20:56.939882  4194 leader_election.cc:343] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:56.943924  5016 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:56.945910  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93" is_pre_election: true
I20260709 12:20:56.946794  4149 raft_consensus.cc:2468] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 2.
I20260709 12:20:56.949470  4193 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:56.959573  4668 raft_consensus.cc:2804] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:20:56.960050  4668 raft_consensus.cc:493] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:56.960374  4668 raft_consensus.cc:3060] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:56.960677  4659 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:56.961263  4659 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:56.963685  4659 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:56.965618  4919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:56.966449  4919 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 1.
I20260709 12:20:56.971253  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa" is_pre_election: true
I20260709 12:20:56.971957  4080 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 3.
I20260709 12:20:56.972437  4120 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:56.982462  4983 raft_consensus.cc:2804] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:20:56.983024  4983 raft_consensus.cc:493] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:56.983433  4983 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:56.984230  4668 raft_consensus.cc:515] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:56.987860  4668 leader_election.cc:290] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:56.989012  4903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
W20260709 12:20:56.990453  4194 leader_election.cc:343] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:56.988903  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08603ff2d9145e0b85e975bac56312c" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:20:56.991921  4154 raft_consensus.cc:3060] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:56.994957  4833 tablet_bootstrap.cc:492] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:56.995880  4833 tablet_bootstrap.cc:492] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:56.996541  4833 ts_tablet_manager.cc:1403] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.125s	user 0.016s	sys 0.000s
I20260709 12:20:56.998937  4833 raft_consensus.cc:359] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:56.999677  4833 raft_consensus.cc:740] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.000110  4154 raft_consensus.cc:2468] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 3.
I20260709 12:20:57.000335  4833 consensus_queue.cc:260] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.001569  4193 leader_election.cc:304] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:57.002709  4833 ts_tablet_manager.cc:1434] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:20:57.002801  5016 raft_consensus.cc:2804] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:57.003626  5016 raft_consensus.cc:697] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 3 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:20:57.004235  4833 tablet_bootstrap.cc:492] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.004480  5016 consensus_queue.cc:237] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.005867  4983 raft_consensus.cc:515] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:57.009900  4903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
I20260709 12:20:57.010695  4903 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 4
I20260709 12:20:57.028821  3903 catalog_manager.cc:5697] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 1 to 3, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 3 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:57.046402  4983 leader_election.cc:290] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 4 election: Requested vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:20:57.054366  4903 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 4.
I20260709 12:20:57.054955  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fc90d55c744447aaa17e745cf58157c" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:20:57.055640  4080 raft_consensus.cc:3060] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:57.056041  4120 leader_election.cc:304] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:20:57.056588  4833 tablet_bootstrap.cc:492] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.057022  4659 raft_consensus.cc:2804] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:57.057646  4833 tablet_bootstrap.cc:492] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.058497  4833 ts_tablet_manager.cc:1403] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.055s	user 0.018s	sys 0.015s
I20260709 12:20:57.062007  4833 raft_consensus.cc:359] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.062814  4833 raft_consensus.cc:740] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.063691  4833 consensus_queue.cc:260] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.064977  4080 raft_consensus.cc:2468] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6994df41341e478d89f7c105f550ff93 in term 4.
I20260709 12:20:57.057580  4659 raft_consensus.cc:697] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 4 LEADER]: Becoming Leader. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:20:57.069090  4659 consensus_queue.cc:237] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:57.073619  4833 ts_tablet_manager.cc:1434] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:20:57.074952  4833 tablet_bootstrap.cc:492] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.080075  4903 raft_consensus.cc:3060] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:57.103896  3903 catalog_manager.cc:5697] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6994df41341e478d89f7c105f550ff93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:57.119877  4833 tablet_bootstrap.cc:492] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.121057  4833 tablet_bootstrap.cc:492] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.122332  4833 ts_tablet_manager.cc:1403] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.048s	user 0.017s	sys 0.010s
I20260709 12:20:57.127631  4833 raft_consensus.cc:359] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:57.128422  4833 raft_consensus.cc:740] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.129179  4833 consensus_queue.cc:260] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:57.131677  4833 ts_tablet_manager.cc:1434] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:20:57.132561  4833 tablet_bootstrap.cc:492] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.153599  4864 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:57.154301  4864 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.157867  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:20:57.158620  4223 raft_consensus.cc:2468] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:20:57.159122  4903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543" is_pre_election: true
I20260709 12:20:57.161773  4864 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
W20260709 12:20:57.161439  4046 leader_election.cc:343] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:57.163656  4046 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:57.167184  4985 raft_consensus.cc:2804] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:20:57.171595  4833 tablet_bootstrap.cc:492] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.172703  4833 tablet_bootstrap.cc:492] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.173910  4833 ts_tablet_manager.cc:1403] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.042s	user 0.020s	sys 0.003s
I20260709 12:20:57.177086  4833 raft_consensus.cc:359] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.177817  4833 raft_consensus.cc:740] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.178328  4985 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:57.178893  4985 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:57.178503  4833 consensus_queue.cc:260] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.190644  4833 ts_tablet_manager.cc:1434] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:20:57.191835  4833 tablet_bootstrap.cc:492] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.229827  4985 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.232852  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:20:57.233580  4220 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:57.245702  4985 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 3 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:20:57.247999  4904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "41009fde51f045c49be6461b76a27543"
W20260709 12:20:57.250944  4046 leader_election.cc:343] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:57.252897  4833 tablet_bootstrap.cc:492] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.255281  4833 tablet_bootstrap.cc:492] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.256052  4833 ts_tablet_manager.cc:1403] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.064s	user 0.045s	sys 0.004s
I20260709 12:20:57.256266  4220 raft_consensus.cc:2468] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 3.
I20260709 12:20:57.259231  4833 raft_consensus.cc:359] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:57.260246  4833 raft_consensus.cc:740] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.260668  4046 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543
I20260709 12:20:57.261149  4833 consensus_queue.cc:260] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:20:57.261550  4864 raft_consensus.cc:2804] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:57.262283  4864 raft_consensus.cc:697] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 3 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:20:57.263087  4864 consensus_queue.cc:237] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.269822  4833 ts_tablet_manager.cc:1434] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:20:57.270952  4833 tablet_bootstrap.cc:492] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.277985  3903 catalog_manager.cc:5697] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 1 to 3, leader changed from 41009fde51f045c49be6461b76a27543 (127.26.123.1) to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 3 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:57.307919  4833 tablet_bootstrap.cc:492] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.308992  4833 tablet_bootstrap.cc:492] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.309715  4833 ts_tablet_manager.cc:1403] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.008s
I20260709 12:20:57.312470  4833 raft_consensus.cc:359] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.313107  4833 raft_consensus.cc:740] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.313692  4833 consensus_queue.cc:260] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.315845  4833 ts_tablet_manager.cc:1434] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:20:57.316788  4833 tablet_bootstrap.cc:492] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.333905  4833 tablet_bootstrap.cc:492] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.334863  4833 tablet_bootstrap.cc:492] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.335685  4833 ts_tablet_manager.cc:1403] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:20:57.338661  4833 raft_consensus.cc:359] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.339430  4833 raft_consensus.cc:740] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.340180  4833 consensus_queue.cc:260] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.342623  4833 ts_tablet_manager.cc:1434] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:57.344259  4833 tablet_bootstrap.cc:492] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.362389  4833 tablet_bootstrap.cc:492] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.363379  4833 tablet_bootstrap.cc:492] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.364193  4833 ts_tablet_manager.cc:1403] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:20:57.367077  4833 raft_consensus.cc:359] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.367846  4833 raft_consensus.cc:740] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.368603  4833 consensus_queue.cc:260] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.370998  4833 ts_tablet_manager.cc:1434] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:20:57.372630  4833 tablet_bootstrap.cc:492] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.390734  4833 tablet_bootstrap.cc:492] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.391927  4833 tablet_bootstrap.cc:492] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.392668  4833 ts_tablet_manager.cc:1403] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.000s
I20260709 12:20:57.395587  4833 raft_consensus.cc:359] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.396377  4833 raft_consensus.cc:740] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.397069  4833 consensus_queue.cc:260] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.399408  4833 ts_tablet_manager.cc:1434] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.006s	user 0.009s	sys 0.000s
I20260709 12:20:57.400290  4833 tablet_bootstrap.cc:492] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.400512  4904 raft_consensus.cc:3060] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:20:57.410898  4904 raft_consensus.cc:1275] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:57.412253  4668 consensus_queue.cc:1048] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:57.417023  4833 tablet_bootstrap.cc:492] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.418229  4833 tablet_bootstrap.cc:492] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.419281  4833 ts_tablet_manager.cc:1403] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 12:20:57.422477  4833 raft_consensus.cc:359] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.423301  4833 raft_consensus.cc:740] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.424043  4833 consensus_queue.cc:260] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:20:57.426488  4833 ts_tablet_manager.cc:1434] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:57.430110  4833 tablet_bootstrap.cc:492] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: Bootstrap starting.
I20260709 12:20:57.444844  4154 raft_consensus.cc:1275] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:57.447778  4668 consensus_queue.cc:1048] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:57.457340  4833 tablet_bootstrap.cc:492] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:57.458577  4833 tablet_bootstrap.cc:492] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: Bootstrap complete.
I20260709 12:20:57.459352  4833 ts_tablet_manager.cc:1403] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.012s
I20260709 12:20:57.462261  4833 raft_consensus.cc:359] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.463055  4833 raft_consensus.cc:740] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41009fde51f045c49be6461b76a27543, State: Initialized, Role: FOLLOWER
I20260709 12:20:57.463953  4833 consensus_queue.cc:260] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:20:57.466439  4833 ts_tablet_manager.cc:1434] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:57.504606  4903 raft_consensus.cc:1275] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 4 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:20:57.506094  4997 consensus_queue.cc:1048] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:57.525496  4077 raft_consensus.cc:1275] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 4 FOLLOWER]: Refusing update from remote peer 6994df41341e478d89f7c105f550ff93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:20:57.527889  4659 consensus_queue.cc:1048] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:57.681692  4903 raft_consensus.cc:3060] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:57.687049  4223 raft_consensus.cc:1275] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:57.689087  4987 consensus_queue.cc:1048] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:57.695338  4904 raft_consensus.cc:3060] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:57.715507  4919 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:20:57.729858  4919 raft_consensus.cc:1275] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:57.731328  4864 consensus_queue.cc:1048] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:57.790469  4904 raft_consensus.cc:3060] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:05.955233 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:06.009088  4080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24281667f1364c6fac542e54ed413502"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:42446
I20260709 12:21:06.009835  4080 raft_consensus.cc:606] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 3 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:06.023181  4149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355"
dest_uuid: "6994df41341e478d89f7c105f550ff93"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:38048
I20260709 12:21:06.023837  4149 raft_consensus.cc:606] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:06.026566  4080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "724697c71b61499aa0daadd2e66af805"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
mode: GRACEFUL
new_leader_uuid: "72940e4e52944adb81608e286ad58a59"
 from {username='slave'} at 127.0.0.1:42446
I20260709 12:21:06.027161  4080 raft_consensus.cc:606] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Received request to transfer leadership to TS 72940e4e52944adb81608e286ad58a59
I20260709 12:21:06.030174  4080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
mode: GRACEFUL
new_leader_uuid: "72940e4e52944adb81608e286ad58a59"
 from {username='slave'} at 127.0.0.1:42446
I20260709 12:21:06.030849  4080 raft_consensus.cc:606] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Received request to transfer leadership to TS 72940e4e52944adb81608e286ad58a59
I20260709 12:21:06.033648  4080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e203dae959394f4e90051ad4746a9291"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:42446
I20260709 12:21:06.034336  4080 raft_consensus.cc:606] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:06.036871  4080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:42446
I20260709 12:21:06.037401  4080 raft_consensus.cc:606] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:06.049041  4221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d"
dest_uuid: "72940e4e52944adb81608e286ad58a59"
mode: GRACEFUL
new_leader_uuid: "25902f00fb5d41679245f23dd19654aa"
 from {username='slave'} at 127.0.0.1:46714
I20260709 12:21:06.049592  4221 raft_consensus.cc:606] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Received request to transfer leadership to TS 25902f00fb5d41679245f23dd19654aa
I20260709 12:21:06.050870 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:21:06.053009 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:06.054199 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:06.093499  5066 raft_consensus.cc:993] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59: : Instructing follower 25902f00fb5d41679245f23dd19654aa to start an election
I20260709 12:21:06.094247  5069 raft_consensus.cc:1081] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Signalling peer 25902f00fb5d41679245f23dd19654aa to start an election
I20260709 12:21:06.095942  4080 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
 from {username='slave'} at 127.0.0.1:53998
I20260709 12:21:06.096614  4080 raft_consensus.cc:493] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:06.097061  4080 raft_consensus.cc:3060] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.106236  4080 raft_consensus.cc:515] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:06.109457  4080 leader_election.cc:290] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:21:06.109781  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:06.110728  4221 raft_consensus.cc:3055] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:06.110857  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe179cc2289146d98b10419917a7a36d" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:06.111531  4149 raft_consensus.cc:3060] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.112334  4221 raft_consensus.cc:740] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:21:06.113214  4221 consensus_queue.cc:260] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:06.114570  4221 raft_consensus.cc:3060] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.122057  4221 raft_consensus.cc:2468] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:21:06.123412  4046 leader_election.cc:304] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:06.121479  4149 raft_consensus.cc:2468] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:21:06.124753  5068 raft_consensus.cc:2804] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:06.125263  5068 raft_consensus.cc:697] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:06.126217  5068 consensus_queue.cc:237] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:06.140892  3903 catalog_manager.cc:5697] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 1 to 2, leader changed from 72940e4e52944adb81608e286ad58a59 (127.26.123.4) to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 2 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.160591  5064 raft_consensus.cc:993] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:06.161067  5064 raft_consensus.cc:1081] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:06.162794  4904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e203dae959394f4e90051ad4746a9291"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51316
I20260709 12:21:06.163339  4904 raft_consensus.cc:493] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:06.163623  4904 raft_consensus.cc:3060] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.171321  4904 raft_consensus.cc:515] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:06.173192  4904 leader_election.cc:290] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:06.174046  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e203dae959394f4e90051ad4746a9291" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:06.174225  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e203dae959394f4e90051ad4746a9291" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:06.174862  4223 raft_consensus.cc:3060] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.174921  4080 raft_consensus.cc:3055] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:06.175421  4080 raft_consensus.cc:740] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:06.176334  4080 consensus_queue.cc:260] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:06.177865  4080 raft_consensus.cc:3060] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.183491  4223 raft_consensus.cc:2468] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:06.184909  4830 leader_election.cc:304] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:06.185910  4080 raft_consensus.cc:2468] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:06.186980  5081 raft_consensus.cc:2804] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:06.188375  5081 raft_consensus.cc:697] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:06.189409  5081 consensus_queue.cc:237] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:06.201120  3903 catalog_manager.cc:5697] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 25902f00fb5d41679245f23dd19654aa (127.26.123.2) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.285351  5065 raft_consensus.cc:993] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:06.286019  5063 raft_consensus.cc:1081] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:06.287550  4904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51332
I20260709 12:21:06.288049  4904 raft_consensus.cc:493] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:06.288393  4904 raft_consensus.cc:3060] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.295914  4904 raft_consensus.cc:515] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:21:06.297472  4904 leader_election.cc:290] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:21:06.298370  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:06.298909  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ae1da0dfadb4b17bebb0c20ce3ec355" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:06.299063  4080 raft_consensus.cc:3060] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.299544  4149 raft_consensus.cc:3055] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:06.299861  4149 raft_consensus.cc:740] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:21:06.300664  4149 consensus_queue.cc:260] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:21:06.302170  4149 raft_consensus.cc:3060] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.309185  4080 raft_consensus.cc:2468] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:06.310441  4830 leader_election.cc:304] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:21:06.311299  5081 raft_consensus.cc:2804] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:06.311820  5081 raft_consensus.cc:697] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:06.312003  4149 raft_consensus.cc:2468] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:06.312789  5081 consensus_queue.cc:237] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:21:06.322861  3903 catalog_manager.cc:5697] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 6994df41341e478d89f7c105f550ff93 (127.26.123.3) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.352288  5064 raft_consensus.cc:993] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa: : Instructing follower 72940e4e52944adb81608e286ad58a59 to start an election
I20260709 12:21:06.352885  5068 raft_consensus.cc:1081] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Signalling peer 72940e4e52944adb81608e286ad58a59 to start an election
I20260709 12:21:06.354557  4221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959"
dest_uuid: "72940e4e52944adb81608e286ad58a59"
 from {username='slave'} at 127.0.0.1:58246
I20260709 12:21:06.355020  4221 raft_consensus.cc:493] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:06.355262  4221 raft_consensus.cc:3060] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.361340  4221 raft_consensus.cc:515] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:06.363539  4221 leader_election.cc:290] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:21:06.364439  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:06.365127  4080 raft_consensus.cc:3055] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:06.365365  4080 raft_consensus.cc:740] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:06.364545  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74b93cd1c6f94fdf869528b9411a7959" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:06.366416  4149 raft_consensus.cc:3060] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.366271  4080 consensus_queue.cc:260] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:06.367736  4080 raft_consensus.cc:3060] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.376437  4080 raft_consensus.cc:2468] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 2.
I20260709 12:21:06.377758  4194 leader_election.cc:304] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:06.378684  5069 raft_consensus.cc:2804] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:06.379196  5069 raft_consensus.cc:697] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:21:06.380153  5069 consensus_queue.cc:237] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:06.382622  4149 raft_consensus.cc:2468] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 2.
I20260709 12:21:06.393335  3903 catalog_manager.cc:5697] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 1 to 2, leader changed from 25902f00fb5d41679245f23dd19654aa (127.26.123.2) to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:06.450937  5064 raft_consensus.cc:993] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:06.451517  5068 raft_consensus.cc:1081] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 3 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:06.453192  4904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24281667f1364c6fac542e54ed413502"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51316
I20260709 12:21:06.453855  4904 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:06.454149  4904 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:06.460229  4904 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:06.461875  4904 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 4 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:06.462663  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:06.462968  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24281667f1364c6fac542e54ed413502" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:06.463261  4221 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:06.463559  4080 raft_consensus.cc:3055] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:06.463915  4080 raft_consensus.cc:740] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:06.464668  4080 consensus_queue.cc:260] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:06.465713  4080 raft_consensus.cc:3060] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:06.469161  4221 raft_consensus.cc:2468] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 4.
I20260709 12:21:06.470752  4830 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:06.471728  5081 raft_consensus.cc:2804] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:06.471899  4080 raft_consensus.cc:2468] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 4.
I20260709 12:21:06.472290  5081 raft_consensus.cc:697] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 4 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:06.473398  5081 consensus_queue.cc:237] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:06.485351  3903 catalog_manager.cc:5697] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 3 to 4, leader changed from 25902f00fb5d41679245f23dd19654aa (127.26.123.2) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 4 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.514667  5064 raft_consensus.cc:993] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa: : Instructing follower 72940e4e52944adb81608e286ad58a59 to start an election
I20260709 12:21:06.515229  5068 raft_consensus.cc:1081] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Signalling peer 72940e4e52944adb81608e286ad58a59 to start an election
I20260709 12:21:06.516887  4223 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "724697c71b61499aa0daadd2e66af805"
dest_uuid: "72940e4e52944adb81608e286ad58a59"
 from {username='slave'} at 127.0.0.1:58246
I20260709 12:21:06.517403  4223 raft_consensus.cc:493] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:06.517792  4223 raft_consensus.cc:3060] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.525209  4223 raft_consensus.cc:515] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:06.527259  4223 leader_election.cc:290] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:21:06.528086  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "724697c71b61499aa0daadd2e66af805" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:06.528482  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "724697c71b61499aa0daadd2e66af805" candidate_uuid: "72940e4e52944adb81608e286ad58a59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:06.528748  4079 raft_consensus.cc:3055] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:06.529174  4149 raft_consensus.cc:3060] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.529181  4079 raft_consensus.cc:740] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:06.530288  4079 consensus_queue.cc:260] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:06.531488  4079 raft_consensus.cc:3060] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.536932  4149 raft_consensus.cc:2468] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 2.
I20260709 12:21:06.540817  4079 raft_consensus.cc:2468] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72940e4e52944adb81608e286ad58a59 in term 2.
I20260709 12:21:06.541095  4193 leader_election.cc:304] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6994df41341e478d89f7c105f550ff93, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:06.542137  5069 raft_consensus.cc:2804] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:06.542815  5069 raft_consensus.cc:697] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Becoming Leader. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:21:06.545058  5069 consensus_queue.cc:237] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:06.558547  4223 raft_consensus.cc:1275] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.559789  4152 raft_consensus.cc:1275] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.559855  5064 consensus_queue.cc:1048] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:06.560809  3903 catalog_manager.cc:5697] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 reported cstate change: term changed from 1 to 2, leader changed from 25902f00fb5d41679245f23dd19654aa (127.26.123.2) to 72940e4e52944adb81608e286ad58a59 (127.26.123.4). New cstate: current_term: 2 leader_uuid: "72940e4e52944adb81608e286ad58a59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:06.561918  5064 consensus_queue.cc:1048] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:06.629832  5064 raft_consensus.cc:993] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:06.630374  5064 raft_consensus.cc:1081] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:06.631933  4904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51316
I20260709 12:21:06.632457  4904 raft_consensus.cc:493] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:06.632853  4904 raft_consensus.cc:3060] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.639276  4904 raft_consensus.cc:515] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:06.641283  4904 leader_election.cc:290] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:06.642978  4223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:06.642802  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0991ca4c9bd47b393e9566d1a6ecfc5" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:06.643582  4223 raft_consensus.cc:3060] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.643648  4079 raft_consensus.cc:3055] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:06.644052  4079 raft_consensus.cc:740] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:06.644933  4079 consensus_queue.cc:260] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:06.646385  4079 raft_consensus.cc:3060] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:06.652160  4223 raft_consensus.cc:2468] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:06.652947  4080 raft_consensus.cc:1275] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.653431  4830 leader_election.cc:304] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:06.654337  5081 raft_consensus.cc:2804] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:06.654619  4079 raft_consensus.cc:2468] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:06.654953  5082 consensus_queue.cc:1048] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:06.655625  5081 raft_consensus.cc:697] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:06.656728  5081 consensus_queue.cc:237] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:06.670845  3900 catalog_manager.cc:5697] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 25902f00fb5d41679245f23dd19654aa (127.26.123.2) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:06.676250  4223 raft_consensus.cc:1275] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.678299  5081 consensus_queue.cc:1048] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:06.732208  4149 raft_consensus.cc:1275] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.733319  5081 consensus_queue.cc:1048] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:06.745726  4079 raft_consensus.cc:1275] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.749213  5082 consensus_queue.cc:1048] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:06.794251  4152 raft_consensus.cc:1275] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.795694  5087 consensus_queue.cc:1048] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:06.806284  4079 raft_consensus.cc:1275] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:06.807688  5087 consensus_queue.cc:1048] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:06.897248  4079 raft_consensus.cc:1275] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 4 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:06.898694  5082 consensus_queue.cc:1048] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:06.917711  4223 raft_consensus.cc:1275] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 4 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:06.922675  5081 consensus_queue.cc:1048] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.044261  4149 raft_consensus.cc:1275] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.045545  5069 consensus_queue.cc:1048] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.055617  4079 raft_consensus.cc:1275] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 72940e4e52944adb81608e286ad58a59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.058388  5066 consensus_queue.cc:1048] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.103329 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:07.149590  4149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9"
dest_uuid: "6994df41341e478d89f7c105f550ff93"
mode: GRACEFUL
new_leader_uuid: "25902f00fb5d41679245f23dd19654aa"
 from {username='slave'} at 127.0.0.1:38048
I20260709 12:21:07.150377  4149 raft_consensus.cc:606] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Received request to transfer leadership to TS 25902f00fb5d41679245f23dd19654aa
I20260709 12:21:07.155381  4149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da"
dest_uuid: "6994df41341e478d89f7c105f550ff93"
mode: GRACEFUL
new_leader_uuid: "25902f00fb5d41679245f23dd19654aa"
 from {username='slave'} at 127.0.0.1:38048
I20260709 12:21:07.155931  4149 raft_consensus.cc:606] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Received request to transfer leadership to TS 25902f00fb5d41679245f23dd19654aa
I20260709 12:21:07.158671  4220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8"
dest_uuid: "72940e4e52944adb81608e286ad58a59"
mode: GRACEFUL
new_leader_uuid: "25902f00fb5d41679245f23dd19654aa"
 from {username='slave'} at 127.0.0.1:46714
I20260709 12:21:07.159288  4220 raft_consensus.cc:606] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Received request to transfer leadership to TS 25902f00fb5d41679245f23dd19654aa
I20260709 12:21:07.164760  4079 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:42446
I20260709 12:21:07.165319  4079 raft_consensus.cc:606] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:07.169337  4149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07"
dest_uuid: "6994df41341e478d89f7c105f550ff93"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:38048
I20260709 12:21:07.169781  5064 raft_consensus.cc:993] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:07.170109  4149 raft_consensus.cc:606] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:07.170358  5068 raft_consensus.cc:1081] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:07.172308  4919 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51316
I20260709 12:21:07.172766  4919 raft_consensus.cc:493] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:07.172693  4149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1"
dest_uuid: "6994df41341e478d89f7c105f550ff93"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:38048
I20260709 12:21:07.173352  4919 raft_consensus.cc:3060] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.173588  4149 raft_consensus.cc:606] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:07.182255 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:21:07.184420 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:07.185214 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:07.186483  4919 raft_consensus.cc:515] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:07.189507  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:07.189782  4919 leader_election.cc:290] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:07.190436  4220 raft_consensus.cc:3060] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.190932  4079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f07f64b93d44269e6666cc4482c983" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:07.191694  4079 raft_consensus.cc:3055] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:07.192019  4079 raft_consensus.cc:740] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:07.193004  4079 consensus_queue.cc:260] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:07.194485  4079 raft_consensus.cc:3060] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.200764  4220 raft_consensus.cc:2468] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:07.201185  4079 raft_consensus.cc:2468] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:07.202530  4830 leader_election.cc:304] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:21:07.203429  5081 raft_consensus.cc:2804] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:07.203879  5081 raft_consensus.cc:697] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:07.204764  5081 consensus_queue.cc:237] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:07.215942  3901 catalog_manager.cc:5697] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 25902f00fb5d41679245f23dd19654aa (127.26.123.2) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:07.216712  4079 raft_consensus.cc:1275] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.218554  5081 consensus_queue.cc:1048] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.251408  4220 raft_consensus.cc:1275] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.253667  5081 consensus_queue.cc:1048] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.320353  5071 raft_consensus.cc:993] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:07.320895  5065 raft_consensus.cc:1081] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:07.322618  4903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51332
I20260709 12:21:07.323252  4903 raft_consensus.cc:493] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:07.323421  5065 raft_consensus.cc:993] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93: : Instructing follower 25902f00fb5d41679245f23dd19654aa to start an election
I20260709 12:21:07.323704  4903 raft_consensus.cc:3060] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:07.323956  5063 raft_consensus.cc:1081] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Signalling peer 25902f00fb5d41679245f23dd19654aa to start an election
I20260709 12:21:07.327586  4080 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
 from {username='slave'} at 127.0.0.1:53996
I20260709 12:21:07.328173  4080 raft_consensus.cc:493] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:07.328506  4080 raft_consensus.cc:3060] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.331622  4903 raft_consensus.cc:515] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:07.334964  4903 leader_election.cc:290] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 3 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:21:07.335204  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:07.335906  4149 raft_consensus.cc:3055] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:07.336236  4149 raft_consensus.cc:740] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:21:07.337132  4149 consensus_queue.cc:260] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:07.337772  4080 raft_consensus.cc:515] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:07.336078  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:07.338737  4220 raft_consensus.cc:3060] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:07.342937  4080 leader_election.cc:290] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:21:07.343042  4149 raft_consensus.cc:3060] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:07.344439  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:07.347218  4221 raft_consensus.cc:3060] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.349611  4220 raft_consensus.cc:2468] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 3.
I20260709 12:21:07.349853  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "581a73aef577436b9cc60234f338e9b9" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:07.350555  4152 raft_consensus.cc:3055] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:07.350873  4152 raft_consensus.cc:740] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:21:07.351055  4830 leader_election.cc:304] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:07.351697  4152 consensus_queue.cc:260] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:07.352080  5081 raft_consensus.cc:2804] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:07.352722  5081 raft_consensus.cc:697] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 3 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:07.353160  4152 raft_consensus.cc:3060] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.353672  5081 consensus_queue.cc:237] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:07.359390  4221 raft_consensus.cc:2468] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:21:07.368095  4152 raft_consensus.cc:2468] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:21:07.368640  4046 leader_election.cc:304] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:07.369339  4149 raft_consensus.cc:2468] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 3.
I20260709 12:21:07.369609  5064 raft_consensus.cc:2804] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:07.370216  5064 raft_consensus.cc:697] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:07.369445  5106 raft_consensus.cc:993] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59: : Instructing follower 25902f00fb5d41679245f23dd19654aa to start an election
I20260709 12:21:07.371069  5064 consensus_queue.cc:237] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:07.371784  5106 raft_consensus.cc:1081] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Signalling peer 25902f00fb5d41679245f23dd19654aa to start an election
I20260709 12:21:07.374799  4079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
 from {username='slave'} at 127.0.0.1:53998
I20260709 12:21:07.375365  4079 raft_consensus.cc:493] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:07.375720  4079 raft_consensus.cc:3060] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.389125  3900 catalog_manager.cc:5697] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 2 to 3, leader changed from 6994df41341e478d89f7c105f550ff93 (127.26.123.3) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 3 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:07.394542  3901 catalog_manager.cc:5697] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 1 to 2, leader changed from 6994df41341e478d89f7c105f550ff93 (127.26.123.3) to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 2 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:07.401154  4079 raft_consensus.cc:515] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:07.404112  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:07.404742  4149 raft_consensus.cc:3060] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.404934  4079 leader_election.cc:290] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:21:07.406742  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d4786cea7ba49e49a9800f80ef684b8" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:07.407416  4221 raft_consensus.cc:3055] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:07.407723  4221 raft_consensus.cc:740] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:21:07.408492  4221 consensus_queue.cc:260] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:07.409503  4221 raft_consensus.cc:3060] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.413806  4149 raft_consensus.cc:2468] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:21:07.415129  4045 leader_election.cc:304] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:21:07.415555  4221 raft_consensus.cc:2468] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:21:07.415956  5064 raft_consensus.cc:2804] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:07.416441  5064 raft_consensus.cc:697] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:07.417274  5064 consensus_queue.cc:237] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:07.428005  3901 catalog_manager.cc:5697] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 1 to 2, leader changed from 72940e4e52944adb81608e286ad58a59 (127.26.123.4) to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 2 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:07.613252  5063 raft_consensus.cc:993] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93: : Instructing follower 25902f00fb5d41679245f23dd19654aa to start an election
I20260709 12:21:07.613684  5065 raft_consensus.cc:1081] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Signalling peer 25902f00fb5d41679245f23dd19654aa to start an election
I20260709 12:21:07.615128  4079 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
 from {username='slave'} at 127.0.0.1:53996
I20260709 12:21:07.615605  4079 raft_consensus.cc:493] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:07.615875  4079 raft_consensus.cc:3060] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.622174  4079 raft_consensus.cc:515] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:07.623993  4079 leader_election.cc:290] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:21:07.624600  4149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:07.625029  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ded6cf162b34debb6ea7d81c85685da" candidate_uuid: "25902f00fb5d41679245f23dd19654aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:07.625305  4149 raft_consensus.cc:3055] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:07.625654  4220 raft_consensus.cc:3060] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.625664  4149 raft_consensus.cc:740] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:21:07.626791  4149 consensus_queue.cc:260] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:07.628085  4149 raft_consensus.cc:3060] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.633631  4220 raft_consensus.cc:2468] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:21:07.636511  4149 raft_consensus.cc:2468] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25902f00fb5d41679245f23dd19654aa in term 2.
I20260709 12:21:07.637642  4046 leader_election.cc:304] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:07.639004  5117 raft_consensus.cc:2804] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:07.639514  5117 raft_consensus.cc:697] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Becoming Leader. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:07.643133  5117 consensus_queue.cc:237] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:07.650048  4220 raft_consensus.cc:1275] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.651480  5081 consensus_queue.cc:1048] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.656107  3901 catalog_manager.cc:5697] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa reported cstate change: term changed from 1 to 2, leader changed from 6994df41341e478d89f7c105f550ff93 (127.26.123.3) to 25902f00fb5d41679245f23dd19654aa (127.26.123.2). New cstate: current_term: 2 leader_uuid: "25902f00fb5d41679245f23dd19654aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:07.661963  4079 raft_consensus.cc:1275] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.666260  5081 consensus_queue.cc:1048] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.681437  5063 raft_consensus.cc:993] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:07.681994  5063 raft_consensus.cc:1081] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:07.683578  4903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51332
I20260709 12:21:07.684162  4903 raft_consensus.cc:493] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:07.684525  4903 raft_consensus.cc:3060] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.707862  4903 raft_consensus.cc:515] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:07.726795  4903 leader_election.cc:290] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:21:07.731457  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:07.732126  4154 raft_consensus.cc:3055] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:07.732429  4154 raft_consensus.cc:740] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:21:07.733256  4154 consensus_queue.cc:260] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:07.738123  4154 raft_consensus.cc:3060] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.730834  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:07.739423  4220 raft_consensus.cc:3060] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:07.747884  4154 raft_consensus.cc:2468] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:07.749122  4829 leader_election.cc:304] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:21:07.750543  5121 raft_consensus.cc:2804] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:07.751533  5121 raft_consensus.cc:697] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:07.755085  4220 raft_consensus.cc:2468] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:07.755079  5121 consensus_queue.cc:237] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:07.779117  3901 catalog_manager.cc:5697] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 6994df41341e478d89f7c105f550ff93 (127.26.123.3) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:07.789417  4154 raft_consensus.cc:1275] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:07.790616  5121 consensus_queue.cc:1048] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.799825  4220 raft_consensus.cc:1275] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:07.801672  5114 consensus_queue.cc:1048] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:07.887511  4220 raft_consensus.cc:1275] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.888731  5117 consensus_queue.cc:1048] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:07.903188  4154 raft_consensus.cc:1275] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:07.905093  5117 consensus_queue.cc:1048] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.023149  4154 raft_consensus.cc:1275] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.024704  5064 consensus_queue.cc:1048] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.032554  4220 raft_consensus.cc:1275] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.034533  5064 consensus_queue.cc:1048] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.077658  4154 raft_consensus.cc:1275] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.079051  5064 consensus_queue.cc:1048] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.089564  4220 raft_consensus.cc:1275] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 25902f00fb5d41679245f23dd19654aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.091660  5064 consensus_queue.cc:1048] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.167449  4154 raft_consensus.cc:1275] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.168846  5114 consensus_queue.cc:1048] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.169013  4221 raft_consensus.cc:1275] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.170650  5094 consensus_queue.cc:1048] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:08.244590 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:08.296919  4077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "389645cf278940c6803751b614ba1dfa"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:42446
I20260709 12:21:08.297580  4077 raft_consensus.cc:606] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:08.301088  4154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2"
dest_uuid: "6994df41341e478d89f7c105f550ff93"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:38048
I20260709 12:21:08.301585  4154 raft_consensus.cc:606] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:08.304621  4221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c"
dest_uuid: "72940e4e52944adb81608e286ad58a59"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:46714
I20260709 12:21:08.305302  4221 raft_consensus.cc:606] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:08.309140  4080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81ff51ff620749a591f77c58c287e374"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:42446
I20260709 12:21:08.309871  4080 raft_consensus.cc:606] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:08.312719  4154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd"
dest_uuid: "6994df41341e478d89f7c105f550ff93"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:38048
I20260709 12:21:08.315429  4154 raft_consensus.cc:606] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:08.321375  4080 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9"
dest_uuid: "25902f00fb5d41679245f23dd19654aa"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:42446
I20260709 12:21:08.322026  4080 raft_consensus.cc:606] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:08.323203 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:21:08.324810 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:08.325481 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:08.334384  5131 raft_consensus.cc:993] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.335022  5131 raft_consensus.cc:1081] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.336956  4903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "389645cf278940c6803751b614ba1dfa"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51316
I20260709 12:21:08.337572  4903 raft_consensus.cc:493] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:08.337944  4903 raft_consensus.cc:3060] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.345620  4903 raft_consensus.cc:515] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:08.347411  4903 leader_election.cc:290] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:08.348582  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389645cf278940c6803751b614ba1dfa" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:08.348438  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "389645cf278940c6803751b614ba1dfa" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:08.349251  4080 raft_consensus.cc:3055] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:08.349647  4221 raft_consensus.cc:3060] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.349648  4080 raft_consensus.cc:740] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:08.350901  4080 consensus_queue.cc:260] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:08.352119  4080 raft_consensus.cc:3060] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.359086  4080 raft_consensus.cc:2468] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.360410  4830 leader_election.cc:304] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:21:08.361392  5114 raft_consensus.cc:2804] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:08.361917  5114 raft_consensus.cc:697] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:08.362804  5114 consensus_queue.cc:237] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:08.363656  4221 raft_consensus.cc:2468] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.374181  3900 catalog_manager.cc:5697] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 25902f00fb5d41679245f23dd19654aa (127.26.123.2) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:08.387260  5065 raft_consensus.cc:993] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.387804  5071 raft_consensus.cc:1081] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.389671  4903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51332
I20260709 12:21:08.390337  4903 raft_consensus.cc:493] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:08.390761  4903 raft_consensus.cc:3060] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.398147  4903 raft_consensus.cc:515] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:08.399984  4903 leader_election.cc:290] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:21:08.400743  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:08.401031  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98ec68263b9c4d5dbdff0fbfd53629cd" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:08.401410  4154 raft_consensus.cc:3055] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:08.401592  4221 raft_consensus.cc:3060] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.401804  4154 raft_consensus.cc:740] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:21:08.402580  4154 consensus_queue.cc:260] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:08.403584  4154 raft_consensus.cc:3060] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.408885  4221 raft_consensus.cc:2468] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.410445  4830 leader_election.cc:304] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:08.411250  5114 raft_consensus.cc:2804] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:08.411767  5114 raft_consensus.cc:697] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:08.412685  4154 raft_consensus.cc:2468] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.412681  5114 consensus_queue.cc:237] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:08.424504  3900 catalog_manager.cc:5697] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 6994df41341e478d89f7c105f550ff93 (127.26.123.3) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.494252  5065 raft_consensus.cc:993] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.494789  5071 raft_consensus.cc:1081] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.496467  4903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51332
I20260709 12:21:08.497071  4903 raft_consensus.cc:493] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:08.497363  4903 raft_consensus.cc:3060] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.503475  4903 raft_consensus.cc:515] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:08.505726  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:08.505990  4903 leader_election.cc:290] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:21:08.506481  4154 raft_consensus.cc:3055] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:08.506897  4154 raft_consensus.cc:740] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6994df41341e478d89f7c105f550ff93, State: Running, Role: LEADER
I20260709 12:21:08.507740  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72ce1dc8db9548058ab94a9115c111f2" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:08.508383  4221 raft_consensus.cc:3060] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.507783  4154 consensus_queue.cc:260] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:08.509399  4154 raft_consensus.cc:3060] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.517117  4154 raft_consensus.cc:2468] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.518462  4829 leader_election.cc:304] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:21:08.519464  5094 raft_consensus.cc:2804] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:08.516233  4221 raft_consensus.cc:2468] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.519994  5094 raft_consensus.cc:697] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:08.520982  5094 consensus_queue.cc:237] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:08.544471  3900 catalog_manager.cc:5697] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 6994df41341e478d89f7c105f550ff93 (127.26.123.3) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.583343  5087 raft_consensus.cc:993] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.584322  5087 raft_consensus.cc:1081] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.586591  4903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51314
I20260709 12:21:08.587307  4903 raft_consensus.cc:493] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:08.587728  4903 raft_consensus.cc:3060] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.597896  4903 raft_consensus.cc:515] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:08.599531  4903 leader_election.cc:290] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389)
I20260709 12:21:08.600143  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:08.600528  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d413f51fcd4460aee2adb16b951a7c" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:08.600806  4152 raft_consensus.cc:3060] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.601105  4221 raft_consensus.cc:3055] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:08.601380  4221 raft_consensus.cc:740] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:21:08.602064  4221 consensus_queue.cc:260] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:08.603051  4221 raft_consensus.cc:3060] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.606501  4152 raft_consensus.cc:2468] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.607877  4829 leader_election.cc:304] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:21:08.608717  5081 raft_consensus.cc:2804] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:08.609275  5081 raft_consensus.cc:697] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:08.609807  4221 raft_consensus.cc:2468] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.610308  5081 consensus_queue.cc:237] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:08.614806  5064 raft_consensus.cc:993] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.615437  5131 raft_consensus.cc:1081] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.616986  4903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51316
I20260709 12:21:08.617456  4903 raft_consensus.cc:493] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:08.617791  4903 raft_consensus.cc:3060] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.621402  3900 catalog_manager.cc:5697] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 72940e4e52944adb81608e286ad58a59 (127.26.123.4) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.624341  4903 raft_consensus.cc:515] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:21:08.626812  4903 leader_election.cc:290] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:21:08.627529  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:08.628221  4080 raft_consensus.cc:3055] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:08.628542  4080 raft_consensus.cc:740] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:08.628808  4152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0bab1596ed74079b6efee7c9fefe4f9" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6994df41341e478d89f7c105f550ff93"
I20260709 12:21:08.629441  4152 raft_consensus.cc:3060] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.629280  4080 consensus_queue.cc:260] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:21:08.630998  4080 raft_consensus.cc:3060] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:08.636281  4152 raft_consensus.cc:2468] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.638685  4829 leader_election.cc:304] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93; no voters: 
I20260709 12:21:08.639679  4080 raft_consensus.cc:2468] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:08.640064  5081 raft_consensus.cc:2804] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:08.640606  5081 raft_consensus.cc:697] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:08.641460  5081 consensus_queue.cc:237] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } }
I20260709 12:21:08.651456  3900 catalog_manager.cc:5697] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 25902f00fb5d41679245f23dd19654aa (127.26.123.2) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:08.799490  5064 raft_consensus.cc:993] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.799906  5131 raft_consensus.cc:1081] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:08.801412  4903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81ff51ff620749a591f77c58c287e374"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51316
I20260709 12:21:08.801945  4903 raft_consensus.cc:493] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:08.802309  4903 raft_consensus.cc:3060] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:08.809278  4903 raft_consensus.cc:515] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:08.812566  4903 leader_election.cc:290] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 3 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:08.813584  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:08.814165  4080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81ff51ff620749a591f77c58c287e374" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:08.815434  4080 raft_consensus.cc:3055] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:08.815730  4080 raft_consensus.cc:740] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 25902f00fb5d41679245f23dd19654aa, State: Running, Role: LEADER
I20260709 12:21:08.816509  4080 consensus_queue.cc:260] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:08.817909  4080 raft_consensus.cc:3060] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:08.819214  4221 raft_consensus.cc:3060] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:08.832027  4221 raft_consensus.cc:2468] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 3.
I20260709 12:21:08.832235  4080 raft_consensus.cc:2468] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 3.
I20260709 12:21:08.833230  4830 leader_election.cc:304] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59; no voters: 
I20260709 12:21:08.835065  5081 raft_consensus.cc:2804] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:08.835397  4221 raft_consensus.cc:1275] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.836216  5081 raft_consensus.cc:697] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 3 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:08.837075  5094 consensus_queue.cc:1048] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.838156  5081 consensus_queue.cc:237] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:08.850735  4080 raft_consensus.cc:1275] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.850649  3900 catalog_manager.cc:5697] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 2 to 3, leader changed from 25902f00fb5d41679245f23dd19654aa (127.26.123.2) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 3 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:08.852190  5094 consensus_queue.cc:1048] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.953333  4220 raft_consensus.cc:1275] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.955168  5094 consensus_queue.cc:1048] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:08.973032  4154 raft_consensus.cc:1275] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:08.974488  5094 consensus_queue.cc:1048] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.004941  4154 raft_consensus.cc:1275] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:09.006359  5094 consensus_queue.cc:1048] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.026252  4220 raft_consensus.cc:1275] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:09.027940  5081 consensus_queue.cc:1048] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.057723  4154 raft_consensus.cc:1275] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:09.059381  5081 consensus_queue.cc:1048] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.074206  4077 raft_consensus.cc:1275] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:09.076958  5094 consensus_queue.cc:1048] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.216233  4154 raft_consensus.cc:1275] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:09.217867  5081 consensus_queue.cc:1048] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.228647  4220 raft_consensus.cc:1275] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:09.230479  5081 consensus_queue.cc:1048] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.321545  4077 raft_consensus.cc:1275] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:09.322857  5081 consensus_queue.cc:1048] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:09.338917  4220 raft_consensus.cc:1275] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:09.340927  5081 consensus_queue.cc:1048] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:09.374815 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:09.417657  4221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5"
dest_uuid: "72940e4e52944adb81608e286ad58a59"
mode: GRACEFUL
new_leader_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:46714
I20260709 12:21:09.418342  4221 raft_consensus.cc:606] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Received request to transfer leadership to TS 41009fde51f045c49be6461b76a27543
I20260709 12:21:09.419850 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:21:09.422098 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:09.423004 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:09.938021  5106 raft_consensus.cc:993] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59: : Instructing follower 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:09.938499  5087 raft_consensus.cc:1081] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Signalling peer 41009fde51f045c49be6461b76a27543 to start an election
I20260709 12:21:09.939932  4919 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5"
dest_uuid: "41009fde51f045c49be6461b76a27543"
 from {username='slave'} at 127.0.0.1:51314
I20260709 12:21:09.940397  4919 raft_consensus.cc:493] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:09.940742  4919 raft_consensus.cc:3060] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:09.947140  4919 raft_consensus.cc:515] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:09.948726  4919 leader_election.cc:290] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Requested vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:09.949551  4221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72940e4e52944adb81608e286ad58a59"
I20260709 12:21:09.949707  4077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefb699ae684288b994f3255f4c89b5" candidate_uuid: "41009fde51f045c49be6461b76a27543" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "25902f00fb5d41679245f23dd19654aa"
I20260709 12:21:09.950383  4221 raft_consensus.cc:3055] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:09.950428  4077 raft_consensus.cc:3060] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:09.950923  4221 raft_consensus.cc:740] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 72940e4e52944adb81608e286ad58a59, State: Running, Role: LEADER
I20260709 12:21:09.951754  4221 consensus_queue.cc:260] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:09.953076  4221 raft_consensus.cc:3060] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:09.958235  4077 raft_consensus.cc:2468] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:09.959647  4830 leader_election.cc:304] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543; no voters: 
I20260709 12:21:09.960577  4221 raft_consensus.cc:2468] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41009fde51f045c49be6461b76a27543 in term 2.
I20260709 12:21:09.960932  5094 raft_consensus.cc:2804] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:09.961751  5094 raft_consensus.cc:697] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Becoming Leader. State: Replica: 41009fde51f045c49be6461b76a27543, State: Running, Role: LEADER
I20260709 12:21:09.962666  5094 consensus_queue.cc:237] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:09.974577  3903 catalog_manager.cc:5697] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 reported cstate change: term changed from 1 to 2, leader changed from 72940e4e52944adb81608e286ad58a59 (127.26.123.4) to 41009fde51f045c49be6461b76a27543 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "41009fde51f045c49be6461b76a27543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:10.467554 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:10.508898 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:10.510596 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:10.511147 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:10.552513 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:10.571173  4077 raft_consensus.cc:1275] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:10.572610  5114 consensus_queue.cc:1048] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:10.578459  4220 raft_consensus.cc:1275] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Refusing update from remote peer 41009fde51f045c49be6461b76a27543: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:10.580014  5114 consensus_queue.cc:1048] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:10.601130 27116 tablet_server.cc:179] TabletServer@127.26.123.1:41077 shutting down...
W20260709 12:21:10.607457  4194 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 220 similar messages]
W20260709 12:21:10.615662  4194 consensus_peers.cc:597] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.618394  4194 consensus_peers.cc:597] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.637243  4194 consensus_peers.cc:597] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.651376  4120 consensus_peers.cc:597] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.660435  4194 consensus_peers.cc:597] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.661906  4194 consensus_peers.cc:597] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.662488  4194 consensus_peers.cc:597] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.669546  4120 consensus_peers.cc:597] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.677466  4046 consensus_peers.cc:597] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.700700  4120 consensus_peers.cc:597] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.700872 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:21:10.701591  4120 consensus_peers.cc:597] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.702486 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.703599 27116 raft_consensus.cc:2243] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.704690 27116 raft_consensus.cc:2272] T fcb63c6a1eed482f9d368536f4aaadf1 P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.707911 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.708917 27116 raft_consensus.cc:2243] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.709455 27116 raft_consensus.cc:2272] T 7f1c0613280b411f98f4807034b0a1cb P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.712631 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.713624 27116 raft_consensus.cc:2243] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.714165 27116 raft_consensus.cc:2272] T e08603ff2d9145e0b85e975bac56312c P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.717255 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.718261 27116 raft_consensus.cc:2243] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.718775 27116 raft_consensus.cc:2272] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.721907 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.722630 27116 raft_consensus.cc:2243] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.723119 27116 raft_consensus.cc:2272] T 7c72e9bb69f746c191320d2437c3ce19 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:10.724494  4194 consensus_peers.cc:597] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.726074 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.726881 27116 raft_consensus.cc:2243] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:21:10.727083  4194 consensus_peers.cc:597] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.727763 27116 raft_consensus.cc:2272] T ac91fc64d6874b49a749e20bbf04ad07 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.730337 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.730999 27116 raft_consensus.cc:2243] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.731848 27116 raft_consensus.cc:2272] T 389645cf278940c6803751b614ba1dfa P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.734138 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.734786 27116 raft_consensus.cc:2243] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.735242 27116 raft_consensus.cc:2272] T 18dadd11589447aea90415c88b77e312 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.737586 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.738235 27116 raft_consensus.cc:2243] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.738677 27116 raft_consensus.cc:2272] T 01529dc6d8944b9697b9a87c4e4bf450 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.740851 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.741449 27116 raft_consensus.cc:2243] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.741909 27116 raft_consensus.cc:2272] T ebb6ad79685348f4a82858f9f7844c48 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.744136 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.744801 27116 raft_consensus.cc:2243] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.745653 27116 raft_consensus.cc:2272] T 72ce1dc8db9548058ab94a9115c111f2 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.747983 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.748734 27116 raft_consensus.cc:2243] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.749208 27116 raft_consensus.cc:2272] T 9233488a2b6745f8a012d9edc348333a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.752218 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.753161 27116 raft_consensus.cc:2243] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.753676 27116 raft_consensus.cc:2272] T 73f2a3fbee56421aa8768a4d7f275ba9 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.756359 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.757025 27116 raft_consensus.cc:2243] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.757457 27116 raft_consensus.cc:2272] T 11d7a192af654f9fa50d695be708ca81 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.759892 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.760551 27116 raft_consensus.cc:2243] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.760979 27116 raft_consensus.cc:2272] T 64bc045a9f78405ebd413d12e6236f61 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.763352 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.763967 27116 raft_consensus.cc:2243] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.764549 27116 raft_consensus.cc:2272] T 75948129b8264d75972f4660893bd024 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.766857 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.767477 27116 raft_consensus.cc:2243] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.767936 27116 raft_consensus.cc:2272] T 65e4899b95af40fd9836e281f4c23e17 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.770771 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.771612 27116 raft_consensus.cc:2243] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.772644 27116 raft_consensus.cc:2272] T f0bab1596ed74079b6efee7c9fefe4f9 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:10.773209  4194 consensus_peers.cc:597] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.774971 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.775763 27116 raft_consensus.cc:2243] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.776593 27116 raft_consensus.cc:2272] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.779249 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.779872 27116 raft_consensus.cc:2243] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.780321 27116 raft_consensus.cc:2272] T 26349fc0d5664798a248220d968f5269 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.782673 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.783319 27116 raft_consensus.cc:2243] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.783753 27116 raft_consensus.cc:2272] T 3331199805b64fa8bf0d5e8847ad17b8 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.786496 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.787111 27116 raft_consensus.cc:2243] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.787887 27116 raft_consensus.cc:2272] T e203dae959394f4e90051ad4746a9291 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.790215 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.790824 27116 raft_consensus.cc:2243] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.791213 27116 raft_consensus.cc:2272] T 35f92fa80bd34e60924d5c77f79df421 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.793586 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.794238 27116 raft_consensus.cc:2243] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.795110 27116 raft_consensus.cc:2272] T 24281667f1364c6fac542e54ed413502 P 41009fde51f045c49be6461b76a27543 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.797323 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.798007 27116 raft_consensus.cc:2243] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.798889 27116 raft_consensus.cc:2272] T 76d413f51fcd4460aee2adb16b951a7c P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.801553 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.802486 27116 raft_consensus.cc:2243] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.803112 27116 raft_consensus.cc:2272] T cb8a7e4c51b446578c698dcf9a253001 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:10.805794  4046 consensus_peers.cc:597] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.806200 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.807149 27116 raft_consensus.cc:2243] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.808130 27116 raft_consensus.cc:2272] T a0f07f64b93d44269e6666cc4482c983 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.810802 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.811548 27116 raft_consensus.cc:2243] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.812543 27116 raft_consensus.cc:2272] T bcefb699ae684288b994f3255f4c89b5 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.816087 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.816895 27116 raft_consensus.cc:2243] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.817523 27116 raft_consensus.cc:2272] T 778cd7f085494ba1842e5f758d7f9828 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.820785 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.821485 27116 raft_consensus.cc:2243] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.822062 27116 raft_consensus.cc:2272] T c2adcdb890e0483095cb9923dce850a1 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.824313 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.825115 27116 raft_consensus.cc:2243] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.826174 27116 raft_consensus.cc:2272] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.828657 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.829267 27116 raft_consensus.cc:2243] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.829774 27116 raft_consensus.cc:2272] T a252087aa0c9426cb3067f2f9004dd29 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.832525 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.833302 27116 raft_consensus.cc:2243] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.833796 27116 raft_consensus.cc:2272] T 9fda924a0568433fa70ab72d4a0c3968 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.836050 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.836869 27116 raft_consensus.cc:2243] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.837344 27116 raft_consensus.cc:2272] T 91cdbe8d3035425abc337e6e27846e42 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.840025 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.840626 27116 raft_consensus.cc:2243] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.841048 27116 raft_consensus.cc:2272] T d14347d13af8448bb12300afb6a90639 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.843250 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.843823 27116 raft_consensus.cc:2243] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.844651 27116 raft_consensus.cc:2272] T 98ec68263b9c4d5dbdff0fbfd53629cd P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.847121 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.847679 27116 raft_consensus.cc:2243] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.848165 27116 raft_consensus.cc:2272] T 7fc90d55c744447aaa17e745cf58157c P 41009fde51f045c49be6461b76a27543 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.850484 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.851058 27116 raft_consensus.cc:2243] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.851459 27116 raft_consensus.cc:2272] T b08446fff0954c05b5311d3e76153e58 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.854334 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.855000 27116 raft_consensus.cc:2243] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.855597 27116 raft_consensus.cc:2272] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.858316 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.858887 27116 raft_consensus.cc:2243] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.859336 27116 raft_consensus.cc:2272] T d10e7c5b29b14b4e8c90d79a5201bf26 P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.861779 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.862461 27116 raft_consensus.cc:2243] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.863389 27116 raft_consensus.cc:2272] T 81ff51ff620749a591f77c58c287e374 P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.866255 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.867189 27116 raft_consensus.cc:2243] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.867673 27116 raft_consensus.cc:2272] T 48a9a743ab6141218e529314c318f16b P 41009fde51f045c49be6461b76a27543 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.870814 27116 tablet_replica.cc:333] stopping tablet replica
W20260709 12:21:10.871119  4046 consensus_peers.cc:597] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.871464 27116 raft_consensus.cc:2243] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:10.871834  4120 consensus_peers.cc:597] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.872181 27116 raft_consensus.cc:2272] T 6dd58e37cb1847738fdf9dd220c2280a P 41009fde51f045c49be6461b76a27543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.874794 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.875355 27116 raft_consensus.cc:2243] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.875866 27116 raft_consensus.cc:2272] T c07edbbf4ef64ddbbad929d20ce78c4e P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.878568 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:10.879321 27116 raft_consensus.cc:2243] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.879944 27116 raft_consensus.cc:2272] T 525557ea38a44858b86a97b0c444ecc4 P 41009fde51f045c49be6461b76a27543 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:10.890854  4046 consensus_peers.cc:597] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.906244  4120 consensus_peers.cc:597] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.911193  4194 consensus_peers.cc:597] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.917095  4046 consensus_peers.cc:597] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.917948  4046 consensus_peers.cc:597] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.922664  4046 consensus_peers.cc:597] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.925817  4120 consensus_peers.cc:597] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.939954  4120 consensus_peers.cc:597] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.953866  4120 consensus_peers.cc:597] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.981916  4120 consensus_peers.cc:597] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.990043  4120 consensus_peers.cc:597] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:10.990881 27116 tablet_server.cc:196] TabletServer@127.26.123.1:41077 shutdown complete.
W20260709 12:21:11.014675  4194 consensus_peers.cc:597] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.021065  4046 consensus_peers.cc:597] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.100546  4194 consensus_peers.cc:597] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 -> Peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Couldn't send request to peer 41009fde51f045c49be6461b76a27543. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.106765 27116 tablet_server.cc:179] TabletServer@127.26.123.2:0 shutting down...
W20260709 12:21:11.125130  4120 consensus_peers.cc:597] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.138787  4194 consensus_peers.cc:597] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.153118  4194 consensus_peers.cc:597] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.180526 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:11.181427 27116 tablet_replica.cc:333] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.182446 27116 raft_consensus.cc:2243] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.183099 27116 raft_consensus.cc:2272] T 24281667f1364c6fac542e54ed413502 P 25902f00fb5d41679245f23dd19654aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.185707 27116 tablet_replica.cc:333] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.186528 27116 raft_consensus.cc:2243] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.187422 27116 raft_consensus.cc:2272] T fe179cc2289146d98b10419917a7a36d P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.189951 27116 tablet_replica.cc:333] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.190591 27116 raft_consensus.cc:2243] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.191510 27116 raft_consensus.cc:2272] T 435b4d42ac35468f8a95973a14a9d6e3 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.193900 27116 tablet_replica.cc:333] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.194568 27116 raft_consensus.cc:2243] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.195566 27116 raft_consensus.cc:2272] T 581a73aef577436b9cc60234f338e9b9 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.197526  4120 consensus_peers.cc:597] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.198276 27116 tablet_replica.cc:333] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.199091 27116 raft_consensus.cc:2243] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.200001 27116 raft_consensus.cc:2272] T 35f92fa80bd34e60924d5c77f79df421 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.202593 27116 tablet_replica.cc:333] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.203303 27116 raft_consensus.cc:2243] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.203811 27116 raft_consensus.cc:2272] T 389645cf278940c6803751b614ba1dfa P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.206295 27116 tablet_replica.cc:333] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.207043 27116 raft_consensus.cc:2243] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.208195 27116 raft_consensus.cc:2272] T d10e7c5b29b14b4e8c90d79a5201bf26 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.210700 27116 tablet_replica.cc:333] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.211331 27116 raft_consensus.cc:2243] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.211853 27116 raft_consensus.cc:2272] T 724697c71b61499aa0daadd2e66af805 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.213990 27116 tablet_replica.cc:333] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.214583 27116 raft_consensus.cc:2243] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.215004 27116 raft_consensus.cc:2272] T 81ff51ff620749a591f77c58c287e374 P 25902f00fb5d41679245f23dd19654aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.217001 27116 tablet_replica.cc:333] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.217634 27116 raft_consensus.cc:2243] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.218065 27116 raft_consensus.cc:2272] T c2adcdb890e0483095cb9923dce850a1 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.220324 27116 tablet_replica.cc:333] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.220959 27116 raft_consensus.cc:2243] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.221453 27116 raft_consensus.cc:2272] T 778cd7f085494ba1842e5f758d7f9828 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.223585 27116 tablet_replica.cc:333] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.224198 27116 raft_consensus.cc:2243] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.225081 27116 raft_consensus.cc:2272] T cb8a7e4c51b446578c698dcf9a253001 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.227501 27116 tablet_replica.cc:333] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.228454 27116 raft_consensus.cc:2243] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.228992 27116 raft_consensus.cc:2272] T bcefb699ae684288b994f3255f4c89b5 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.232489 27116 tablet_replica.cc:333] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.233494 27116 raft_consensus.cc:2243] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.234025 27116 raft_consensus.cc:2272] T a0f07f64b93d44269e6666cc4482c983 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.236519 27116 tablet_replica.cc:333] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.237485 27116 raft_consensus.cc:2243] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.238117 27116 raft_consensus.cc:2272] T 9dc6c1656d03459babbca1538b0ae0ef P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.241235 27116 tablet_replica.cc:333] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.242166 27116 raft_consensus.cc:2243] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.243266 27116 raft_consensus.cc:2272] T 7e92ca9c3274410aa6ea1aaa11671569 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.246364 27116 tablet_replica.cc:333] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.247179 27116 raft_consensus.cc:2243] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.248140 27116 raft_consensus.cc:2272] T a252087aa0c9426cb3067f2f9004dd29 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.250890 27116 tablet_replica.cc:333] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.251753 27116 raft_consensus.cc:2243] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.252345 27116 raft_consensus.cc:2272] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.253715  4120 consensus_peers.cc:597] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.255000 27116 tablet_replica.cc:333] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.255752 27116 raft_consensus.cc:2243] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.256259 27116 raft_consensus.cc:2272] T 01529dc6d8944b9697b9a87c4e4bf450 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.258666 27116 tablet_replica.cc:333] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.259407 27116 raft_consensus.cc:2243] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.260332 27116 raft_consensus.cc:2272] T 38b7f39842e94931a878f3003c1353e1 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.262800 27116 tablet_replica.cc:333] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.263567 27116 raft_consensus.cc:2243] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.264038 27116 raft_consensus.cc:2272] T 6dd58e37cb1847738fdf9dd220c2280a P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.266796 27116 tablet_replica.cc:333] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.267575 27116 raft_consensus.cc:2243] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.268105 27116 raft_consensus.cc:2272] T c07edbbf4ef64ddbbad929d20ce78c4e P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.269198  4194 consensus_peers.cc:597] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.271013 27116 tablet_replica.cc:333] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.272064 27116 raft_consensus.cc:2243] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.272980 27116 raft_consensus.cc:2272] T 9d4786cea7ba49e49a9800f80ef684b8 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.276048 27116 tablet_replica.cc:333] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.276942 27116 raft_consensus.cc:2243] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.277546 27116 raft_consensus.cc:2272] T 3331199805b64fa8bf0d5e8847ad17b8 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.280602 27116 tablet_replica.cc:333] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.281540 27116 raft_consensus.cc:2243] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.282150 27116 raft_consensus.cc:2272] T e203dae959394f4e90051ad4746a9291 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.285122 27116 tablet_replica.cc:333] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.286028 27116 raft_consensus.cc:2243] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.287132 27116 raft_consensus.cc:2272] T 7ded6cf162b34debb6ea7d81c85685da P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.289659 27116 tablet_replica.cc:333] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.290321 27116 raft_consensus.cc:2243] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.290807 27116 raft_consensus.cc:2272] T e7536eba19a0467ab642af1d051f5b77 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.293392 27116 tablet_replica.cc:333] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.294045 27116 raft_consensus.cc:2243] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.294960 27116 raft_consensus.cc:2272] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.297305 27116 tablet_replica.cc:333] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.297946 27116 raft_consensus.cc:2243] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.298432 27116 raft_consensus.cc:2272] T 65e4899b95af40fd9836e281f4c23e17 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.300544 27116 tablet_replica.cc:333] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.301139 27116 raft_consensus.cc:2243] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.302098 27116 raft_consensus.cc:2272] T 75948129b8264d75972f4660893bd024 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.304420 27116 tablet_replica.cc:333] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.305210 27116 raft_consensus.cc:2243] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.305768 27116 raft_consensus.cc:2272] T 69fa505a55eb4823a412608c712fb378 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.308709 27116 tablet_replica.cc:333] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.309360 27116 raft_consensus.cc:2243] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.310024 27116 raft_consensus.cc:2272] T d14347d13af8448bb12300afb6a90639 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.313082 27116 tablet_replica.cc:333] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.313907 27116 raft_consensus.cc:2243] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.314492 27116 raft_consensus.cc:2272] T 91cdbe8d3035425abc337e6e27846e42 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.317603 27116 tablet_replica.cc:333] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.318604 27116 raft_consensus.cc:2243] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.319183 27116 raft_consensus.cc:2272] T 9233488a2b6745f8a012d9edc348333a P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.322065 27116 tablet_replica.cc:333] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.322666 27116 raft_consensus.cc:2243] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.323550 27116 raft_consensus.cc:2272] T 73f2a3fbee56421aa8768a4d7f275ba9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.326155 27116 tablet_replica.cc:333] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.326797 27116 raft_consensus.cc:2243] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.327332 27116 raft_consensus.cc:2272] T 11d7a192af654f9fa50d695be708ca81 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.329777 27116 tablet_replica.cc:333] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.330420 27116 raft_consensus.cc:2243] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.330986 27116 raft_consensus.cc:2272] T e9fe17de2e1845dd959d6b3d892cda55 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.333665 27116 tablet_replica.cc:333] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.334502 27116 raft_consensus.cc:2243] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.335059 27116 raft_consensus.cc:2272] T 74b93cd1c6f94fdf869528b9411a7959 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.337711 27116 tablet_replica.cc:333] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.338569 27116 raft_consensus.cc:2243] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.339794 27116 raft_consensus.cc:2272] T b08446fff0954c05b5311d3e76153e58 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.342880 27116 tablet_replica.cc:333] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.343657 27116 raft_consensus.cc:2243] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.344277 27116 raft_consensus.cc:2272] T 7fc90d55c744447aaa17e745cf58157c P 25902f00fb5d41679245f23dd19654aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.347095 27116 tablet_replica.cc:333] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.347667 27116 raft_consensus.cc:2243] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.348120 27116 raft_consensus.cc:2272] T f0bab1596ed74079b6efee7c9fefe4f9 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.350453 27116 tablet_replica.cc:333] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.374154 27116 raft_consensus.cc:2243] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.376809 27116 raft_consensus.cc:2272] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 25902f00fb5d41679245f23dd19654aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.384174 27116 tablet_replica.cc:333] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.385859 27116 raft_consensus.cc:2243] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.387251 27116 raft_consensus.cc:2272] T 63fcc76766934d05b1a2f4cc1f997eb9 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.392937 27116 tablet_replica.cc:333] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa: stopping tablet replica
I20260709 12:21:11.394960 27116 raft_consensus.cc:2243] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.396464 27116 raft_consensus.cc:2272] T 7c72e9bb69f746c191320d2437c3ce19 P 25902f00fb5d41679245f23dd19654aa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.396782  4194 consensus_peers.cc:597] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.429119  4120 consensus_peers.cc:597] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.434450  4194 consensus_peers.cc:597] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.437655  4194 consensus_peers.cc:597] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.462553  4120 consensus_peers.cc:597] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.463626  4120 consensus_peers.cc:597] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.492966  4194 consensus_peers.cc:597] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.498258  4120 consensus_peers.cc:597] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.505476  4120 consensus_peers.cc:597] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.506021  4194 consensus_peers.cc:597] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.522492  4194 consensus_peers.cc:597] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.537701  4120 consensus_peers.cc:597] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.590155 27116 tablet_server.cc:196] TabletServer@127.26.123.2:0 shutdown complete.
W20260709 12:21:11.653815  4120 consensus_peers.cc:597] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:11.664659  4120 consensus_peers.cc:597] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 -> Peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Couldn't send request to peer 25902f00fb5d41679245f23dd19654aa. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.727039 27116 tablet_server.cc:179] TabletServer@127.26.123.3:0 shutting down...
W20260709 12:21:11.757186  4193 consensus_peers.cc:597] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 -> Peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Couldn't send request to peer 6994df41341e478d89f7c105f550ff93. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.759523  5195 raft_consensus.cc:493] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:11.760303  5195 raft_consensus.cc:515] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:11.764811  5195 leader_election.cc:290] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:21:11.766060  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb63c6a1eed482f9d368536f4aaadf1" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:21:11.767755  5106 raft_consensus.cc:493] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:11.768361  5106 raft_consensus.cc:515] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:11.772277  5106 leader_election.cc:290] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:11.775020  5071 raft_consensus.cc:493] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
W20260709 12:21:11.774421  4120 leader_election.cc:336] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:21:11.776043  4120 leader_election.cc:304] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:21:11.775954  5071 raft_consensus.cc:515] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:11.777077  5172 raft_consensus.cc:2749] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:11.778118  5071 leader_election.cc:290] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 72940e4e52944adb81608e286ad58a59 (127.26.123.4:42389), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
W20260709 12:21:11.779217  4194 leader_election.cc:336] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:21:11.780051  4194 leader_election.cc:336] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111)
I20260709 12:21:11.780025  4220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac91fc64d6874b49a749e20bbf04ad07" candidate_uuid: "6994df41341e478d89f7c105f550ff93" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "72940e4e52944adb81608e286ad58a59" is_pre_election: true
I20260709 12:21:11.780700  4194 leader_election.cc:304] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:21:11.781821  5106 raft_consensus.cc:2749] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:11.791528  4120 leader_election.cc:336] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:21:11.792093  4120 leader_election.cc:304] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6994df41341e478d89f7c105f550ff93; no voters: 41009fde51f045c49be6461b76a27543, 72940e4e52944adb81608e286ad58a59
I20260709 12:21:11.792881  5195 raft_consensus.cc:2749] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:11.793555 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:21:11.794005  4193 consensus_peers.cc:597] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 -> Peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Couldn't send request to peer 6994df41341e478d89f7c105f550ff93. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.795070 27116 tablet_replica.cc:333] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.796243 27116 raft_consensus.cc:2243] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.796763 27116 raft_consensus.cc:2272] T e08603ff2d9145e0b85e975bac56312c P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.799513 27116 tablet_replica.cc:333] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.800226 27116 raft_consensus.cc:2243] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.800714 27116 raft_consensus.cc:2272] T 18dadd11589447aea90415c88b77e312 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.803251 27116 tablet_replica.cc:333] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.804241 27116 raft_consensus.cc:2243] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.804986 27116 raft_consensus.cc:2272] T fe179cc2289146d98b10419917a7a36d P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.808579 27116 tablet_replica.cc:333] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
W20260709 12:21:11.808943  4193 consensus_peers.cc:597] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 -> Peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Couldn't send request to peer 6994df41341e478d89f7c105f550ff93. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.809623 27116 raft_consensus.cc:2243] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.810400 27116 raft_consensus.cc:2272] T 435b4d42ac35468f8a95973a14a9d6e3 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.813190 27116 tablet_replica.cc:333] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.814127 27116 raft_consensus.cc:2243] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.814683 27116 raft_consensus.cc:2272] T 7f1c0613280b411f98f4807034b0a1cb P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.817507 27116 tablet_replica.cc:333] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.818344 27116 raft_consensus.cc:2243] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.819005 27116 raft_consensus.cc:2272] T 76d413f51fcd4460aee2adb16b951a7c P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.821478 27116 tablet_replica.cc:333] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.822366 27116 raft_consensus.cc:2243] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.822988 27116 raft_consensus.cc:2272] T 724697c71b61499aa0daadd2e66af805 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.825942  4193 consensus_peers.cc:597] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 -> Peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Couldn't send request to peer 6994df41341e478d89f7c105f550ff93. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.826249 27116 tablet_replica.cc:333] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.827076 27116 raft_consensus.cc:2243] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.827630 27116 raft_consensus.cc:2272] T 35f92fa80bd34e60924d5c77f79df421 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.830255 27116 tablet_replica.cc:333] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.830920 27116 raft_consensus.cc:2243] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.831951 27116 raft_consensus.cc:2272] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.835399 27116 tablet_replica.cc:333] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.836519 27116 raft_consensus.cc:2243] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.837342 27116 raft_consensus.cc:2272] T d10e7c5b29b14b4e8c90d79a5201bf26 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.840581 27116 tablet_replica.cc:333] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.841305 27116 raft_consensus.cc:2243] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.841918 27116 raft_consensus.cc:2272] T 581a73aef577436b9cc60234f338e9b9 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.842274  4193 consensus_peers.cc:597] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 -> Peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Couldn't send request to peer 6994df41341e478d89f7c105f550ff93. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.844511 27116 tablet_replica.cc:333] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.845320 27116 raft_consensus.cc:2243] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.845897 27116 raft_consensus.cc:2272] T fcb63c6a1eed482f9d368536f4aaadf1 P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.848258 27116 tablet_replica.cc:333] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.848951 27116 raft_consensus.cc:2243] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.849447 27116 raft_consensus.cc:2272] T 9fda924a0568433fa70ab72d4a0c3968 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.851799 27116 tablet_replica.cc:333] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.852510 27116 raft_consensus.cc:2243] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.853174 27116 raft_consensus.cc:2272] T a252087aa0c9426cb3067f2f9004dd29 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.856149 27116 tablet_replica.cc:333] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.856977 27116 raft_consensus.cc:2243] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.857837 27116 raft_consensus.cc:2272] T 75948129b8264d75972f4660893bd024 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.860764 27116 tablet_replica.cc:333] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.861455 27116 raft_consensus.cc:2243] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.862530 27116 raft_consensus.cc:2272] T 69fa505a55eb4823a412608c712fb378 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.864802 27116 tablet_replica.cc:333] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.865417 27116 raft_consensus.cc:2243] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.866338 27116 raft_consensus.cc:2272] T 64bc045a9f78405ebd413d12e6236f61 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.868898 27116 tablet_replica.cc:333] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.869695 27116 raft_consensus.cc:2243] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.870755 27116 raft_consensus.cc:2272] T 65e4899b95af40fd9836e281f4c23e17 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.873914 27116 tablet_replica.cc:333] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.874825 27116 raft_consensus.cc:2243] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.875442 27116 raft_consensus.cc:2272] T ebb6ad79685348f4a82858f9f7844c48 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.878661 27116 tablet_replica.cc:333] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.879582 27116 raft_consensus.cc:2243] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.880309 27116 raft_consensus.cc:2272] T 72ce1dc8db9548058ab94a9115c111f2 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.883970 27116 tablet_replica.cc:333] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.884829 27116 raft_consensus.cc:2243] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.885601 27116 raft_consensus.cc:2272] T 38b7f39842e94931a878f3003c1353e1 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.886194  4193 consensus_peers.cc:597] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 -> Peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Couldn't send request to peer 6994df41341e478d89f7c105f550ff93. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.888614 27116 tablet_replica.cc:333] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.889612 27116 raft_consensus.cc:2243] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.890933 27116 raft_consensus.cc:2272] T 48a9a743ab6141218e529314c318f16b P 6994df41341e478d89f7c105f550ff93 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.891438  5087 raft_consensus.cc:493] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:11.892009  5087 raft_consensus.cc:515] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:11.894021  5087 leader_election.cc:290] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:11.894258 27116 tablet_replica.cc:333] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.895298 27116 raft_consensus.cc:2243] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.896657 27116 raft_consensus.cc:2272] T 6dd58e37cb1847738fdf9dd220c2280a P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.901278 27116 tablet_replica.cc:333] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
W20260709 12:21:11.901552  4194 leader_election.cc:336] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:21:11.902266 27116 raft_consensus.cc:2243] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:21:11.902935  4194 leader_election.cc:336] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111)
I20260709 12:21:11.903429  4194 leader_election.cc:304] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:21:11.903740 27116 raft_consensus.cc:2272] T 9dc6c1656d03459babbca1538b0ae0ef P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.905290  5087 raft_consensus.cc:2749] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:11.907572 27116 tablet_replica.cc:333] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.908593 27116 raft_consensus.cc:2243] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.909385 27116 raft_consensus.cc:2272] T 7e92ca9c3274410aa6ea1aaa11671569 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.912761 27116 tablet_replica.cc:333] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.913642 27116 raft_consensus.cc:2243] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.914402 27116 raft_consensus.cc:2272] T 98ec68263b9c4d5dbdff0fbfd53629cd P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.917697 27116 tablet_replica.cc:333] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.918603 27116 raft_consensus.cc:2243] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.919373 27116 raft_consensus.cc:2272] T 7ded6cf162b34debb6ea7d81c85685da P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.922801 27116 tablet_replica.cc:333] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.923774 27116 raft_consensus.cc:2243] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.924425 27116 raft_consensus.cc:2272] T e7536eba19a0467ab642af1d051f5b77 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.927634 27116 tablet_replica.cc:333] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.928539 27116 raft_consensus.cc:2243] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.929446 27116 raft_consensus.cc:2272] T f0bab1596ed74079b6efee7c9fefe4f9 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.932910 27116 tablet_replica.cc:333] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.933815 27116 raft_consensus.cc:2243] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.934582 27116 raft_consensus.cc:2272] T 4ae1da0dfadb4b17bebb0c20ce3ec355 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.938055 27116 tablet_replica.cc:333] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.938956 27116 raft_consensus.cc:2243] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.940230 27116 raft_consensus.cc:2272] T 63fcc76766934d05b1a2f4cc1f997eb9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.945497 27116 tablet_replica.cc:333] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.946352 27116 raft_consensus.cc:2243] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.947681 27116 raft_consensus.cc:2272] T 26349fc0d5664798a248220d968f5269 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.951225 27116 tablet_replica.cc:333] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.952199 27116 raft_consensus.cc:2243] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.953290 27116 raft_consensus.cc:2272] T c07edbbf4ef64ddbbad929d20ce78c4e P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.954205  4193 consensus_peers.cc:597] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 -> Peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Couldn't send request to peer 6994df41341e478d89f7c105f550ff93. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:11.956569 27116 tablet_replica.cc:333] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.957540 27116 raft_consensus.cc:2243] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.959333 27116 raft_consensus.cc:2272] T 9d4786cea7ba49e49a9800f80ef684b8 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.961687 27116 tablet_replica.cc:333] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.962371 27116 raft_consensus.cc:2243] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.962891 27116 raft_consensus.cc:2272] T ac91fc64d6874b49a749e20bbf04ad07 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.965272 27116 tablet_replica.cc:333] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.965978 27116 raft_consensus.cc:2243] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.966507 27116 raft_consensus.cc:2272] T 73f2a3fbee56421aa8768a4d7f275ba9 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.968994 27116 tablet_replica.cc:333] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.969624 27116 raft_consensus.cc:2243] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.970681 27116 raft_consensus.cc:2272] T 11d7a192af654f9fa50d695be708ca81 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.973035 27116 tablet_replica.cc:333] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.973822 27116 raft_consensus.cc:2243] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.975255 27116 raft_consensus.cc:2272] T c2adcdb890e0483095cb9923dce850a1 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.978544 27116 tablet_replica.cc:333] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.979394 27116 raft_consensus.cc:2243] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.980139 27116 raft_consensus.cc:2272] T cb8a7e4c51b446578c698dcf9a253001 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.983044 27116 tablet_replica.cc:333] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.983834 27116 raft_consensus.cc:2243] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.985352 27116 raft_consensus.cc:2272] T e9fe17de2e1845dd959d6b3d892cda55 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.988006 27116 tablet_replica.cc:333] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.988624 27116 raft_consensus.cc:2243] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.989131 27116 raft_consensus.cc:2272] T 74b93cd1c6f94fdf869528b9411a7959 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.991518 27116 tablet_replica.cc:333] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.992065 27116 raft_consensus.cc:2243] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.993073 27116 raft_consensus.cc:2272] T 3331199805b64fa8bf0d5e8847ad17b8 P 6994df41341e478d89f7c105f550ff93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.995527 27116 tablet_replica.cc:333] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:11.996254 27116 raft_consensus.cc:2243] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.997344 27116 raft_consensus.cc:2272] T 7fc90d55c744447aaa17e745cf58157c P 6994df41341e478d89f7c105f550ff93 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.999790 27116 tablet_replica.cc:333] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93: stopping tablet replica
I20260709 12:21:12.000319 27116 raft_consensus.cc:2243] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.000753  5087 raft_consensus.cc:493] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.000958 27116 raft_consensus.cc:2272] T 525557ea38a44858b86a97b0c444ecc4 P 6994df41341e478d89f7c105f550ff93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.001278  5087 raft_consensus.cc:515] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:12.002194  5106 raft_consensus.cc:493] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.002909  5106 raft_consensus.cc:515] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:12.003533  5087 leader_election.cc:290] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:21:12.005251  5106 leader_election.cc:290] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:21:12.006834  4193 leader_election.cc:336] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:12.010063  4194 leader_election.cc:336] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:21:12.010457  4194 leader_election.cc:304] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:21:12.011075  5087 raft_consensus.cc:2749] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:12.011315  4194 leader_election.cc:336] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:21:12.013604  4194 leader_election.cc:336] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111)
I20260709 12:21:12.014029  4194 leader_election.cc:304] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:21:12.014654  5087 raft_consensus.cc:2749] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:12.016301  5087 raft_consensus.cc:493] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.016908  5087 raft_consensus.cc:515] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:12.019176  5087 leader_election.cc:290] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
W20260709 12:21:12.021975  4193 leader_election.cc:336] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:12.022665  5087 raft_consensus.cc:493] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.023396  5087 raft_consensus.cc:515] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
W20260709 12:21:12.024931  4194 leader_election.cc:336] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:21:12.025470  4194 leader_election.cc:304] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:21:12.025583  5087 leader_election.cc:290] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:21:12.026271  5106 raft_consensus.cc:2749] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:12.028301  4193 leader_election.cc:336] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:12.032220  4194 leader_election.cc:336] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:21:12.032728  4194 leader_election.cc:304] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:21:12.033538  5106 raft_consensus.cc:2749] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:12.051333  5106 raft_consensus.cc:493] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.051901  5106 raft_consensus.cc:515] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:12.054039  5106 leader_election.cc:290] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:21:12.059592  4194 leader_election.cc:336] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:21:12.060309  4194 leader_election.cc:336] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111)
I20260709 12:21:12.060715  4194 leader_election.cc:304] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:21:12.061890  5087 raft_consensus.cc:2749] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:21:12.062631  4193 consensus_peers.cc:597] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 -> Peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Couldn't send request to peer 6994df41341e478d89f7c105f550ff93. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:12.084903  5087 raft_consensus.cc:493] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.085489  5087 raft_consensus.cc:515] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:12.087136  5087 leader_election.cc:290] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:21:12.091785  4194 leader_election.cc:336] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:21:12.092442  4194 leader_election.cc:336] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111)
I20260709 12:21:12.092844  4194 leader_election.cc:304] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:21:12.093474  5087 raft_consensus.cc:2749] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:12.093953 27116 tablet_server.cc:196] TabletServer@127.26.123.3:0 shutdown complete.
I20260709 12:21:12.140964  5087 raft_consensus.cc:493] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.141831  5087 raft_consensus.cc:515] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:12.144286  5087 leader_election.cc:290] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
W20260709 12:21:12.149258  4194 leader_election.cc:336] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:21:12.149899  4193 leader_election.cc:336] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Network error: Client connection negotiation failed: client connection to 127.26.123.3:39469: connect: Connection refused (error 111)
I20260709 12:21:12.150406  4193 leader_election.cc:304] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:21:12.151221  5087 raft_consensus.cc:2749] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:12.176093  4193 consensus_peers.cc:597] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 -> Peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Couldn't send request to peer 6994df41341e478d89f7c105f550ff93. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.3:39469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:12.187233 27116 tablet_server.cc:179] TabletServer@127.26.123.4:0 shutting down...
I20260709 12:21:12.198495  5087 raft_consensus.cc:493] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.199224  5087 raft_consensus.cc:515] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } }
I20260709 12:21:12.202929  5087 leader_election.cc:290] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469), 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077)
I20260709 12:21:12.207530  5087 raft_consensus.cc:493] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.208244  5087 raft_consensus.cc:515] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:12.210954  5087 leader_election.cc:290] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
I20260709 12:21:12.224207  5177 raft_consensus.cc:493] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 41009fde51f045c49be6461b76a27543)
I20260709 12:21:12.224902  5177 raft_consensus.cc:515] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } } peers { permanent_uuid: "41009fde51f045c49be6461b76a27543" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 41077 } } peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } }
I20260709 12:21:12.227494  5177 leader_election.cc:290] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077), 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667)
W20260709 12:21:12.223073  4194 leader_election.cc:336] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:21:12.229202  4194 leader_election.cc:336] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111)
W20260709 12:21:12.232285  4194 leader_election.cc:336] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
I20260709 12:21:12.232751  4194 leader_election.cc:304] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:21:12.234594  5177 raft_consensus.cc:2749] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:12.236014  4193 leader_election.cc:336] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Network error: Client connection negotiation failed: client connection to 127.26.123.3:39469: connect: Connection refused (error 111)
I20260709 12:21:12.236487  4193 leader_election.cc:304] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 41009fde51f045c49be6461b76a27543, 6994df41341e478d89f7c105f550ff93
I20260709 12:21:12.237480  5106 raft_consensus.cc:2749] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:12.246282  4194 leader_election.cc:336] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 41009fde51f045c49be6461b76a27543 (127.26.123.1:41077): Network error: Client connection negotiation failed: client connection to 127.26.123.1:41077: connect: Connection refused (error 111)
W20260709 12:21:12.252027  4194 leader_election.cc:336] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111)
I20260709 12:21:12.252589  4194 leader_election.cc:304] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 41009fde51f045c49be6461b76a27543
I20260709 12:21:12.254087  5177 raft_consensus.cc:2749] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:12.258075 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:12.259061 27116 tablet_replica.cc:333] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.259984 27116 raft_consensus.cc:2243] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.260989 27116 raft_consensus.cc:2272] T e08603ff2d9145e0b85e975bac56312c P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.264032 27116 tablet_replica.cc:333] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.265064 27116 raft_consensus.cc:2243] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.266222 27116 raft_consensus.cc:2272] T 18dadd11589447aea90415c88b77e312 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.269467 27116 tablet_replica.cc:333] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.271524 27116 raft_consensus.cc:2243] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.272730 27116 raft_consensus.cc:2272] T 9233488a2b6745f8a012d9edc348333a P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.275506 27116 tablet_replica.cc:333] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.276202 27116 raft_consensus.cc:2243] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.276768 27116 raft_consensus.cc:2272] T fe179cc2289146d98b10419917a7a36d P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.279541 27116 tablet_replica.cc:333] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.280330 27116 raft_consensus.cc:2243] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.281354 27116 raft_consensus.cc:2272] T 7f1c0613280b411f98f4807034b0a1cb P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.283689 27116 tablet_replica.cc:333] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.284404 27116 raft_consensus.cc:2243] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.284929 27116 raft_consensus.cc:2272] T 435b4d42ac35468f8a95973a14a9d6e3 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.287532 27116 tablet_replica.cc:333] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.288182 27116 raft_consensus.cc:2243] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.288714 27116 raft_consensus.cc:2272] T 3e9cdd46c01a43e3a92c6065a0a15e12 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.291199 27116 tablet_replica.cc:333] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.291981 27116 raft_consensus.cc:2243] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.292687 27116 raft_consensus.cc:2272] T 389645cf278940c6803751b614ba1dfa P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.295922 27116 tablet_replica.cc:333] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.296700 27116 raft_consensus.cc:2243] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.297231 27116 raft_consensus.cc:2272] T 81ff51ff620749a591f77c58c287e374 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.299677 27116 tablet_replica.cc:333] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.300397 27116 raft_consensus.cc:2243] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.301615 27116 raft_consensus.cc:2272] T 724697c71b61499aa0daadd2e66af805 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.304610 27116 tablet_replica.cc:333] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.305302 27116 raft_consensus.cc:2243] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.305860 27116 raft_consensus.cc:2272] T b08446fff0954c05b5311d3e76153e58 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.308202 27116 tablet_replica.cc:333] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.308929 27116 raft_consensus.cc:2243] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.309579 27116 raft_consensus.cc:2272] T 581a73aef577436b9cc60234f338e9b9 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.311941 27116 tablet_replica.cc:333] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.312570 27116 raft_consensus.cc:2243] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.313050 27116 raft_consensus.cc:2272] T fcb63c6a1eed482f9d368536f4aaadf1 P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.315352 27116 tablet_replica.cc:333] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.316250 27116 raft_consensus.cc:2243] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.316787 27116 raft_consensus.cc:2272] T 24281667f1364c6fac542e54ed413502 P 72940e4e52944adb81608e286ad58a59 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.319140 27116 tablet_replica.cc:333] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.319774 27116 raft_consensus.cc:2243] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.320310 27116 raft_consensus.cc:2272] T 76d413f51fcd4460aee2adb16b951a7c P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.322744 27116 tablet_replica.cc:333] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.323421 27116 raft_consensus.cc:2243] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.323907 27116 raft_consensus.cc:2272] T e9fe17de2e1845dd959d6b3d892cda55 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.326138 27116 tablet_replica.cc:333] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.326766 27116 raft_consensus.cc:2243] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.327734 27116 raft_consensus.cc:2272] T 74b93cd1c6f94fdf869528b9411a7959 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.330174 27116 tablet_replica.cc:333] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.330794 27116 raft_consensus.cc:2243] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.331686 27116 raft_consensus.cc:2272] T 01529dc6d8944b9697b9a87c4e4bf450 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.334038 27116 tablet_replica.cc:333] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.334663 27116 raft_consensus.cc:2243] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.335180 27116 raft_consensus.cc:2272] T 38b7f39842e94931a878f3003c1353e1 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.337402 27116 tablet_replica.cc:333] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.338083 27116 raft_consensus.cc:2243] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.338557 27116 raft_consensus.cc:2272] T 72ce1dc8db9548058ab94a9115c111f2 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.339085  5177 raft_consensus.cc:493] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 25902f00fb5d41679245f23dd19654aa)
I20260709 12:21:12.339560  5177 raft_consensus.cc:515] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:12.341075 27116 tablet_replica.cc:333] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.341594  5177 leader_election.cc:290] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:21:12.342046 27116 raft_consensus.cc:2243] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.343179 27116 raft_consensus.cc:2272] T ebb6ad79685348f4a82858f9f7844c48 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:12.345939  4193 leader_election.cc:336] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Network error: Client connection negotiation failed: client connection to 127.26.123.3:39469: connect: Connection refused (error 111)
I20260709 12:21:12.346283 27116 tablet_replica.cc:333] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
W20260709 12:21:12.346534  4194 leader_election.cc:336] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111)
I20260709 12:21:12.346998  4194 leader_election.cc:304] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:21:12.347271 27116 raft_consensus.cc:2243] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.347781  5177 raft_consensus.cc:2749] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:12.348001 27116 raft_consensus.cc:2272] T 69fa505a55eb4823a412608c712fb378 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.350533 27116 tablet_replica.cc:333] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.351163 27116 raft_consensus.cc:2243] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.351650 27116 raft_consensus.cc:2272] T 64bc045a9f78405ebd413d12e6236f61 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.353870 27116 tablet_replica.cc:333] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.354447 27116 raft_consensus.cc:2243] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.355460 27116 raft_consensus.cc:2272] T 778cd7f085494ba1842e5f758d7f9828 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.357661 27116 tablet_replica.cc:333] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.358310 27116 raft_consensus.cc:2243] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.358798 27116 raft_consensus.cc:2272] T bcefb699ae684288b994f3255f4c89b5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.361017 27116 tablet_replica.cc:333] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.361615 27116 raft_consensus.cc:2243] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.362115 27116 raft_consensus.cc:2272] T a0f07f64b93d44269e6666cc4482c983 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.364243 27116 tablet_replica.cc:333] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.364809 27116 raft_consensus.cc:2243] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.365284 27116 raft_consensus.cc:2272] T 98ec68263b9c4d5dbdff0fbfd53629cd P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.367420 27116 tablet_replica.cc:333] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.368005 27116 raft_consensus.cc:2243] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.368928 27116 raft_consensus.cc:2272] T 9fda924a0568433fa70ab72d4a0c3968 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.371069 27116 tablet_replica.cc:333] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.371644 27116 raft_consensus.cc:2243] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.372102 27116 raft_consensus.cc:2272] T f0991ca4c9bd47b393e9566d1a6ecfc5 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.374264 27116 tablet_replica.cc:333] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.374851 27116 raft_consensus.cc:2243] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.375311 27116 raft_consensus.cc:2272] T 44ae7ddfcc1e49d4b6847e9c895edde7 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.377403 27116 tablet_replica.cc:333] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.378002 27116 raft_consensus.cc:2243] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.378520 27116 raft_consensus.cc:2272] T 63fcc76766934d05b1a2f4cc1f997eb9 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.380563 27116 tablet_replica.cc:333] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.381101 27116 raft_consensus.cc:2243] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.381491 27116 raft_consensus.cc:2272] T 26349fc0d5664798a248220d968f5269 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.383563 27116 tablet_replica.cc:333] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.384106 27116 raft_consensus.cc:2243] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.384536 27116 raft_consensus.cc:2272] T 9d4786cea7ba49e49a9800f80ef684b8 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.386610 27116 tablet_replica.cc:333] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.387125 27116 raft_consensus.cc:2243] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.387532 27116 raft_consensus.cc:2272] T ac91fc64d6874b49a749e20bbf04ad07 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.389668 27116 tablet_replica.cc:333] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.390185 27116 raft_consensus.cc:2243] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.390625 27116 raft_consensus.cc:2272] T e203dae959394f4e90051ad4746a9291 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.392786 27116 tablet_replica.cc:333] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.393339 27116 raft_consensus.cc:2243] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.394213 27116 raft_consensus.cc:2272] T e7536eba19a0467ab642af1d051f5b77 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.396277 27116 tablet_replica.cc:333] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.396780 27116 raft_consensus.cc:2243] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.397203 27116 raft_consensus.cc:2272] T 7ded6cf162b34debb6ea7d81c85685da P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.399339 27116 tablet_replica.cc:333] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.399818 27116 raft_consensus.cc:2243] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.400241 27116 raft_consensus.cc:2272] T 48a9a743ab6141218e529314c318f16b P 72940e4e52944adb81608e286ad58a59 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.402288 27116 tablet_replica.cc:333] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.402779 27116 raft_consensus.cc:2243] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.403602 27116 raft_consensus.cc:2272] T d14347d13af8448bb12300afb6a90639 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.405751 27116 tablet_replica.cc:333] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.406270 27116 raft_consensus.cc:2243] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.407205 27116 raft_consensus.cc:2272] T 91cdbe8d3035425abc337e6e27846e42 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.407433  5177 raft_consensus.cc:493] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 25902f00fb5d41679245f23dd19654aa)
I20260709 12:21:12.408069  5177 raft_consensus.cc:515] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25902f00fb5d41679245f23dd19654aa" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43667 } } peers { permanent_uuid: "6994df41341e478d89f7c105f550ff93" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 39469 } } peers { permanent_uuid: "72940e4e52944adb81608e286ad58a59" member_type: VOTER last_known_addr { host: "127.26.123.4" port: 42389 } }
I20260709 12:21:12.409685 27116 tablet_replica.cc:333] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.410238  5177 leader_election.cc:290] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667), 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469)
I20260709 12:21:12.410543 27116 raft_consensus.cc:2243] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.411751 27116 raft_consensus.cc:2272] T 525557ea38a44858b86a97b0c444ecc4 P 72940e4e52944adb81608e286ad58a59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.414433 27116 tablet_replica.cc:333] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
W20260709 12:21:12.414662  4193 leader_election.cc:336] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6994df41341e478d89f7c105f550ff93 (127.26.123.3:39469): Network error: Client connection negotiation failed: client connection to 127.26.123.3:39469: connect: Connection refused (error 111)
I20260709 12:21:12.415174 27116 raft_consensus.cc:2243] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:12.415264  4194 leader_election.cc:336] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 25902f00fb5d41679245f23dd19654aa (127.26.123.2:43667): Network error: Client connection negotiation failed: client connection to 127.26.123.2:43667: connect: Connection refused (error 111)
I20260709 12:21:12.415791  4194 leader_election.cc:304] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72940e4e52944adb81608e286ad58a59; no voters: 25902f00fb5d41679245f23dd19654aa, 6994df41341e478d89f7c105f550ff93
I20260709 12:21:12.415875 27116 raft_consensus.cc:2272] T 9dc6c1656d03459babbca1538b0ae0ef P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.416553  5177 raft_consensus.cc:2749] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:12.418499 27116 tablet_replica.cc:333] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.419170 27116 raft_consensus.cc:2243] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.419653 27116 raft_consensus.cc:2272] T 7e92ca9c3274410aa6ea1aaa11671569 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.421912 27116 tablet_replica.cc:333] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59: stopping tablet replica
I20260709 12:21:12.422394 27116 raft_consensus.cc:2243] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.423282 27116 raft_consensus.cc:2272] T 7c72e9bb69f746c191320d2437c3ce19 P 72940e4e52944adb81608e286ad58a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.501935 27116 tablet_server.cc:196] TabletServer@127.26.123.4:0 shutdown complete.
I20260709 12:21:12.579907 27116 master.cc:562] Master@127.26.123.62:34017 shutting down...
I20260709 12:21:12.604720 27116 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:12.605427 27116 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.605793 27116 tablet_replica.cc:333] T 00000000000000000000000000000000 P 24e095ac04c346e8bc5cdb69e7f59d95: stopping tablet replica
I20260709 12:21:12.628896 27116 master.cc:584] Master@127.26.123.62:34017 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (33544 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:21:12.720842 27116 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.123.62:40191
I20260709 12:21:12.722429 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:12.728543  5200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:12.729207  5201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:12.729697  5203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:12.730208 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:12.731248 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:12.731441 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:12.731568 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599672731556 us; error 0 us; skew 500 ppm
I20260709 12:21:12.732076 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:12.734579 27116 webserver.cc:533] Webserver started at http://127.26.123.62:34401/ using document root <none> and password file <none>
I20260709 12:21:12.735069 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:12.735236 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:12.735469 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:12.736781 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/master-0-root/instance:
uuid: "7f01cd74e1714f289929e6739cbef069"
format_stamp: "Formatted at 2026-07-09 12:21:12 on dist-test-slave-9q7n"
I20260709 12:21:12.742175 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:21:12.745934  5208 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:12.746729 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:21:12.747036 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/master-0-root
uuid: "7f01cd74e1714f289929e6739cbef069"
format_stamp: "Formatted at 2026-07-09 12:21:12 on dist-test-slave-9q7n"
I20260709 12:21:12.747318 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:12.767292 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:12.768657 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:12.811626 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.62:40191
I20260709 12:21:12.811769  5259 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.62:40191 every 8 connection(s)
I20260709 12:21:12.815932  5260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:12.829178  5260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069: Bootstrap starting.
I20260709 12:21:12.834791  5260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:12.839607  5260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069: No bootstrap required, opened a new log
I20260709 12:21:12.842237  5260 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f01cd74e1714f289929e6739cbef069" member_type: VOTER }
I20260709 12:21:12.842681  5260 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:12.842864  5260 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f01cd74e1714f289929e6739cbef069, State: Initialized, Role: FOLLOWER
I20260709 12:21:12.843474  5260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f01cd74e1714f289929e6739cbef069" member_type: VOTER }
I20260709 12:21:12.843954  5260 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:12.844167  5260 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:12.844380  5260 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:12.850294  5260 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f01cd74e1714f289929e6739cbef069" member_type: VOTER }
I20260709 12:21:12.850903  5260 leader_election.cc:304] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f01cd74e1714f289929e6739cbef069; no voters: 
I20260709 12:21:12.852118  5260 leader_election.cc:290] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:12.852591  5263 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:12.854058  5263 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 1 LEADER]: Becoming Leader. State: Replica: 7f01cd74e1714f289929e6739cbef069, State: Running, Role: LEADER
I20260709 12:21:12.854820  5263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f01cd74e1714f289929e6739cbef069" member_type: VOTER }
I20260709 12:21:12.855273  5260 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:12.861860  5264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f01cd74e1714f289929e6739cbef069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f01cd74e1714f289929e6739cbef069" member_type: VOTER } }
I20260709 12:21:12.861666  5265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f01cd74e1714f289929e6739cbef069. Latest consensus state: current_term: 1 leader_uuid: "7f01cd74e1714f289929e6739cbef069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f01cd74e1714f289929e6739cbef069" member_type: VOTER } }
I20260709 12:21:12.862609  5264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:12.862694  5265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:12.864809  5271 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:12.872603  5271 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:12.873776 27116 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:12.882810  5271 catalog_manager.cc:1383] Generated new cluster ID: 6adb016fc0334110b88457697ee5de03
I20260709 12:21:12.883164  5271 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:12.902976  5271 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:12.904460  5271 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:12.919875  5271 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069: Generated new TSK 0
I20260709 12:21:12.920604  5271 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:12.940879 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:12.947701  5281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:12.948653  5282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:12.953119 27116 server_base.cc:1061] running on GCE node
W20260709 12:21:12.952991  5284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:12.954620 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:12.954831 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:12.954957 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599672954945 us; error 0 us; skew 500 ppm
I20260709 12:21:12.955480 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:12.958215 27116 webserver.cc:533] Webserver started at http://127.26.123.1:34373/ using document root <none> and password file <none>
I20260709 12:21:12.958725 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:12.958916 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:12.959151 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:12.960331 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-0-root/instance:
uuid: "bc6531ba57ad413a87df2a7cf63367c1"
format_stamp: "Formatted at 2026-07-09 12:21:12 on dist-test-slave-9q7n"
I20260709 12:21:12.965436 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.007s
I20260709 12:21:12.969318  5289 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:12.970256 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:12.970578 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "bc6531ba57ad413a87df2a7cf63367c1"
format_stamp: "Formatted at 2026-07-09 12:21:12 on dist-test-slave-9q7n"
I20260709 12:21:12.970896 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:12.983773 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:12.985246 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:12.987082 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:12.989771 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:12.990002 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:12.990239 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:12.990422 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:13.037459 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:33819
I20260709 12:21:13.037580  5351 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:33819 every 8 connection(s)
I20260709 12:21:13.043081 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:13.051044  5356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:13.051682  5357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:13.055469  5359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:13.056128 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:13.057708 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:13.058001 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:13.058060  5352 heartbeater.cc:344] Connected to a master server at 127.26.123.62:40191
I20260709 12:21:13.058183 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599673058164 us; error 0 us; skew 500 ppm
I20260709 12:21:13.058588  5352 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:13.058964 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:13.059461  5352 heartbeater.cc:507] Master 127.26.123.62:40191 requested a full tablet report, sending...
I20260709 12:21:13.061884 27116 webserver.cc:533] Webserver started at http://127.26.123.2:40229/ using document root <none> and password file <none>
I20260709 12:21:13.061899  5225 ts_manager.cc:194] Registered new tserver with Master: bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819)
I20260709 12:21:13.062585 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:13.062813 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:13.063083 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:13.064127  5225 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53526
I20260709 12:21:13.064384 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-1-root/instance:
uuid: "06d54f6275524a6b99d7846015e61326"
format_stamp: "Formatted at 2026-07-09 12:21:13 on dist-test-slave-9q7n"
I20260709 12:21:13.069257 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:13.072957  5364 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:13.073809 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:13.074115 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-1-root
uuid: "06d54f6275524a6b99d7846015e61326"
format_stamp: "Formatted at 2026-07-09 12:21:13 on dist-test-slave-9q7n"
I20260709 12:21:13.074399 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:13.092710 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:13.094164 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:13.095850 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:13.098348 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:13.098567 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:13.098798 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:13.098966 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:13.142894 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.2:44697
I20260709 12:21:13.143069  5426 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.2:44697 every 8 connection(s)
I20260709 12:21:13.148103 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:13.156257  5430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:13.156927  5431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:13.160045  5433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:13.160836 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:13.161996  5427 heartbeater.cc:344] Connected to a master server at 127.26.123.62:40191
I20260709 12:21:13.162181 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:21:13.162442  5427 heartbeater.cc:461] Registering TS with master...
W20260709 12:21:13.162495 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:13.162811 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599673162788 us; error 0 us; skew 500 ppm
I20260709 12:21:13.163352  5427 heartbeater.cc:507] Master 127.26.123.62:40191 requested a full tablet report, sending...
I20260709 12:21:13.163442 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:13.165972  5225 ts_manager.cc:194] Registered new tserver with Master: 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:13.166620 27116 webserver.cc:533] Webserver started at http://127.26.123.3:42683/ using document root <none> and password file <none>
I20260709 12:21:13.167392 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:13.167630 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:13.167943 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:13.168052  5225 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53530
I20260709 12:21:13.169683 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-2-root/instance:
uuid: "7acc97f6aaa944fb892f5f2f050449de"
format_stamp: "Formatted at 2026-07-09 12:21:13 on dist-test-slave-9q7n"
I20260709 12:21:13.174777 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:21:13.178507  5438 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:13.179353 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:13.179677 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-2-root
uuid: "7acc97f6aaa944fb892f5f2f050449de"
format_stamp: "Formatted at 2026-07-09 12:21:13 on dist-test-slave-9q7n"
I20260709 12:21:13.179982 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:13.191777 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:13.193128 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:13.194752 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:13.197263 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:13.197484 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:13.197726 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:13.197908 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:21:13.241787 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.3:41297
I20260709 12:21:13.241906  5500 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.3:41297 every 8 connection(s)
I20260709 12:21:13.257372  5501 heartbeater.cc:344] Connected to a master server at 127.26.123.62:40191
I20260709 12:21:13.257906  5501 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:13.258738  5501 heartbeater.cc:507] Master 127.26.123.62:40191 requested a full tablet report, sending...
I20260709 12:21:13.260942  5225 ts_manager.cc:194] Registered new tserver with Master: 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:13.262090 27116 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016252872s
I20260709 12:21:13.262770  5225 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53532
I20260709 12:21:13.290668  5225 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53544:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:13.293900  5225 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:13.373406  5463 tablet_service.cc:1511] Processing CreateTablet for tablet 7360d5e0f39948b1aaaf652369b525d3 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:21:13.375329  5463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7360d5e0f39948b1aaaf652369b525d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.379065  5462 tablet_service.cc:1511] Processing CreateTablet for tablet fe4161fa651346d69b76af4ca768e397 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:21:13.380899  5462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4161fa651346d69b76af4ca768e397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.380537  5465 tablet_service.cc:1511] Processing CreateTablet for tablet 5ce5ea9c5a1e40149b4ee83a962ec4bf (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:21:13.379184  5464 tablet_service.cc:1511] Processing CreateTablet for tablet 7e42c417384f41ef9b5f807b1875d3a1 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:21:13.383778  5465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ce5ea9c5a1e40149b4ee83a962ec4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.373406  5466 tablet_service.cc:1511] Processing CreateTablet for tablet 268ee27bb535447db8be5d649000ca9f (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:21:13.386636  5466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 268ee27bb535447db8be5d649000ca9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.388898  5464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e42c417384f41ef9b5f807b1875d3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.391865  5392 tablet_service.cc:1511] Processing CreateTablet for tablet 268ee27bb535447db8be5d649000ca9f (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:21:13.393496  5392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 268ee27bb535447db8be5d649000ca9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.397131  5391 tablet_service.cc:1511] Processing CreateTablet for tablet 5ce5ea9c5a1e40149b4ee83a962ec4bf (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:21:13.398824  5391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ce5ea9c5a1e40149b4ee83a962ec4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.399186  5389 tablet_service.cc:1511] Processing CreateTablet for tablet 7360d5e0f39948b1aaaf652369b525d3 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:21:13.400759  5389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7360d5e0f39948b1aaaf652369b525d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.403105  5390 tablet_service.cc:1511] Processing CreateTablet for tablet 7e42c417384f41ef9b5f807b1875d3a1 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:21:13.404982  5390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e42c417384f41ef9b5f807b1875d3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.407369  5461 tablet_service.cc:1511] Processing CreateTablet for tablet cce83284c7664c1da3f565197bacbe18 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:21:13.407688  5388 tablet_service.cc:1511] Processing CreateTablet for tablet fe4161fa651346d69b76af4ca768e397 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:21:13.409238  5388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4161fa651346d69b76af4ca768e397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.409273  5461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cce83284c7664c1da3f565197bacbe18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.431668  5466 tablet_service.cc:1511] Processing CreateTablet for tablet 73c014a910d24602b825542f7fa781a1 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:21:13.433270  5466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73c014a910d24602b825542f7fa781a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.435406  5387 tablet_service.cc:1511] Processing CreateTablet for tablet cce83284c7664c1da3f565197bacbe18 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:21:13.436920  5387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cce83284c7664c1da3f565197bacbe18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.440907  5386 tablet_service.cc:1511] Processing CreateTablet for tablet 73c014a910d24602b825542f7fa781a1 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:21:13.443013  5385 tablet_service.cc:1511] Processing CreateTablet for tablet 076abc288e1345c6a56724b2d2c8dfc9 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:21:13.444597  5385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 076abc288e1345c6a56724b2d2c8dfc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.445091  5460 tablet_service.cc:1511] Processing CreateTablet for tablet 076abc288e1345c6a56724b2d2c8dfc9 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:21:13.446640  5460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 076abc288e1345c6a56724b2d2c8dfc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.450855  5386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73c014a910d24602b825542f7fa781a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.455101  5317 tablet_service.cc:1511] Processing CreateTablet for tablet 268ee27bb535447db8be5d649000ca9f (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:21:13.456712  5317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 268ee27bb535447db8be5d649000ca9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.459110  5316 tablet_service.cc:1511] Processing CreateTablet for tablet 5ce5ea9c5a1e40149b4ee83a962ec4bf (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:21:13.460774  5316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ce5ea9c5a1e40149b4ee83a962ec4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.463084  5315 tablet_service.cc:1511] Processing CreateTablet for tablet 7e42c417384f41ef9b5f807b1875d3a1 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:21:13.464700  5315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e42c417384f41ef9b5f807b1875d3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.466272  5313 tablet_service.cc:1511] Processing CreateTablet for tablet fe4161fa651346d69b76af4ca768e397 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:21:13.467883  5313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe4161fa651346d69b76af4ca768e397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.471761  5314 tablet_service.cc:1511] Processing CreateTablet for tablet 7360d5e0f39948b1aaaf652369b525d3 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:21:13.472512  5312 tablet_service.cc:1511] Processing CreateTablet for tablet cce83284c7664c1da3f565197bacbe18 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:21:13.479138  5310 tablet_service.cc:1511] Processing CreateTablet for tablet 076abc288e1345c6a56724b2d2c8dfc9 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:21:13.480432  5314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7360d5e0f39948b1aaaf652369b525d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.481979  5311 tablet_service.cc:1511] Processing CreateTablet for tablet 73c014a910d24602b825542f7fa781a1 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:21:13.483490  5311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73c014a910d24602b825542f7fa781a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.486220  5310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 076abc288e1345c6a56724b2d2c8dfc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.490166  5312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cce83284c7664c1da3f565197bacbe18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.530205  5516 tablet_bootstrap.cc:492] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:13.536710  5517 tablet_bootstrap.cc:492] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:13.537629  5309 tablet_service.cc:1511] Processing CreateTablet for tablet aecccb1632bd493dabfe2344f6a7dd39 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:21:13.539225  5309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aecccb1632bd493dabfe2344f6a7dd39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.544741  5516 tablet_bootstrap.cc:654] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.545666  5384 tablet_service.cc:1511] Processing CreateTablet for tablet aecccb1632bd493dabfe2344f6a7dd39 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:21:13.547258  5384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aecccb1632bd493dabfe2344f6a7dd39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.563127  5517 tablet_bootstrap.cc:654] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.579950  5310 tablet_service.cc:1511] Processing CreateTablet for tablet 1feedeecfd9247498d0a76ac3d5e7e26 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:21:13.581599  5310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1feedeecfd9247498d0a76ac3d5e7e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.583002  5466 tablet_service.cc:1511] Processing CreateTablet for tablet 1feedeecfd9247498d0a76ac3d5e7e26 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:21:13.584507  5466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1feedeecfd9247498d0a76ac3d5e7e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.594918  5385 tablet_service.cc:1511] Processing CreateTablet for tablet 1feedeecfd9247498d0a76ac3d5e7e26 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:21:13.596489  5385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1feedeecfd9247498d0a76ac3d5e7e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.602898  5462 tablet_service.cc:1511] Processing CreateTablet for tablet aecccb1632bd493dabfe2344f6a7dd39 (DEFAULT_TABLE table=test-workload [id=700783dd37fc4108bfedefc1d7a83165]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:21:13.604499  5462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aecccb1632bd493dabfe2344f6a7dd39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:13.610358  5516 tablet_bootstrap.cc:492] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:13.611889  5516 ts_tablet_manager.cc:1403] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.004s
I20260709 12:21:13.615331  5516 raft_consensus.cc:359] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.616101  5516 raft_consensus.cc:385] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.616402  5516 raft_consensus.cc:740] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.617211  5516 consensus_queue.cc:260] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.619871  5501 heartbeater.cc:499] Master 127.26.123.62:40191 was elected leader, sending a full tablet report...
I20260709 12:21:13.620324  5516 ts_tablet_manager.cc:1434] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:13.622720  5516 tablet_bootstrap.cc:492] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:13.623219  5518 tablet_bootstrap.cc:492] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:13.630867  5516 tablet_bootstrap.cc:654] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.640960  5517 tablet_bootstrap.cc:492] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:13.641624  5517 ts_tablet_manager.cc:1403] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.105s	user 0.021s	sys 0.013s
I20260709 12:21:13.643847  5516 tablet_bootstrap.cc:492] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:13.644475  5516 ts_tablet_manager.cc:1403] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.001s
I20260709 12:21:13.644832  5517 raft_consensus.cc:359] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.645702  5517 raft_consensus.cc:385] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.646076  5517 raft_consensus.cc:740] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.647104  5517 consensus_queue.cc:260] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.648238  5516 raft_consensus.cc:359] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:13.649123  5516 raft_consensus.cc:385] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.649581  5516 raft_consensus.cc:740] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.650671  5516 consensus_queue.cc:260] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:13.653218  5516 ts_tablet_manager.cc:1434] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:13.654491  5516 tablet_bootstrap.cc:492] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:13.655700  5518 tablet_bootstrap.cc:654] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.657019  5517 ts_tablet_manager.cc:1434] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:21:13.657992  5352 heartbeater.cc:499] Master 127.26.123.62:40191 was elected leader, sending a full tablet report...
I20260709 12:21:13.662088  5516 tablet_bootstrap.cc:654] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.676608  5517 tablet_bootstrap.cc:492] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:13.688046  5517 tablet_bootstrap.cc:654] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.691023  5516 tablet_bootstrap.cc:492] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:13.691608  5516 ts_tablet_manager.cc:1403] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.037s	user 0.028s	sys 0.008s
I20260709 12:21:13.694897  5516 raft_consensus.cc:359] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.695695  5516 raft_consensus.cc:385] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.695768  5518 tablet_bootstrap.cc:492] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:13.696070  5516 raft_consensus.cc:740] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.696422  5518 ts_tablet_manager.cc:1403] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.016s
I20260709 12:21:13.697129  5516 consensus_queue.cc:260] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.699872  5518 raft_consensus.cc:359] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.700436  5516 ts_tablet_manager.cc:1434] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:13.700482  5518 raft_consensus.cc:385] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.701037  5518 raft_consensus.cc:740] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.701929  5516 tablet_bootstrap.cc:492] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:13.701929  5518 consensus_queue.cc:260] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.703321  5517 tablet_bootstrap.cc:492] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:13.703990  5517 ts_tablet_manager.cc:1403] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.010s
I20260709 12:21:13.707432  5517 raft_consensus.cc:359] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.708374  5517 raft_consensus.cc:385] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.708770  5517 raft_consensus.cc:740] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.709923  5516 tablet_bootstrap.cc:654] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.709830  5517 consensus_queue.cc:260] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.711987  5518 ts_tablet_manager.cc:1434] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.004s
I20260709 12:21:13.712510  5517 ts_tablet_manager.cc:1434] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.008s
I20260709 12:21:13.712852  5427 heartbeater.cc:499] Master 127.26.123.62:40191 was elected leader, sending a full tablet report...
I20260709 12:21:13.713996  5517 tablet_bootstrap.cc:492] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:13.718290  5518 tablet_bootstrap.cc:492] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:13.726517  5518 tablet_bootstrap.cc:654] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.727970  5517 tablet_bootstrap.cc:654] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.744277  5517 tablet_bootstrap.cc:492] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:13.744293  5518 tablet_bootstrap.cc:492] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:13.744895  5517 ts_tablet_manager.cc:1403] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.004s
I20260709 12:21:13.744917  5518 ts_tablet_manager.cc:1403] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.005s
W20260709 12:21:13.747000  5502 tablet.cc:2406] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:21:13.747010  5516 tablet_bootstrap.cc:492] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:13.747922  5516 ts_tablet_manager.cc:1403] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.046s	user 0.024s	sys 0.014s
I20260709 12:21:13.748016  5517 raft_consensus.cc:359] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.748888  5517 raft_consensus.cc:385] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.748421  5518 raft_consensus.cc:359] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:13.749213  5517 raft_consensus.cc:740] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.749596  5518 raft_consensus.cc:385] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.749976  5518 raft_consensus.cc:740] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.750152  5517 consensus_queue.cc:260] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.750845  5518 consensus_queue.cc:260] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:13.751536  5516 raft_consensus.cc:359] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.752267  5516 raft_consensus.cc:385] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.752431  5517 ts_tablet_manager.cc:1434] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:13.752597  5516 raft_consensus.cc:740] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.753684  5517 tablet_bootstrap.cc:492] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:13.753800  5516 consensus_queue.cc:260] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.758949  5516 ts_tablet_manager.cc:1434] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:21:13.760125  5516 tablet_bootstrap.cc:492] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:13.761626  5517 tablet_bootstrap.cc:654] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.767273  5516 tablet_bootstrap.cc:654] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.771668  5518 ts_tablet_manager.cc:1434] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:21:13.771970  5517 tablet_bootstrap.cc:492] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:13.773054  5518 tablet_bootstrap.cc:492] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:13.773274  5527 raft_consensus.cc:493] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:13.773094  5517 ts_tablet_manager.cc:1403] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.002s
I20260709 12:21:13.773949  5527 raft_consensus.cc:515] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:13.775218  5516 tablet_bootstrap.cc:492] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:13.775868  5516 ts_tablet_manager.cc:1403] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:21:13.777262  5517 raft_consensus.cc:359] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:13.778154  5517 raft_consensus.cc:385] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.778460  5517 raft_consensus.cc:740] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.779589  5527 leader_election.cc:290] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297), bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819)
I20260709 12:21:13.779778  5516 raft_consensus.cc:359] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.781438  5516 raft_consensus.cc:385] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.779264  5517 consensus_queue.cc:260] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:13.784235  5521 raft_consensus.cc:493] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:13.785050  5517 ts_tablet_manager.cc:1434] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:21:13.784924  5521 raft_consensus.cc:515] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.781886  5516 raft_consensus.cc:740] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.786347  5517 tablet_bootstrap.cc:492] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:13.787037  5516 consensus_queue.cc:260] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.794952  5517 tablet_bootstrap.cc:654] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.794955  5518 tablet_bootstrap.cc:654] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.808446  5516 ts_tablet_manager.cc:1434] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.032s	user 0.015s	sys 0.000s
I20260709 12:21:13.805958  5521 leader_election.cc:290] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:13.809661  5516 tablet_bootstrap.cc:492] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:13.822198  5516 tablet_bootstrap.cc:654] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.825265  5517 tablet_bootstrap.cc:492] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:13.825897  5517 ts_tablet_manager.cc:1403] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.000s
I20260709 12:21:13.830644  5327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:13.831169  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7360d5e0f39948b1aaaf652369b525d3" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:13.831588  5327 raft_consensus.cc:2468] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 0.
I20260709 12:21:13.831776  5517 raft_consensus.cc:359] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.832077  5326 raft_consensus.cc:2468] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 0.
I20260709 12:21:13.832615  5517 raft_consensus.cc:385] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.832978  5517 raft_consensus.cc:740] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.833338  5367 leader_election.cc:304] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:13.833976  5441 leader_election.cc:304] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:13.833863  5517 consensus_queue.cc:260] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.834427  5527 raft_consensus.cc:2804] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:13.834897  5527 raft_consensus.cc:493] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:13.835273  5527 raft_consensus.cc:3060] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:13.835325  5521 raft_consensus.cc:2804] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:13.835858  5521 raft_consensus.cc:493] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:13.836202  5521 raft_consensus.cc:3060] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:13.836580  5517 ts_tablet_manager.cc:1434] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:21:13.837821  5517 tablet_bootstrap.cc:492] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:13.843590  5518 tablet_bootstrap.cc:492] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:13.844343  5518 ts_tablet_manager.cc:1403] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.072s	user 0.008s	sys 0.004s
I20260709 12:21:13.844507  5527 raft_consensus.cc:515] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:13.847746  5518 raft_consensus.cc:359] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.848590  5518 raft_consensus.cc:385] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.848888  5518 raft_consensus.cc:740] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.849035  5527 leader_election.cc:290] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 election: Requested vote from peers 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297), bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819)
I20260709 12:21:13.849321  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:13.849689  5518 consensus_queue.cc:260] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.850183  5326 raft_consensus.cc:3060] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:13.852064  5518 ts_tablet_manager.cc:1434] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:13.853179  5527 raft_consensus.cc:493] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:13.853816  5527 raft_consensus.cc:515] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.856518  5517 tablet_bootstrap.cc:654] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.856985  5327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ce5ea9c5a1e40149b4ee83a962ec4bf" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:13.857874  5327 raft_consensus.cc:2468] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 0.
I20260709 12:21:13.853179  5518 tablet_bootstrap.cc:492] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:13.859366  5367 leader_election.cc:304] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:13.845139  5521 raft_consensus.cc:515] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.864223  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:13.865013  5476 raft_consensus.cc:2468] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 0.
I20260709 12:21:13.865394  5527 leader_election.cc:290] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:13.866115  5527 raft_consensus.cc:2804] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:13.866488  5527 raft_consensus.cc:493] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:13.866822  5527 raft_consensus.cc:3060] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:13.867254  5518 tablet_bootstrap.cc:654] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:13.867820  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:13.864246  5327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7360d5e0f39948b1aaaf652369b525d3" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:13.869163  5327 raft_consensus.cc:3060] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:13.869612  5537 raft_consensus.cc:493] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:13.870550  5516 tablet_bootstrap.cc:492] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:13.870155  5537 raft_consensus.cc:515] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.871191  5516 ts_tablet_manager.cc:1403] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.004s
I20260709 12:21:13.869133  5326 raft_consensus.cc:2468] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 1.
I20260709 12:21:13.875317  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:13.876101  5476 raft_consensus.cc:3060] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:13.866923  5521 leader_election.cc:290] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:13.893815  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e42c417384f41ef9b5f807b1875d3a1" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:13.894524  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ce5ea9c5a1e40149b4ee83a962ec4bf" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
W20260709 12:21:13.895630  5441 leader_election.cc:343] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:13.896764  5537 leader_election.cc:290] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
W20260709 12:21:13.897298  5366 leader_election.cc:343] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:13.898542  5516 raft_consensus.cc:359] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.899439  5516 raft_consensus.cc:385] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.899752  5516 raft_consensus.cc:740] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.900593  5516 consensus_queue.cc:260] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.903167  5516 ts_tablet_manager.cc:1434] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:21:13.904388  5516 tablet_bootstrap.cc:492] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:13.909096  5518 tablet_bootstrap.cc:492] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:13.910672  5527 raft_consensus.cc:515] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.911865  5518 ts_tablet_manager.cc:1403] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.059s	user 0.016s	sys 0.015s
I20260709 12:21:13.913206  5527 leader_election.cc:290] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:13.914609  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ce5ea9c5a1e40149b4ee83a962ec4bf" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:13.915310  5475 raft_consensus.cc:3060] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:13.922299  5367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:33819, user_credentials={real_user=slave}} blocked reactor thread for 63031.5us
I20260709 12:21:13.923933  5367 leader_election.cc:304] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:13.924465  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ce5ea9c5a1e40149b4ee83a962ec4bf" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:13.924796  5476 raft_consensus.cc:2468] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 1.
I20260709 12:21:13.924919  5516 tablet_bootstrap.cc:654] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.925216  5326 raft_consensus.cc:3060] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:13.926393  5518 raft_consensus.cc:359] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.922600  5327 raft_consensus.cc:2468] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 1.
I20260709 12:21:13.927327  5517 tablet_bootstrap.cc:492] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:13.927976  5542 raft_consensus.cc:2804] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:13.928529  5402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7360d5e0f39948b1aaaf652369b525d3" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:13.928663  5517 ts_tablet_manager.cc:1403] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.091s	user 0.032s	sys 0.011s
I20260709 12:21:13.927423  5518 raft_consensus.cc:385] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.929479  5518 raft_consensus.cc:740] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.928707  5542 raft_consensus.cc:697] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 1 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:13.930770  5401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7360d5e0f39948b1aaaf652369b525d3" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:13.931164  5542 consensus_queue.cc:237] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:13.932389  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e42c417384f41ef9b5f807b1875d3a1" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:13.933218  5400 raft_consensus.cc:2468] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 0.
I20260709 12:21:13.935079  5441 leader_election.cc:304] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:13.936199  5521 raft_consensus.cc:493] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:13.936738  5521 raft_consensus.cc:515] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.939256  5521 leader_election.cc:290] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:13.941358  5327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076abc288e1345c6a56724b2d2c8dfc9" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:13.942431  5327 raft_consensus.cc:2468] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 0.
I20260709 12:21:13.944943  5441 leader_election.cc:304] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:13.945752  5521 raft_consensus.cc:2804] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:13.945441  5518 consensus_queue.cc:260] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.946311  5521 raft_consensus.cc:697] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:13.946977  5517 raft_consensus.cc:359] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.947635  5516 tablet_bootstrap.cc:492] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:13.947284  5521 consensus_queue.cc:237] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.947811  5517 raft_consensus.cc:385] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.948237  5517 raft_consensus.cc:740] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.948302  5516 ts_tablet_manager.cc:1403] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.007s
I20260709 12:21:13.949302  5517 consensus_queue.cc:260] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.951965  5516 raft_consensus.cc:359] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.952734  5516 raft_consensus.cc:385] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:13.953027  5516 raft_consensus.cc:740] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:13.954107  5517 ts_tablet_manager.cc:1434] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.025s	user 0.011s	sys 0.011s
I20260709 12:21:13.954037  5516 consensus_queue.cc:260] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.954984  5537 raft_consensus.cc:2804] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:13.955310  5517 tablet_bootstrap.cc:492] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:13.955435  5537 raft_consensus.cc:493] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:13.955861  5537 raft_consensus.cc:3060] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:13.963075  5517 tablet_bootstrap.cc:654] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.963888  5516 ts_tablet_manager.cc:1434] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.005s
I20260709 12:21:13.965154  5516 tablet_bootstrap.cc:492] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:13.949236  5518 ts_tablet_manager.cc:1434] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.037s	user 0.012s	sys 0.002s
I20260709 12:21:13.968773  5518 tablet_bootstrap.cc:492] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:13.972931  5516 tablet_bootstrap.cc:654] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.975231  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076abc288e1345c6a56724b2d2c8dfc9" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:13.976159  5400 raft_consensus.cc:2468] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 0.
I20260709 12:21:13.977491  5518 tablet_bootstrap.cc:654] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:13.977717  5475 raft_consensus.cc:2468] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 1.
W20260709 12:21:13.977331  5442 leader_election.cc:343] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:13.987632  5222 catalog_manager.cc:5697] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 0 to 1, leader changed from <none> to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 1 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:13.988709  5442 leader_election.cc:343] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:13.989856  5442 leader_election.cc:304] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, 7acc97f6aaa944fb892f5f2f050449de; no voters: bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:13.965042  5326 raft_consensus.cc:2468] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 1.
I20260709 12:21:13.985947  5224 catalog_manager.cc:5697] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:13.991464  5367 leader_election.cc:304] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:13.992316  5542 raft_consensus.cc:2804] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:13.992758  5542 raft_consensus.cc:697] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 1 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:13.993494  5542 consensus_queue.cc:237] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.994251  5521 raft_consensus.cc:2804] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:13.994753  5521 raft_consensus.cc:493] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:13.995095  5521 raft_consensus.cc:3060] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:13.995702  5537 raft_consensus.cc:515] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:13.999281  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076abc288e1345c6a56724b2d2c8dfc9" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:13.999928  5400 raft_consensus.cc:3060] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.000473  5516 tablet_bootstrap.cc:492] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:14.001063  5516 ts_tablet_manager.cc:1403] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.008s
I20260709 12:21:13.999078  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "076abc288e1345c6a56724b2d2c8dfc9" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:14.003286  5326 raft_consensus.cc:3060] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.004354  5516 raft_consensus.cc:359] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.005149  5516 raft_consensus.cc:385] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.005410  5516 raft_consensus.cc:740] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.006626  5516 consensus_queue.cc:260] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.008082  5537 leader_election.cc:290] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:14.008888  5516 ts_tablet_manager.cc:1434] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:14.010048  5516 tablet_bootstrap.cc:492] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:14.010545  5523 raft_consensus.cc:493] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.010869  5400 raft_consensus.cc:2468] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 1.
I20260709 12:21:14.011121  5523 raft_consensus.cc:515] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.012311  5442 leader_election.cc:304] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, 7acc97f6aaa944fb892f5f2f050449de; no voters: 
I20260709 12:21:14.013309  5546 raft_consensus.cc:2804] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:14.009936  5518 tablet_bootstrap.cc:492] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:14.014856  5518 ts_tablet_manager.cc:1403] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.005s
I20260709 12:21:14.018123  5516 tablet_bootstrap.cc:654] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.014252  5523 leader_election.cc:290] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.025874  5546 raft_consensus.cc:697] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:14.032277  5546 consensus_queue.cc:237] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.034099  5518 raft_consensus.cc:359] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.034905  5521 raft_consensus.cc:515] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.034934  5518 raft_consensus.cc:385] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.037003  5518 raft_consensus.cc:740] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.038538  5518 consensus_queue.cc:260] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.040848  5518 ts_tablet_manager.cc:1434] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.000s
I20260709 12:21:14.044281  5326 raft_consensus.cc:2468] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 1.
I20260709 12:21:14.044687  5327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e42c417384f41ef9b5f807b1875d3a1" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:14.045413  5327 raft_consensus.cc:3060] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.042239  5518 tablet_bootstrap.cc:492] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:14.056454  5517 tablet_bootstrap.cc:492] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:14.057113  5517 ts_tablet_manager.cc:1403] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.102s	user 0.023s	sys 0.017s
I20260709 12:21:14.060837  5517 raft_consensus.cc:359] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.061631  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:14.062371  5475 raft_consensus.cc:2468] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 0.
I20260709 12:21:14.063827  5291 leader_election.cc:304] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.064330  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:14.064685  5523 raft_consensus.cc:2804] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:14.061707  5517 raft_consensus.cc:385] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.065120  5523 raft_consensus.cc:493] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:14.065357  5517 raft_consensus.cc:740] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.065601  5523 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:14.066038  5293 leader_election.cc:343] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:14.066355  5517 consensus_queue.cc:260] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.070766  5327 raft_consensus.cc:2468] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 1.
I20260709 12:21:14.067174  5521 leader_election.cc:290] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:14.068507  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e42c417384f41ef9b5f807b1875d3a1" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:14.072358  5518 tablet_bootstrap.cc:654] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.072822  5441 leader_election.cc:304] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.074280  5554 raft_consensus.cc:2804] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:14.074784  5554 raft_consensus.cc:697] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:14.075659  5554 consensus_queue.cc:237] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.077515  5400 raft_consensus.cc:3060] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.082396  5523 raft_consensus.cc:515] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.084609  5523 leader_election.cc:290] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 election: Requested vote from peers 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.085589  5401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:14.085779  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:14.086496  5475 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:14.087422  5293 leader_election.cc:343] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:14.092844  5400 raft_consensus.cc:2468] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 1.
I20260709 12:21:14.097086  5475 raft_consensus.cc:2468] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 1.
I20260709 12:21:14.086520  5517 ts_tablet_manager.cc:1434] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.003s
I20260709 12:21:14.100137  5516 tablet_bootstrap.cc:492] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:14.099967  5291 leader_election.cc:304] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 06d54f6275524a6b99d7846015e61326
I20260709 12:21:14.100757  5516 ts_tablet_manager.cc:1403] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.012s
I20260709 12:21:14.101388  5523 raft_consensus.cc:2804] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:14.102005  5523 raft_consensus.cc:697] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 LEADER]: Becoming Leader. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Running, Role: LEADER
I20260709 12:21:14.102908  5523 consensus_queue.cc:237] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.100612  5517 tablet_bootstrap.cc:492] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:14.103955  5516 raft_consensus.cc:359] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.104679  5516 raft_consensus.cc:385] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.104945  5516 raft_consensus.cc:740] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.105688  5516 consensus_queue.cc:260] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.108408  5516 ts_tablet_manager.cc:1434] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:14.109803  5516 tablet_bootstrap.cc:492] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de: Bootstrap starting.
I20260709 12:21:14.111877  5517 tablet_bootstrap.cc:654] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.100124  5518 tablet_bootstrap.cc:492] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:14.113319  5518 ts_tablet_manager.cc:1403] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.013s
I20260709 12:21:14.117027  5518 raft_consensus.cc:359] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.123539  5518 raft_consensus.cc:385] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.122753  5222 catalog_manager.cc:5697] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 0 to 1, leader changed from <none> to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 1 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.123965  5518 raft_consensus.cc:740] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.125046  5518 consensus_queue.cc:260] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.126142  5222 catalog_manager.cc:5697] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 0 to 1, leader changed from <none> to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 1 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.128073  5518 ts_tablet_manager.cc:1434] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 12:21:14.129424  5518 tablet_bootstrap.cc:492] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:14.135787  5518 tablet_bootstrap.cc:654] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.123101  5516 tablet_bootstrap.cc:654] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.143216  5224 catalog_manager.cc:5697] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "bc6531ba57ad413a87df2a7cf63367c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.144564  5518 tablet_bootstrap.cc:492] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:14.144833  5517 tablet_bootstrap.cc:492] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:14.145153  5518 ts_tablet_manager.cc:1403] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.002s
I20260709 12:21:14.145349  5517 ts_tablet_manager.cc:1403] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.012s
I20260709 12:21:14.144220  5516 tablet_bootstrap.cc:492] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de: No bootstrap required, opened a new log
I20260709 12:21:14.147135  5516 ts_tablet_manager.cc:1403] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.004s
I20260709 12:21:14.148895  5518 raft_consensus.cc:359] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.149761  5518 raft_consensus.cc:385] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.150099  5518 raft_consensus.cc:740] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.150555  5517 raft_consensus.cc:359] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.150882  5518 consensus_queue.cc:260] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.152613  5518 ts_tablet_manager.cc:1434] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:21:14.152966  5516 raft_consensus.cc:359] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.153656  5518 tablet_bootstrap.cc:492] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:14.151280  5517 raft_consensus.cc:385] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.154230  5517 raft_consensus.cc:740] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.155735  5225 catalog_manager.cc:5697] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.155778  5517 consensus_queue.cc:260] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.153674  5516 raft_consensus.cc:385] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.159447  5516 raft_consensus.cc:740] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.160341  5516 consensus_queue.cc:260] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.161342  5518 tablet_bootstrap.cc:654] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.162189  5517 ts_tablet_manager.cc:1434] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:21:14.163267  5517 tablet_bootstrap.cc:492] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:14.171277  5517 tablet_bootstrap.cc:654] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.175426  5546 raft_consensus.cc:493] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.176005  5546 raft_consensus.cc:515] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.177942  5546 leader_election.cc:290] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:14.178428  5546 raft_consensus.cc:493] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.178798  5327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:14.179847  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
W20260709 12:21:14.180452  5441 leader_election.cc:343] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:14.181205  5442 leader_election.cc:343] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:14.178783  5546 raft_consensus.cc:515] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.181789  5442 leader_election.cc:304] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de; no voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:14.185772  5554 raft_consensus.cc:2749] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:14.186725  5542 raft_consensus.cc:493] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.187286  5542 raft_consensus.cc:515] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.187522  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:14.188320  5400 raft_consensus.cc:2468] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 0.
I20260709 12:21:14.189463  5542 leader_election.cc:290] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.189709  5516 ts_tablet_manager.cc:1434] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de: Time spent starting tablet: real 0.042s	user 0.001s	sys 0.004s
I20260709 12:21:14.189487  5327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:14.191850  5327 raft_consensus.cc:2468] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 0.
I20260709 12:21:14.191038  5442 leader_election.cc:304] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, 7acc97f6aaa944fb892f5f2f050449de; no voters: 
I20260709 12:21:14.184046  5546 leader_election.cc:290] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:14.192962  5546 raft_consensus.cc:2804] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:14.193205  5518 tablet_bootstrap.cc:492] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:14.193449  5546 raft_consensus.cc:493] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:14.193773  5518 ts_tablet_manager.cc:1403] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.040s	user 0.035s	sys 0.004s
I20260709 12:21:14.193859  5546 raft_consensus.cc:3060] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.191900  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:14.194581  5326 raft_consensus.cc:2468] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 0.
I20260709 12:21:14.193979  5523 raft_consensus.cc:493] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.195837  5367 leader_election.cc:304] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.196954  5518 raft_consensus.cc:359] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.197647  5518 raft_consensus.cc:385] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.198083  5518 raft_consensus.cc:740] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.198630  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:14.197614  5542 raft_consensus.cc:2804] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:14.199467  5542 raft_consensus.cc:493] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:14.199844  5542 raft_consensus.cc:3060] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.198951  5518 consensus_queue.cc:260] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.195665  5523 raft_consensus.cc:515] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.204953  5518 ts_tablet_manager.cc:1434] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 12:21:14.205021  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:14.205914  5400 raft_consensus.cc:2468] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 0.
I20260709 12:21:14.206159  5518 tablet_bootstrap.cc:492] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:14.206746  5546 raft_consensus.cc:515] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.207109  5293 leader_election.cc:304] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.207991  5475 raft_consensus.cc:2393] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06d54f6275524a6b99d7846015e61326 in current term 1: Already voted for candidate 7acc97f6aaa944fb892f5f2f050449de in this term.
I20260709 12:21:14.208199  5560 raft_consensus.cc:2804] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:14.208587  5560 raft_consensus.cc:493] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:14.208863  5560 raft_consensus.cc:3060] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.211058  5523 leader_election.cc:290] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.212517  5517 tablet_bootstrap.cc:492] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:14.213208  5517 ts_tablet_manager.cc:1403] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.002s
I20260709 12:21:14.214639  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:14.215432  5475 raft_consensus.cc:2468] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 0.
I20260709 12:21:14.217530  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:14.218227  5326 raft_consensus.cc:3060] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.218941  5542 raft_consensus.cc:515] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.220849  5518 tablet_bootstrap.cc:654] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.221858  5542 leader_election.cc:290] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.221997  5327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:14.224123  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:14.224751  5476 raft_consensus.cc:2393] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 06d54f6275524a6b99d7846015e61326 in current term 1: Already voted for candidate 7acc97f6aaa944fb892f5f2f050449de in this term.
I20260709 12:21:14.225700  5366 leader_election.cc:304] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d54f6275524a6b99d7846015e61326; no voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:14.227001  5542 raft_consensus.cc:2749] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:14.227298  5560 raft_consensus.cc:515] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.228361  5326 raft_consensus.cc:2468] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 1.
I20260709 12:21:14.229815  5441 leader_election.cc:304] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.226908  5517 raft_consensus.cc:359] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.229897  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:14.231024  5517 raft_consensus.cc:385] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.231187  5400 raft_consensus.cc:3060] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.231464  5517 raft_consensus.cc:740] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.232101  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:14.231118  5554 raft_consensus.cc:2804] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:14.233096  5554 raft_consensus.cc:697] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:14.232494  5517 consensus_queue.cc:260] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.233999  5554 consensus_queue.cc:237] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.235095  5475 raft_consensus.cc:3060] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.239311  5400 raft_consensus.cc:2468] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 1.
I20260709 12:21:14.235409  5560 leader_election.cc:290] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 election: Requested vote from peers 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.240566  5293 leader_election.cc:304] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.236007  5518 tablet_bootstrap.cc:492] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:14.241521  5518 ts_tablet_manager.cc:1403] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.009s
I20260709 12:21:14.242077  5560 raft_consensus.cc:2804] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:14.243930  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:14.237978  5517 ts_tablet_manager.cc:1434] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.005s
I20260709 12:21:14.244583  5518 raft_consensus.cc:359] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.244935  5546 leader_election.cc:290] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:14.245357  5518 raft_consensus.cc:385] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.245761  5560 raft_consensus.cc:697] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 LEADER]: Becoming Leader. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Running, Role: LEADER
I20260709 12:21:14.246447  5517 tablet_bootstrap.cc:492] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:14.244755  5400 raft_consensus.cc:2393] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7acc97f6aaa944fb892f5f2f050449de in current term 1: Already voted for candidate 06d54f6275524a6b99d7846015e61326 in this term.
I20260709 12:21:14.246703  5560 consensus_queue.cc:237] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.249205  5475 raft_consensus.cc:2468] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 1.
I20260709 12:21:14.245692  5518 raft_consensus.cc:740] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.250579  5518 consensus_queue.cc:260] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.253120  5518 ts_tablet_manager.cc:1434] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:21:14.254119  5518 tablet_bootstrap.cc:492] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326: Bootstrap starting.
I20260709 12:21:14.260313  5518 tablet_bootstrap.cc:654] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.264779  5224 catalog_manager.cc:5697] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 0 to 1, leader changed from <none> to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 1 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.271591  5517 tablet_bootstrap.cc:654] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:14.274546  5518 tablet_bootstrap.cc:492] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326: No bootstrap required, opened a new log
I20260709 12:21:14.275070  5518 ts_tablet_manager.cc:1403] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.007s
I20260709 12:21:14.277882  5518 raft_consensus.cc:359] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.278409  5518 raft_consensus.cc:385] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.278677  5518 raft_consensus.cc:740] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.279482  5518 consensus_queue.cc:260] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.281760  5518 ts_tablet_manager.cc:1434] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:21:14.284274  5224 catalog_manager.cc:5697] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "bc6531ba57ad413a87df2a7cf63367c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.288136  5517 tablet_bootstrap.cc:492] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: No bootstrap required, opened a new log
I20260709 12:21:14.288661  5517 ts_tablet_manager.cc:1403] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.000s
I20260709 12:21:14.291589  5517 raft_consensus.cc:359] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.292273  5517 raft_consensus.cc:385] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:14.292568  5517 raft_consensus.cc:740] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:14.293371  5517 consensus_queue.cc:260] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.297291  5517 ts_tablet_manager.cc:1434] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:14.350659  5560 raft_consensus.cc:493] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.351188  5560 raft_consensus.cc:515] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.353251  5560 leader_election.cc:290] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.354112  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:14.354343  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:14.354754  5400 raft_consensus.cc:2468] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 0.
I20260709 12:21:14.354959  5475 raft_consensus.cc:2468] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 0.
I20260709 12:21:14.355770  5293 leader_election.cc:304] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.356568  5560 raft_consensus.cc:2804] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:14.356945  5560 raft_consensus.cc:493] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:14.357198  5560 raft_consensus.cc:3060] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.358175  5542 raft_consensus.cc:493] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.358870  5542 raft_consensus.cc:515] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.361122  5542 leader_election.cc:290] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.361984  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7360d5e0f39948b1aaaf652369b525d3" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:14.362201  5542 raft_consensus.cc:493] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.362416  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7360d5e0f39948b1aaaf652369b525d3" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:14.363052  5326 raft_consensus.cc:2393] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06d54f6275524a6b99d7846015e61326 in current term 1: Already voted for candidate 7acc97f6aaa944fb892f5f2f050449de in this term.
I20260709 12:21:14.362808  5542 raft_consensus.cc:515] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.364475  5367 leader_election.cc:304] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d54f6275524a6b99d7846015e61326; no voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:14.365166  5527 raft_consensus.cc:3060] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.365557  5560 raft_consensus.cc:515] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.366325  5542 leader_election.cc:290] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.366631  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:14.367350  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:14.367755  5560 leader_election.cc:290] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 election: Requested vote from peers 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.368258  5475 raft_consensus.cc:2393] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06d54f6275524a6b99d7846015e61326 in current term 1: Already voted for candidate bc6531ba57ad413a87df2a7cf63367c1 in this term.
I20260709 12:21:14.369004  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:14.369310  5366 leader_election.cc:304] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d54f6275524a6b99d7846015e61326; no voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:14.369654  5476 raft_consensus.cc:3060] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.370743  5542 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.371187  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "bc6531ba57ad413a87df2a7cf63367c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:14.371798  5400 raft_consensus.cc:3060] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.374126  5527 raft_consensus.cc:2749] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:14.378362  5476 raft_consensus.cc:2468] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 1.
I20260709 12:21:14.378707  5400 raft_consensus.cc:2468] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc6531ba57ad413a87df2a7cf63367c1 in term 1.
I20260709 12:21:14.378971  5542 raft_consensus.cc:2749] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:14.379787  5291 leader_election.cc:304] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.380479  5560 raft_consensus.cc:2804] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:14.380870  5560 raft_consensus.cc:697] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 LEADER]: Becoming Leader. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Running, Role: LEADER
I20260709 12:21:14.381717  5560 consensus_queue.cc:237] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.389618  5225 catalog_manager.cc:5697] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "bc6531ba57ad413a87df2a7cf63367c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.394429  5542 consensus_queue.cc:1048] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.408964  5527 consensus_queue.cc:1048] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.416544  5554 consensus_queue.cc:1048] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:14.420228  5527 consensus_queue.cc:1048] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.433972  5554 consensus_queue.cc:1048] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.445258  5542 consensus_queue.cc:1048] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.452288  5554 consensus_queue.cc:1048] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.469033  5537 consensus_queue.cc:1048] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.515892  5537 raft_consensus.cc:493] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.516371  5537 raft_consensus.cc:515] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.518002  5542 raft_consensus.cc:493] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.518268  5537 leader_election.cc:290] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:14.518630  5542 raft_consensus.cc:515] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.519114  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:14.519495  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:14.519800  5326 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 0.
I20260709 12:21:14.520179  5400 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 0.
I20260709 12:21:14.520906  5542 leader_election.cc:290] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.521099  5441 leader_election.cc:304] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.521920  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1" is_pre_election: true
I20260709 12:21:14.522078  5537 raft_consensus.cc:2804] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:14.522140  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:14.522559  5326 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 0.
I20260709 12:21:14.522676  5537 raft_consensus.cc:493] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:14.523140  5537 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.523650  5367 leader_election.cc:304] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:14.524333  5542 raft_consensus.cc:2804] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:14.524621  5542 raft_consensus.cc:493] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:14.524879  5542 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.529073  5537 raft_consensus.cc:515] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.530117  5476 raft_consensus.cc:2393] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 06d54f6275524a6b99d7846015e61326 in current term 1: Already voted for candidate 7acc97f6aaa944fb892f5f2f050449de in this term.
I20260709 12:21:14.530241  5542 raft_consensus.cc:515] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.531513  5537 leader_election.cc:290] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:14.531782  5326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:14.532399  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:14.532964  5542 leader_election.cc:290] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:14.533185  5400 raft_consensus.cc:2393] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7acc97f6aaa944fb892f5f2f050449de in current term 1: Already voted for candidate 06d54f6275524a6b99d7846015e61326 in this term.
I20260709 12:21:14.533004  5327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:14.532424  5326 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:14.534176  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:14.534951  5476 raft_consensus.cc:2393] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 06d54f6275524a6b99d7846015e61326 in current term 1: Already voted for candidate 7acc97f6aaa944fb892f5f2f050449de in this term.
I20260709 12:21:14.536171  5366 leader_election.cc:304] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d54f6275524a6b99d7846015e61326; no voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:14.536929  5542 raft_consensus.cc:2749] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:14.540277  5326 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 1.
I20260709 12:21:14.541280  5441 leader_election.cc:304] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 06d54f6275524a6b99d7846015e61326
I20260709 12:21:14.542038  5537 raft_consensus.cc:2804] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:14.542654  5537 raft_consensus.cc:697] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:14.543478  5537 consensus_queue.cc:237] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:14.545274  5521 consensus_queue.cc:1048] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.552407  5225 catalog_manager.cc:5697] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 0 to 1, leader changed from <none> to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 1 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.559015  5537 consensus_queue.cc:1048] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.571990  5560 consensus_queue.cc:1048] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.583540  5560 consensus_queue.cc:1048] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.641127  5537 consensus_queue.cc:1048] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.651722  5546 consensus_queue.cc:1048] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.706178  5523 consensus_queue.cc:1048] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.720225  5591 consensus_queue.cc:1048] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:14.771817 27116 ts_manager.cc:295] Set tserver state for bc6531ba57ad413a87df2a7cf63367c1 to MAINTENANCE_MODE
I20260709 12:21:14.772733 27116 tablet_server.cc:179] TabletServer@127.26.123.1:0 shutting down...
I20260709 12:21:14.789233  5591 consensus_queue.cc:1048] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:14.797298 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:14.798448 27116 tablet_replica.cc:333] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.799263 27116 raft_consensus.cc:2243] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.799808 27116 raft_consensus.cc:2272] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.802101 27116 tablet_replica.cc:333] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.802904 27116 raft_consensus.cc:2243] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:14.804154 27116 raft_consensus.cc:2272] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.808388 27116 tablet_replica.cc:333] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.809368 27116 raft_consensus.cc:2243] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.809345  5596 consensus_queue.cc:1048] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:14.810312 27116 raft_consensus.cc:2272] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.813565 27116 tablet_replica.cc:333] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.814215 27116 raft_consensus.cc:2243] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.814680 27116 raft_consensus.cc:2272] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.817251 27116 tablet_replica.cc:333] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.818070 27116 raft_consensus.cc:2243] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.818495 27116 raft_consensus.cc:2272] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.821074 27116 tablet_replica.cc:333] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.821676 27116 raft_consensus.cc:2243] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:14.822599 27116 raft_consensus.cc:2272] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.825038 27116 tablet_replica.cc:333] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.825616 27116 raft_consensus.cc:2243] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.826108 27116 raft_consensus.cc:2272] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.828552 27116 tablet_replica.cc:333] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.829200 27116 raft_consensus.cc:2243] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:14.830096 27116 raft_consensus.cc:2272] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.832253 27116 tablet_replica.cc:333] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.832780 27116 raft_consensus.cc:2243] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.833202 27116 raft_consensus.cc:2272] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:14.835625 27116 tablet_replica.cc:333] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: stopping tablet replica
I20260709 12:21:14.836125 27116 raft_consensus.cc:2243] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:14.836535 27116 raft_consensus.cc:2272] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:14.869078  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:14.869673 27116 tablet_server.cc:196] TabletServer@127.26.123.1:0 shutdown complete.
W20260709 12:21:14.905887  5367 consensus_peers.cc:597] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:14.923816  5441 consensus_peers.cc:597] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:14.966751  5441 consensus_peers.cc:597] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:14.968685  5367 consensus_peers.cc:597] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:15.043962  5441 consensus_peers.cc:597] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:15.117194  5441 consensus_peers.cc:597] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:15.170444  5546 consensus_queue.cc:1048] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:21:15.249918  5441 consensus_peers.cc:597] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:15.622869  5441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111) [suppressed 174 similar messages]
W20260709 12:21:15.870473  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:16.081084  5601 raft_consensus.cc:493] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc6531ba57ad413a87df2a7cf63367c1)
I20260709 12:21:16.081779  5601 raft_consensus.cc:515] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.083750  5601 leader_election.cc:290] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:16.085040  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:16.086315  5546 raft_consensus.cc:493] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc6531ba57ad413a87df2a7cf63367c1)
I20260709 12:21:16.086922  5546 raft_consensus.cc:515] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
W20260709 12:21:16.087601  5367 leader_election.cc:336] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111)
I20260709 12:21:16.088177  5367 leader_election.cc:304] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d54f6275524a6b99d7846015e61326; no voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:16.088984  5601 raft_consensus.cc:2749] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:16.089121  5546 leader_election.cc:290] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:16.090407  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:16.091109  5400 raft_consensus.cc:2468] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 1.
I20260709 12:21:16.092410  5442 leader_election.cc:304] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, 7acc97f6aaa944fb892f5f2f050449de; no voters: 
W20260709 12:21:16.093025  5441 leader_election.cc:336] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111)
I20260709 12:21:16.093258  5546 raft_consensus.cc:2804] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:16.093700  5546 raft_consensus.cc:493] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc6531ba57ad413a87df2a7cf63367c1)
I20260709 12:21:16.094091  5546 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.100976  5546 raft_consensus.cc:515] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.102869  5546 leader_election.cc:290] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:16.103979  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:16.104681  5400 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:16.106487  5441 leader_election.cc:336] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111)
I20260709 12:21:16.111404  5400 raft_consensus.cc:2468] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 2.
I20260709 12:21:16.112576  5442 leader_election.cc:304] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, 7acc97f6aaa944fb892f5f2f050449de; no voters: bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:16.113330  5546 raft_consensus.cc:2804] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:16.113839  5546 raft_consensus.cc:697] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:16.114612  5546 consensus_queue.cc:237] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.122422  5225 catalog_manager.cc:5697] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 1 to 2, leader changed from bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1) to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 2 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:16.217360  5546 raft_consensus.cc:493] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc6531ba57ad413a87df2a7cf63367c1)
I20260709 12:21:16.217916  5546 raft_consensus.cc:515] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.219949  5546 leader_election.cc:290] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:16.221131  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:16.223390  5601 raft_consensus.cc:493] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc6531ba57ad413a87df2a7cf63367c1)
W20260709 12:21:16.223620  5441 leader_election.cc:336] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111)
I20260709 12:21:16.223830  5601 raft_consensus.cc:515] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.224118  5441 leader_election.cc:304] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de; no voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:16.224803  5546 raft_consensus.cc:2749] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:16.225994  5601 leader_election.cc:290] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:16.227155  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de" is_pre_election: true
I20260709 12:21:16.227845  5476 raft_consensus.cc:2468] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 1.
I20260709 12:21:16.228854  5366 leader_election.cc:304] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, 7acc97f6aaa944fb892f5f2f050449de; no voters: 
W20260709 12:21:16.229535  5367 leader_election.cc:336] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111)
I20260709 12:21:16.229542  5601 raft_consensus.cc:2804] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:16.230105  5601 raft_consensus.cc:493] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc6531ba57ad413a87df2a7cf63367c1)
I20260709 12:21:16.230398  5601 raft_consensus.cc:3060] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.236689  5601 raft_consensus.cc:515] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.238509  5601 leader_election.cc:290] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:16.239735  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:16.240481  5476 raft_consensus.cc:3060] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:16.242887  5367 leader_election.cc:336] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111)
I20260709 12:21:16.247392  5476 raft_consensus.cc:2468] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 2.
I20260709 12:21:16.248653  5366 leader_election.cc:304] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, 7acc97f6aaa944fb892f5f2f050449de; no voters: bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:16.249428  5601 raft_consensus.cc:2804] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:16.250536  5601 raft_consensus.cc:697] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:16.251293  5601 consensus_queue.cc:237] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.258553  5225 catalog_manager.cc:5697] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 1 to 2, leader changed from bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1) to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:16.309108  5546 raft_consensus.cc:493] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc6531ba57ad413a87df2a7cf63367c1)
I20260709 12:21:16.309765  5546 raft_consensus.cc:515] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.311743  5546 leader_election.cc:290] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:16.313051  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326" is_pre_election: true
I20260709 12:21:16.313900  5400 raft_consensus.cc:2468] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 1.
I20260709 12:21:16.315035  5442 leader_election.cc:304] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, 7acc97f6aaa944fb892f5f2f050449de; no voters: 
W20260709 12:21:16.315627  5441 leader_election.cc:336] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111)
I20260709 12:21:16.315883  5546 raft_consensus.cc:2804] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:16.316380  5546 raft_consensus.cc:493] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc6531ba57ad413a87df2a7cf63367c1)
I20260709 12:21:16.316740  5546 raft_consensus.cc:3060] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:16.322952  5546 raft_consensus.cc:515] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.324784  5546 leader_election.cc:290] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:16.326159  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:16.326879  5400 raft_consensus.cc:3060] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:16.328590  5441 leader_election.cc:336] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111)
I20260709 12:21:16.334283  5400 raft_consensus.cc:2468] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 2.
I20260709 12:21:16.335546  5442 leader_election.cc:304] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, 7acc97f6aaa944fb892f5f2f050449de; no voters: bc6531ba57ad413a87df2a7cf63367c1
I20260709 12:21:16.336283  5546 raft_consensus.cc:2804] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:16.336818  5546 raft_consensus.cc:697] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:16.337707  5546 consensus_queue.cc:237] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:16.345542  5225 catalog_manager.cc:5697] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 1 to 2, leader changed from bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1) to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 2 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:16.657223  5476 raft_consensus.cc:1275] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.658598  5601 consensus_queue.cc:1048] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:16.662933  5401 raft_consensus.cc:1275] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.664611  5546 consensus_queue.cc:1048] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:16.674602  5367 consensus_peers.cc:597] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:16.687494  5441 consensus_peers.cc:597] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:16.871667  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:21:16.908644  5441 consensus_peers.cc:597] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:16.949450  5401 raft_consensus.cc:1275] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.951057  5615 consensus_queue.cc:1048] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:17.146673  5441 consensus_peers.cc:597] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:17.228485  5367 consensus_peers.cc:597] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:17.306313  5367 consensus_peers.cc:597] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:17.633493  5441 consensus_peers.cc:597] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:17.695574  5441 consensus_peers.cc:597] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:17.731635  5441 consensus_peers.cc:597] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:17.751752  5441 consensus_peers.cc:597] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:17.872852  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:21:18.874092  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:21:19.061378  5367 consensus_peers.cc:597] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:19.075649  5441 consensus_peers.cc:597] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:19.386198  5441 consensus_peers.cc:597] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:19.693261  5367 consensus_peers.cc:597] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:19.740464  5441 consensus_peers.cc:597] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:19.754518  5367 consensus_peers.cc:597] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:19.875159  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:20.019479 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:20.026346  5625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:20.027213  5626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:20.029723  5628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:20.031502 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:20.033066 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:20.033301 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:20.033510 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599680033489 us; error 0 us; skew 500 ppm
I20260709 12:21:20.034160 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:20.036758 27116 webserver.cc:533] Webserver started at http://127.26.123.1:34373/ using document root <none> and password file <none>
I20260709 12:21:20.037356 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:20.037550 27116 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:21:20.041646 27116 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:21:20.045920  5633 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:20.046780 27116 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:21:20.047137 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "bc6531ba57ad413a87df2a7cf63367c1"
format_stamp: "Formatted at 2026-07-09 12:21:12 on dist-test-slave-9q7n"
I20260709 12:21:20.047454 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:20.062953 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:20.064183 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:20.065877 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:20.069788  5640 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 12:21:20.131376 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:21:20.131711 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.063s	user 0.003s	sys 0.000s
I20260709 12:21:20.131986 27116 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:21:20.138202  5640 tablet_bootstrap.cc:492] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.154621  5640 tablet_bootstrap.cc:492] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.156025  5640 tablet_bootstrap.cc:492] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
W20260709 12:21:20.156790  5441 consensus_peers.cc:597] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:20.156992  5640 ts_tablet_manager.cc:1403] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.005s
I20260709 12:21:20.160166  5640 raft_consensus.cc:359] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.160758  5640 raft_consensus.cc:740] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.161490  5640 consensus_queue.cc:260] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.163956  5640 ts_tablet_manager.cc:1434] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:21:20.164745  5640 tablet_bootstrap.cc:492] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.178635  5640 tablet_bootstrap.cc:492] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.179472  5640 tablet_bootstrap.cc:492] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
I20260709 12:21:20.180100  5640 ts_tablet_manager.cc:1403] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.006s
I20260709 12:21:20.182426  5640 raft_consensus.cc:359] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.183005  5640 raft_consensus.cc:740] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.183701  5640 consensus_queue.cc:260] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.185299  5640 ts_tablet_manager.cc:1434] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:21:20.186102  5640 tablet_bootstrap.cc:492] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.189284 27116 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:21:20.189654 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.058s	user 0.051s	sys 0.003s
I20260709 12:21:20.200488  5640 tablet_bootstrap.cc:492] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.201396  5640 tablet_bootstrap.cc:492] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
I20260709 12:21:20.202221  5640 ts_tablet_manager.cc:1403] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.004s
I20260709 12:21:20.204918  5640 raft_consensus.cc:359] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.205523  5640 raft_consensus.cc:740] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.206220  5640 consensus_queue.cc:260] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.208076  5640 ts_tablet_manager.cc:1434] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.003s
I20260709 12:21:20.208792  5640 tablet_bootstrap.cc:492] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.221311  5640 tablet_bootstrap.cc:492] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.222337  5640 tablet_bootstrap.cc:492] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
I20260709 12:21:20.222959  5640 ts_tablet_manager.cc:1403] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 12:21:20.225262  5640 raft_consensus.cc:359] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.225948  5640 raft_consensus.cc:740] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.226562  5640 consensus_queue.cc:260] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.228327  5640 ts_tablet_manager.cc:1434] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:20.229055  5640 tablet_bootstrap.cc:492] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.243058  5640 tablet_bootstrap.cc:492] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.244226  5640 tablet_bootstrap.cc:492] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
I20260709 12:21:20.245078  5640 ts_tablet_manager.cc:1403] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:21:20.248275  5640 raft_consensus.cc:359] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:20.248970 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:33819
I20260709 12:21:20.249069  5640 raft_consensus.cc:740] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.249094  5703 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:33819 every 8 connection(s)
I20260709 12:21:20.250346  5640 consensus_queue.cc:260] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:20.252153  5640 ts_tablet_manager.cc:1434] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:20.253115  5640 tablet_bootstrap.cc:492] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.268716 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:20.299036  5704 heartbeater.cc:344] Connected to a master server at 127.26.123.62:40191
I20260709 12:21:20.299548  5704 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:20.300611  5704 heartbeater.cc:507] Master 127.26.123.62:40191 requested a full tablet report, sending...
I20260709 12:21:20.302800  5640 tablet_bootstrap.cc:492] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:21:20.303457  5497 debug-util.cc:398] Leaking SignalData structure 0x7b080009bb80 after lost signal to thread 27119
I20260709 12:21:20.303982  5640 tablet_bootstrap.cc:492] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
I20260709 12:21:20.304864  5640 ts_tablet_manager.cc:1403] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.020s
W20260709 12:21:20.306124  5497 debug-util.cc:398] Leaking SignalData structure 0x7b08002bee20 after lost signal to thread 5426
I20260709 12:21:20.307538  5640 raft_consensus.cc:359] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.308339  5640 raft_consensus.cc:740] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.309159  5640 consensus_queue.cc:260] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.309509  5224 ts_manager.cc:194] Re-registered known tserver with Master: bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819)
I20260709 12:21:20.311757  5640 ts_tablet_manager.cc:1434] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:20.312538  5475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
mode: GRACEFUL
new_leader_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:47466
I20260709 12:21:20.313424  5475 raft_consensus.cc:606] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Received request to transfer leadership to TS 06d54f6275524a6b99d7846015e61326
I20260709 12:21:20.313930  5640 tablet_bootstrap.cc:492] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.316892  5475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73c014a910d24602b825542f7fa781a1"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
mode: GRACEFUL
new_leader_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:47466
I20260709 12:21:20.317848  5475 raft_consensus.cc:606] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Received request to transfer leadership to TS 06d54f6275524a6b99d7846015e61326
I20260709 12:21:20.321482  5475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
mode: GRACEFUL
new_leader_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:47466
I20260709 12:21:20.322184  5224 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53556
I20260709 12:21:20.322474  5475 raft_consensus.cc:606] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Received request to transfer leadership to TS 06d54f6275524a6b99d7846015e61326
I20260709 12:21:20.324214 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:21:20.325179 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:20.326129 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:20.326519  5704 heartbeater.cc:499] Master 127.26.123.62:40191 was elected leader, sending a full tablet report...
I20260709 12:21:20.341010  5640 tablet_bootstrap.cc:492] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.342202  5640 tablet_bootstrap.cc:492] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
I20260709 12:21:20.343087  5640 ts_tablet_manager.cc:1403] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.003s
I20260709 12:21:20.346081  5640 raft_consensus.cc:359] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.346901  5640 raft_consensus.cc:740] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.347681  5640 consensus_queue.cc:260] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.351404  5640 ts_tablet_manager.cc:1434] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:21:20.352372  5640 tablet_bootstrap.cc:492] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.354903  5623 raft_consensus.cc:993] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de: : Instructing follower 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:20.357267  5642 raft_consensus.cc:1081] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Signalling peer 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:20.359144  5401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73c014a910d24602b825542f7fa781a1"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:57564
I20260709 12:21:20.359619  5678 raft_consensus.cc:3060] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.359865  5401 raft_consensus.cc:493] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.360356  5401 raft_consensus.cc:3060] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.370790  5401 raft_consensus.cc:515] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.373792  5401 leader_election.cc:290] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:20.374866  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:20.375563  5677 raft_consensus.cc:3060] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.379254  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73c014a910d24602b825542f7fa781a1" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:20.380028  5475 raft_consensus.cc:3055] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:20.380332  5475 raft_consensus.cc:740] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:20.381212  5475 consensus_queue.cc:260] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.384269  5475 raft_consensus.cc:3060] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.390448  5640 tablet_bootstrap.cc:492] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.391688  5640 tablet_bootstrap.cc:492] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
I20260709 12:21:20.392450  5640 ts_tablet_manager.cc:1403] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.020s
I20260709 12:21:20.395399  5640 raft_consensus.cc:359] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.396150  5640 raft_consensus.cc:740] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.396915  5640 consensus_queue.cc:260] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.399456  5640 ts_tablet_manager.cc:1434] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:20.400449  5640 tablet_bootstrap.cc:492] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.401027  5677 raft_consensus.cc:2468] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 2.
I20260709 12:21:20.404531  5367 leader_election.cc:304] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:20.405565  5624 raft_consensus.cc:2804] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:20.406139  5624 raft_consensus.cc:697] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:20.407043  5624 consensus_queue.cc:237] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.436069  5225 catalog_manager.cc:5697] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 1 to 2, leader changed from 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3) to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.424273  5475 raft_consensus.cc:2468] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 2.
I20260709 12:21:20.463649  5640 tablet_bootstrap.cc:492] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.464926  5640 tablet_bootstrap.cc:492] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
I20260709 12:21:20.465909  5640 ts_tablet_manager.cc:1403] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.066s	user 0.039s	sys 0.024s
I20260709 12:21:20.468880  5640 raft_consensus.cc:359] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.469728  5640 raft_consensus.cc:740] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.470690  5640 consensus_queue.cc:260] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.473011  5640 ts_tablet_manager.cc:1434] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:20.474053  5640 tablet_bootstrap.cc:492] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap starting.
I20260709 12:21:20.490653  5640 tablet_bootstrap.cc:492] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:20.491689  5640 tablet_bootstrap.cc:492] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: Bootstrap complete.
I20260709 12:21:20.492466  5640 ts_tablet_manager.cc:1403] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.003s
I20260709 12:21:20.495369  5640 raft_consensus.cc:359] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.496126  5640 raft_consensus.cc:740] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc6531ba57ad413a87df2a7cf63367c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:20.496886  5640 consensus_queue.cc:260] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.499084  5640 ts_tablet_manager.cc:1434] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:21:20.630116  5677 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.746845  5623 raft_consensus.cc:993] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de: : Instructing follower 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:20.747381  5642 raft_consensus.cc:1081] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Signalling peer 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:20.749171  5401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:57564
I20260709 12:21:20.749835  5401 raft_consensus.cc:493] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.750169  5401 raft_consensus.cc:3060] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.756336  5401 raft_consensus.cc:515] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.758440  5401 leader_election.cc:290] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 3 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:20.759388  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:20.759721  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feedeecfd9247498d0a76ac3d5e7e26" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:20.760257  5677 raft_consensus.cc:3060] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.760452  5475 raft_consensus.cc:3055] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:20.760864  5475 raft_consensus.cc:740] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:20.761685  5475 consensus_queue.cc:260] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.762871  5475 raft_consensus.cc:3060] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.765635  5678 raft_consensus.cc:3060] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.770289  5677 raft_consensus.cc:2468] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 3.
I20260709 12:21:20.771687  5475 raft_consensus.cc:2468] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 3.
I20260709 12:21:20.771986  5367 leader_election.cc:304] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:20.774020  5624 raft_consensus.cc:2804] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:20.774720  5624 raft_consensus.cc:697] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 3 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:20.776051  5624 consensus_queue.cc:237] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.801020  5623 raft_consensus.cc:993] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de: : Instructing follower 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:20.801810  5642 raft_consensus.cc:1081] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Signalling peer 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:20.800968  5225 catalog_manager.cc:5697] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 2 to 3, leader changed from 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3) to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.803557  5401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:57564
I20260709 12:21:20.804148  5401 raft_consensus.cc:493] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.804509  5401 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.814396  5475 raft_consensus.cc:1275] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:20.816408  5624 consensus_queue.cc:1048] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:21:20.820035  5401 raft_consensus.cc:515] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.824014  5678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:20.824719  5678 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.824676  5476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:20.825807  5476 raft_consensus.cc:3055] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:20.826216  5476 raft_consensus.cc:740] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:20.827343  5476 consensus_queue.cc:260] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.828895  5476 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.834290  5678 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 2.
I20260709 12:21:20.835937  5367 leader_election.cc:304] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:20.839359  5401 leader_election.cc:290] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:20.841136  5619 raft_consensus.cc:2804] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:20.850008  5677 raft_consensus.cc:1275] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:20.854909  5476 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 2.
I20260709 12:21:20.855191  5619 raft_consensus.cc:697] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:20.856496  5619 consensus_queue.cc:237] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:20.861053  5733 consensus_queue.cc:1048] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:20.876281  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:20.876052  5222 catalog_manager.cc:5697] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 1 to 2, leader changed from 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3) to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:21.189323  5475 raft_consensus.cc:1275] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:21.190932  5619 consensus_queue.cc:1048] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:21.200196  5677 raft_consensus.cc:1275] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:21.202212  5624 consensus_queue.cc:1048] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:21.305320  5475 raft_consensus.cc:1275] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.307008  5619 consensus_queue.cc:1048] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:21.319904  5677 raft_consensus.cc:1275] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.321659  5624 consensus_queue.cc:1048] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:21.337998 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:21.357438  5401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
mode: GRACEFUL
new_leader_uuid: "7acc97f6aaa944fb892f5f2f050449de"
 from {username='slave'} at 127.0.0.1:57578
I20260709 12:21:21.358158  5401 raft_consensus.cc:606] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 1 LEADER]: Received request to transfer leadership to TS 7acc97f6aaa944fb892f5f2f050449de
I20260709 12:21:21.360725  5401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
mode: GRACEFUL
new_leader_uuid: "7acc97f6aaa944fb892f5f2f050449de"
 from {username='slave'} at 127.0.0.1:57578
I20260709 12:21:21.361315  5401 raft_consensus.cc:606] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Received request to transfer leadership to TS 7acc97f6aaa944fb892f5f2f050449de
I20260709 12:21:21.362546 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:21.363235 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:21.363986 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:21.651950  5619 raft_consensus.cc:993] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326: : Instructing follower 7acc97f6aaa944fb892f5f2f050449de to start an election
I20260709 12:21:21.652963  5624 raft_consensus.cc:1081] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 1 LEADER]: Signalling peer 7acc97f6aaa944fb892f5f2f050449de to start an election
I20260709 12:21:21.655237  5475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
 from {username='slave'} at 127.0.0.1:47438
I20260709 12:21:21.656219  5475 raft_consensus.cc:493] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:21.656734  5475 raft_consensus.cc:3060] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:21.672503  5475 raft_consensus.cc:515] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:21.675455  5475 leader_election.cc:290] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:21.676681  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:21.676857  5401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:21.677820  5677 raft_consensus.cc:3060] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:21.677879  5401 raft_consensus.cc:3055] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:21.678714  5401 raft_consensus.cc:740] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:21.680184  5401 consensus_queue.cc:260] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:21.682561  5401 raft_consensus.cc:3060] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:21.690184  5677 raft_consensus.cc:2468] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 2.
I20260709 12:21:21.692154  5441 leader_election.cc:304] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:21.693372  5623 raft_consensus.cc:2804] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:21.693969  5623 raft_consensus.cc:697] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:21.694227  5401 raft_consensus.cc:2468] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 2.
I20260709 12:21:21.695365  5623 consensus_queue.cc:237] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:21.704757  5619 raft_consensus.cc:993] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326: : Instructing follower 7acc97f6aaa944fb892f5f2f050449de to start an election
I20260709 12:21:21.705427  5624 raft_consensus.cc:1081] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Signalling peer 7acc97f6aaa944fb892f5f2f050449de to start an election
I20260709 12:21:21.707288  5475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
 from {username='slave'} at 127.0.0.1:47438
I20260709 12:21:21.707916  5475 raft_consensus.cc:493] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:21.706812  5225 catalog_manager.cc:5697] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 1 to 2, leader changed from 06d54f6275524a6b99d7846015e61326 (127.26.123.2) to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 2 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:21.708271  5475 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:21.717422  5475 raft_consensus.cc:515] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:21.721020  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:21.721149  5401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:21.721880  5401 raft_consensus.cc:3055] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:21.722039  5677 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:21.722329  5401 raft_consensus.cc:740] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:21.723632  5401 consensus_queue.cc:260] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:21.725523  5401 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:21.731945  5677 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 3.
I20260709 12:21:21.734249  5441 leader_election.cc:304] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:21.732653  5475 leader_election.cc:290] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 3 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:21.735242  5623 raft_consensus.cc:2804] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:21.735854  5623 raft_consensus.cc:697] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 3 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:21.736275  5401 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 3.
I20260709 12:21:21.737282  5623 consensus_queue.cc:237] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:21.753175  5225 catalog_manager.cc:5697] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 2 to 3, leader changed from 06d54f6275524a6b99d7846015e61326 (127.26.123.2) to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 3 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:21.877887  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:22.080387  5677 raft_consensus.cc:1275] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:22.082065  5623 consensus_queue.cc:1048] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:22.099901  5400 raft_consensus.cc:1275] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:22.102157  5752 consensus_queue.cc:1048] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:22.128109  5677 raft_consensus.cc:1275] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:22.129426  5752 consensus_queue.cc:1048] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:22.133054  5400 raft_consensus.cc:1275] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:22.134687  5752 consensus_queue.cc:1048] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:22.374271 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:22.385313  5475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e42c417384f41ef9b5f807b1875d3a1"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
mode: GRACEFUL
new_leader_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:47466
I20260709 12:21:22.386025  5475 raft_consensus.cc:606] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Received request to transfer leadership to TS 06d54f6275524a6b99d7846015e61326
I20260709 12:21:22.389302  5475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
mode: GRACEFUL
new_leader_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:47466
I20260709 12:21:22.389992  5475 raft_consensus.cc:606] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Received request to transfer leadership to TS 06d54f6275524a6b99d7846015e61326
I20260709 12:21:22.391407 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:22.392184 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:22.393029 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:22.441682  5714 raft_consensus.cc:993] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de: : Instructing follower 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:22.442389  5623 raft_consensus.cc:1081] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Signalling peer 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:22.444209  5400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e42c417384f41ef9b5f807b1875d3a1"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:57564
I20260709 12:21:22.444888  5400 raft_consensus.cc:493] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:22.445297  5400 raft_consensus.cc:3060] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.456362  5400 raft_consensus.cc:515] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:22.458918  5400 leader_election.cc:290] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:22.459908  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e42c417384f41ef9b5f807b1875d3a1" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:22.460253  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e42c417384f41ef9b5f807b1875d3a1" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:22.460721  5677 raft_consensus.cc:3060] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.460996  5475 raft_consensus.cc:3055] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:22.461391  5475 raft_consensus.cc:740] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:22.462464  5475 consensus_queue.cc:260] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:22.463748  5475 raft_consensus.cc:3060] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.470074  5677 raft_consensus.cc:2468] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 2.
I20260709 12:21:22.471640  5367 leader_election.cc:304] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:22.472246  5475 raft_consensus.cc:2468] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 2.
I20260709 12:21:22.472566  5624 raft_consensus.cc:2804] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:22.473965  5624 raft_consensus.cc:697] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:22.475092  5624 consensus_queue.cc:237] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:22.483887  5224 catalog_manager.cc:5697] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 1 to 2, leader changed from 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3) to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:22.503533  5714 raft_consensus.cc:993] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de: : Instructing follower 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:22.504107  5623 raft_consensus.cc:1081] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Signalling peer 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:22.506187  5400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:57564
I20260709 12:21:22.506852  5400 raft_consensus.cc:493] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:22.507249  5400 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:22.514372  5400 raft_consensus.cc:515] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:22.516201  5400 leader_election.cc:290] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 3 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:22.517469  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:22.517366  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:22.518428  5475 raft_consensus.cc:3055] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:22.518604  5677 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:22.518788  5475 raft_consensus.cc:740] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:22.519980  5475 consensus_queue.cc:260] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:22.521239  5475 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:22.528228  5677 raft_consensus.cc:2468] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 3.
I20260709 12:21:22.528954  5475 raft_consensus.cc:2468] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 3.
I20260709 12:21:22.529798  5367 leader_election.cc:304] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:22.530743  5624 raft_consensus.cc:2804] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:22.531165  5624 raft_consensus.cc:697] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 3 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:22.532313  5624 consensus_queue.cc:237] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:22.540822  5224 catalog_manager.cc:5697] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 2 to 3, leader changed from 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3) to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:22.879210  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:22.942255  5475 raft_consensus.cc:1275] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:22.943898  5624 consensus_queue.cc:1048] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:22.962314  5677 raft_consensus.cc:1275] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:22.964424  5756 consensus_queue.cc:1048] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:22.972697  5678 raft_consensus.cc:1275] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:22.975919  5756 consensus_queue.cc:1048] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:22.999266  5475 raft_consensus.cc:1275] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:23.001021  5756 consensus_queue.cc:1048] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:23.400300 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:23.408257  5400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
mode: GRACEFUL
new_leader_uuid: "7acc97f6aaa944fb892f5f2f050449de"
 from {username='slave'} at 127.0.0.1:57578
I20260709 12:21:23.408979  5400 raft_consensus.cc:606] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Received request to transfer leadership to TS 7acc97f6aaa944fb892f5f2f050449de
I20260709 12:21:23.412220  5400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
mode: GRACEFUL
new_leader_uuid: "7acc97f6aaa944fb892f5f2f050449de"
 from {username='slave'} at 127.0.0.1:57578
I20260709 12:21:23.412897  5400 raft_consensus.cc:606] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 3 LEADER]: Received request to transfer leadership to TS 7acc97f6aaa944fb892f5f2f050449de
I20260709 12:21:23.414251 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:23.415066 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:23.415907 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:23.482486  5624 raft_consensus.cc:993] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326: : Instructing follower 7acc97f6aaa944fb892f5f2f050449de to start an election
I20260709 12:21:23.483114  5756 raft_consensus.cc:1081] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 3 LEADER]: Signalling peer 7acc97f6aaa944fb892f5f2f050449de to start an election
I20260709 12:21:23.484747  5475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
 from {username='slave'} at 127.0.0.1:47438
I20260709 12:21:23.485257  5475 raft_consensus.cc:493] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:23.485623  5475 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:23.495422  5475 raft_consensus.cc:515] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:23.497927  5475 leader_election.cc:290] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 4 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:23.498368  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:23.498957  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe4161fa651346d69b76af4ca768e397" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:23.499213  5677 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:23.499615  5400 raft_consensus.cc:3055] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:23.499986  5400 raft_consensus.cc:740] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:23.500814  5400 consensus_queue.cc:260] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:23.502250  5400 raft_consensus.cc:3060] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:23.506752  5677 raft_consensus.cc:2468] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 4.
I20260709 12:21:23.508483  5441 leader_election.cc:304] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:23.510020  5623 raft_consensus.cc:2804] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:23.511590  5623 raft_consensus.cc:697] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 4 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:23.511667  5400 raft_consensus.cc:2468] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 4.
I20260709 12:21:23.512578  5623 consensus_queue.cc:237] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:23.520277  5222 catalog_manager.cc:5697] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 3 to 4, leader changed from 06d54f6275524a6b99d7846015e61326 (127.26.123.2) to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 4 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:23.856354  5756 raft_consensus.cc:993] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326: : Instructing follower 7acc97f6aaa944fb892f5f2f050449de to start an election
I20260709 12:21:23.856871  5768 raft_consensus.cc:1081] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Signalling peer 7acc97f6aaa944fb892f5f2f050449de to start an election
I20260709 12:21:23.858498  5475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
 from {username='slave'} at 127.0.0.1:47438
I20260709 12:21:23.859016  5475 raft_consensus.cc:493] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:23.859319  5475 raft_consensus.cc:3060] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:23.866142  5475 raft_consensus.cc:515] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:23.867832  5475 leader_election.cc:290] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 3 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697)
I20260709 12:21:23.868839  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:23.868911  5400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cce83284c7664c1da3f565197bacbe18" candidate_uuid: "7acc97f6aaa944fb892f5f2f050449de" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "06d54f6275524a6b99d7846015e61326"
I20260709 12:21:23.869652  5677 raft_consensus.cc:3060] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:23.869722  5400 raft_consensus.cc:3055] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:23.870203  5400 raft_consensus.cc:740] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:23.870901  5400 consensus_queue.cc:260] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:23.871991  5400 raft_consensus.cc:3060] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:23.876246  5677 raft_consensus.cc:2468] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 3.
I20260709 12:21:23.877502  5441 leader_election.cc:304] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7acc97f6aaa944fb892f5f2f050449de, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:23.877998  5400 raft_consensus.cc:2468] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7acc97f6aaa944fb892f5f2f050449de in term 3.
I20260709 12:21:23.878331  5623 raft_consensus.cc:2804] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:23.878953  5623 raft_consensus.cc:697] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 3 LEADER]: Becoming Leader. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
W20260709 12:21:23.880201  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:23.879984  5623 consensus_queue.cc:237] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:23.888015  5222 catalog_manager.cc:5697] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de reported cstate change: term changed from 2 to 3, leader changed from 06d54f6275524a6b99d7846015e61326 (127.26.123.2) to 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3). New cstate: current_term: 3 leader_uuid: "7acc97f6aaa944fb892f5f2f050449de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:24.065181  5677 raft_consensus.cc:1275] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 4 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:24.066515  5773 consensus_queue.cc:1048] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:24.077677  5400 raft_consensus.cc:1275] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 4 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:24.079933  5781 consensus_queue.cc:1048] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:24.261106  5400 raft_consensus.cc:1275] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:24.262643  5781 consensus_queue.cc:1048] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.278118  5677 raft_consensus.cc:1275] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Refusing update from remote peer 7acc97f6aaa944fb892f5f2f050449de: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:24.279704  5781 consensus_queue.cc:1048] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.426309 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:24.436542  5475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
mode: GRACEFUL
new_leader_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:47466
I20260709 12:21:24.437119  5475 raft_consensus.cc:606] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Received request to transfer leadership to TS 06d54f6275524a6b99d7846015e61326
I20260709 12:21:24.440665  5475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39"
dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
mode: GRACEFUL
new_leader_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:47466
I20260709 12:21:24.441326  5475 raft_consensus.cc:606] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 3 LEADER]: Received request to transfer leadership to TS 06d54f6275524a6b99d7846015e61326
I20260709 12:21:24.442795 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:24.443434 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:24.444121 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:24.482812  5781 raft_consensus.cc:993] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de: : Instructing follower 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:24.483500  5786 raft_consensus.cc:1081] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 3 LEADER]: Signalling peer 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:24.485088  5400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:57564
I20260709 12:21:24.485612  5400 raft_consensus.cc:493] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:24.485919  5400 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:24.491901  5400 raft_consensus.cc:515] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:24.493551  5400 leader_election.cc:290] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 4 election: Requested vote from peers bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819), 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297)
I20260709 12:21:24.494707  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:24.494626  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aecccb1632bd493dabfe2344f6a7dd39" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:24.495507  5475 raft_consensus.cc:3055] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:24.495584  5677 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:24.495836  5475 raft_consensus.cc:740] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:24.496596  5475 consensus_queue.cc:260] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:24.497581  5475 raft_consensus.cc:3060] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:24.501873  5677 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 4.
I20260709 12:21:24.503106  5367 leader_election.cc:304] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:24.503415  5475 raft_consensus.cc:2468] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 4.
I20260709 12:21:24.503850  5768 raft_consensus.cc:2804] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:24.504982  5768 raft_consensus.cc:697] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 4 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:24.505885  5768 consensus_queue.cc:237] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } }
I20260709 12:21:24.513947  5222 catalog_manager.cc:5697] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 3 to 4, leader changed from 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3) to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 4 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:24.623272  5781 raft_consensus.cc:993] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de: : Instructing follower 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:24.623777  5786 raft_consensus.cc:1081] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Signalling peer 06d54f6275524a6b99d7846015e61326 to start an election
I20260709 12:21:24.625376  5400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f"
dest_uuid: "06d54f6275524a6b99d7846015e61326"
 from {username='slave'} at 127.0.0.1:57564
I20260709 12:21:24.625922  5400 raft_consensus.cc:493] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:24.626223  5400 raft_consensus.cc:3060] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.632436  5400 raft_consensus.cc:515] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:24.634269  5400 leader_election.cc:290] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 3 election: Requested vote from peers 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3:41297), bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819)
I20260709 12:21:24.635200  5475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7acc97f6aaa944fb892f5f2f050449de"
I20260709 12:21:24.635421  5677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "268ee27bb535447db8be5d649000ca9f" candidate_uuid: "06d54f6275524a6b99d7846015e61326" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bc6531ba57ad413a87df2a7cf63367c1"
I20260709 12:21:24.635821  5475 raft_consensus.cc:3055] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:24.636114  5677 raft_consensus.cc:3060] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.636137  5475 raft_consensus.cc:740] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7acc97f6aaa944fb892f5f2f050449de, State: Running, Role: LEADER
I20260709 12:21:24.636937  5475 consensus_queue.cc:260] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:24.638000  5475 raft_consensus.cc:3060] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.642536  5677 raft_consensus.cc:2468] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 3.
I20260709 12:21:24.644171  5367 leader_election.cc:304] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d54f6275524a6b99d7846015e61326, bc6531ba57ad413a87df2a7cf63367c1; no voters: 
I20260709 12:21:24.644886  5475 raft_consensus.cc:2468] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d54f6275524a6b99d7846015e61326 in term 3.
I20260709 12:21:24.645069  5768 raft_consensus.cc:2804] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:24.645773  5768 raft_consensus.cc:697] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 3 LEADER]: Becoming Leader. State: Replica: 06d54f6275524a6b99d7846015e61326, State: Running, Role: LEADER
I20260709 12:21:24.647316  5768 consensus_queue.cc:237] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } }
I20260709 12:21:24.654819  5222 catalog_manager.cc:5697] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 reported cstate change: term changed from 2 to 3, leader changed from 7acc97f6aaa944fb892f5f2f050449de (127.26.123.3) to 06d54f6275524a6b99d7846015e61326 (127.26.123.2). New cstate: current_term: 3 leader_uuid: "06d54f6275524a6b99d7846015e61326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d54f6275524a6b99d7846015e61326" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 44697 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:24.881381  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:25.087877  5678 raft_consensus.cc:1275] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:25.090005  5677 raft_consensus.cc:1275] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 4 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:25.091462  5788 consensus_queue.cc:1048] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:25.091920  5794 consensus_queue.cc:1048] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc6531ba57ad413a87df2a7cf63367c1" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 33819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:25.108434  5475 raft_consensus.cc:1275] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 4 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:25.110972  5801 consensus_queue.cc:1048] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:25.128578  5475 raft_consensus.cc:1275] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Refusing update from remote peer 06d54f6275524a6b99d7846015e61326: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:25.130079  5794 consensus_queue.cc:1048] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7acc97f6aaa944fb892f5f2f050449de" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:25.501433 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:25.534475 27116 tablet_server.cc:179] TabletServer@127.26.123.1:33819 shutting down...
I20260709 12:21:25.567874 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:25.568665 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.569646 27116 raft_consensus.cc:2243] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.570210 27116 raft_consensus.cc:2272] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.572793 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.573560 27116 raft_consensus.cc:2243] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.574052 27116 raft_consensus.cc:2272] T cce83284c7664c1da3f565197bacbe18 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.576422 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.577011 27116 raft_consensus.cc:2243] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.577489 27116 raft_consensus.cc:2272] T fe4161fa651346d69b76af4ca768e397 P bc6531ba57ad413a87df2a7cf63367c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.580078 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.580741 27116 raft_consensus.cc:2243] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.581269 27116 raft_consensus.cc:2272] T aecccb1632bd493dabfe2344f6a7dd39 P bc6531ba57ad413a87df2a7cf63367c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.584766 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.585430 27116 raft_consensus.cc:2243] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.585992 27116 raft_consensus.cc:2272] T 1feedeecfd9247498d0a76ac3d5e7e26 P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.588806 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.589457 27116 raft_consensus.cc:2243] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.590090 27116 raft_consensus.cc:2272] T 268ee27bb535447db8be5d649000ca9f P bc6531ba57ad413a87df2a7cf63367c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.593793 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.594491 27116 raft_consensus.cc:2243] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.594923 27116 raft_consensus.cc:2272] T 7360d5e0f39948b1aaaf652369b525d3 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.597362 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.597972 27116 raft_consensus.cc:2243] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.598415 27116 raft_consensus.cc:2272] T 076abc288e1345c6a56724b2d2c8dfc9 P bc6531ba57ad413a87df2a7cf63367c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.600888 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.601476 27116 raft_consensus.cc:2243] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.602020 27116 raft_consensus.cc:2272] T 73c014a910d24602b825542f7fa781a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.604485 27116 tablet_replica.cc:333] stopping tablet replica
I20260709 12:21:25.605031 27116 raft_consensus.cc:2243] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.605510 27116 raft_consensus.cc:2272] T 7e42c417384f41ef9b5f807b1875d3a1 P bc6531ba57ad413a87df2a7cf63367c1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:25.619712  5367 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 92 similar messages]
W20260709 12:21:25.623040  5367 consensus_peers.cc:597] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:25.639417 27116 tablet_server.cc:196] TabletServer@127.26.123.1:33819 shutdown complete.
I20260709 12:21:25.668645 27116 tablet_server.cc:179] TabletServer@127.26.123.2:0 shutting down...
W20260709 12:21:25.680678  5367 consensus_peers.cc:597] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:25.694226  5367 consensus_peers.cc:597] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:25.695680  5441 consensus_peers.cc:597] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:25.706094 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:25.706951 27116 tablet_replica.cc:333] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
I20260709 12:21:25.708163 27116 raft_consensus.cc:2243] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.708775 27116 raft_consensus.cc:2272] T fe4161fa651346d69b76af4ca768e397 P 06d54f6275524a6b99d7846015e61326 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.711464 27116 tablet_replica.cc:333] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
I20260709 12:21:25.712106 27116 raft_consensus.cc:2243] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.712934 27116 raft_consensus.cc:2272] T aecccb1632bd493dabfe2344f6a7dd39 P 06d54f6275524a6b99d7846015e61326 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.715862 27116 tablet_replica.cc:333] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
I20260709 12:21:25.716471 27116 raft_consensus.cc:2243] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.717342 27116 raft_consensus.cc:2272] T 73c014a910d24602b825542f7fa781a1 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.719696 27116 tablet_replica.cc:333] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
I20260709 12:21:25.720261 27116 raft_consensus.cc:2243] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.720698 27116 raft_consensus.cc:2272] T cce83284c7664c1da3f565197bacbe18 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.722986 27116 tablet_replica.cc:333] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
I20260709 12:21:25.723539 27116 raft_consensus.cc:2243] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.724056 27116 raft_consensus.cc:2272] T 7360d5e0f39948b1aaaf652369b525d3 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.726298 27116 tablet_replica.cc:333] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
I20260709 12:21:25.726854 27116 raft_consensus.cc:2243] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.727285 27116 raft_consensus.cc:2272] T 076abc288e1345c6a56724b2d2c8dfc9 P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.729885 27116 tablet_replica.cc:333] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
I20260709 12:21:25.730396 27116 raft_consensus.cc:2243] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.731189 27116 raft_consensus.cc:2272] T 7e42c417384f41ef9b5f807b1875d3a1 P 06d54f6275524a6b99d7846015e61326 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.733836 27116 tablet_replica.cc:333] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
I20260709 12:21:25.734941 27116 raft_consensus.cc:2243] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.736024 27116 raft_consensus.cc:2272] T 1feedeecfd9247498d0a76ac3d5e7e26 P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.738842 27116 tablet_replica.cc:333] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
I20260709 12:21:25.739626 27116 raft_consensus.cc:2243] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.740648 27116 raft_consensus.cc:2272] T 268ee27bb535447db8be5d649000ca9f P 06d54f6275524a6b99d7846015e61326 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.744108 27116 tablet_replica.cc:333] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326: stopping tablet replica
W20260709 12:21:25.744581  5367 consensus_peers.cc:597] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 -> Peer bc6531ba57ad413a87df2a7cf63367c1 (127.26.123.1:33819): Couldn't send request to peer bc6531ba57ad413a87df2a7cf63367c1. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:33819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:25.745203 27116 raft_consensus.cc:2243] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.746990 27116 raft_consensus.cc:2272] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 06d54f6275524a6b99d7846015e61326 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:25.749351  5442 consensus_peers.cc:597] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de -> Peer 06d54f6275524a6b99d7846015e61326 (127.26.123.2:44697): Couldn't send request to peer 06d54f6275524a6b99d7846015e61326. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:25.773761 27116 tablet_server.cc:196] TabletServer@127.26.123.2:0 shutdown complete.
I20260709 12:21:25.803486 27116 tablet_server.cc:179] TabletServer@127.26.123.3:0 shutting down...
I20260709 12:21:25.828882 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:25.829614 27116 tablet_replica.cc:333] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.830410 27116 raft_consensus.cc:2243] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.830915 27116 raft_consensus.cc:2272] T aecccb1632bd493dabfe2344f6a7dd39 P 7acc97f6aaa944fb892f5f2f050449de [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.834398 27116 tablet_replica.cc:333] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.835057 27116 raft_consensus.cc:2243] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.835938 27116 raft_consensus.cc:2272] T fe4161fa651346d69b76af4ca768e397 P 7acc97f6aaa944fb892f5f2f050449de [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.838404 27116 tablet_replica.cc:333] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.839025 27116 raft_consensus.cc:2243] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.839560 27116 raft_consensus.cc:2272] T 73c014a910d24602b825542f7fa781a1 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.842180 27116 tablet_replica.cc:333] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.842808 27116 raft_consensus.cc:2243] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.843263 27116 raft_consensus.cc:2272] T 5ce5ea9c5a1e40149b4ee83a962ec4bf P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.846210 27116 tablet_replica.cc:333] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.846843 27116 raft_consensus.cc:2243] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.847600 27116 raft_consensus.cc:2272] T cce83284c7664c1da3f565197bacbe18 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.850600 27116 tablet_replica.cc:333] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.851173 27116 raft_consensus.cc:2243] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.851585 27116 raft_consensus.cc:2272] T 7e42c417384f41ef9b5f807b1875d3a1 P 7acc97f6aaa944fb892f5f2f050449de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.853837 27116 tablet_replica.cc:333] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.854372 27116 raft_consensus.cc:2243] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.854856 27116 raft_consensus.cc:2272] T 1feedeecfd9247498d0a76ac3d5e7e26 P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.857038 27116 tablet_replica.cc:333] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.857630 27116 raft_consensus.cc:2243] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:25.858065 27116 raft_consensus.cc:2272] T 268ee27bb535447db8be5d649000ca9f P 7acc97f6aaa944fb892f5f2f050449de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.861312 27116 tablet_replica.cc:333] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.861976 27116 raft_consensus.cc:2243] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.862974 27116 raft_consensus.cc:2272] T 076abc288e1345c6a56724b2d2c8dfc9 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.865528 27116 tablet_replica.cc:333] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de: stopping tablet replica
I20260709 12:21:25.866044 27116 raft_consensus.cc:2243] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.867002 27116 raft_consensus.cc:2272] T 7360d5e0f39948b1aaaf652369b525d3 P 7acc97f6aaa944fb892f5f2f050449de [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:25.882597  5273 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:21:25.922365 27116 tablet_server.cc:196] TabletServer@127.26.123.3:0 shutdown complete.
I20260709 12:21:25.951992 27116 master.cc:562] Master@127.26.123.62:40191 shutting down...
W20260709 12:21:25.964478  5256 debug-util.cc:398] Leaking SignalData structure 0x7b0800271ee0 after lost signal to thread 27119
W20260709 12:21:25.966090  5256 debug-util.cc:398] Leaking SignalData structure 0x7b080003dec0 after lost signal to thread 5259
I20260709 12:21:25.980306 27116 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:25.981208 27116 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:25.981689 27116 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7f01cd74e1714f289929e6739cbef069: stopping tablet replica
W20260709 12:21:27.002020 27116 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 5256)
W20260709 12:21:28.002571 27116 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 5256)
I20260709 12:21:28.562233 27116 master.cc:584] Master@127.26.123.62:40191 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (15891 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:21:28.612361 27116 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.123.62:41417
I20260709 12:21:28.613567 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:28.619462  5813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.620287  5814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.620957  5816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:28.621521 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:28.622684 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:28.622874 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:28.623046 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599688623028 us; error 0 us; skew 500 ppm
I20260709 12:21:28.623657 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:28.626210 27116 webserver.cc:533] Webserver started at http://127.26.123.62:33857/ using document root <none> and password file <none>
I20260709 12:21:28.626770 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:28.626960 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:28.627238 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:28.628458 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/master-0-root/instance:
uuid: "f551b9d3330a42fd9561b58021624508"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-9q7n"
I20260709 12:21:28.633178 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:28.637269  5821 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.638120 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:28.638422 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/master-0-root
uuid: "f551b9d3330a42fd9561b58021624508"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-9q7n"
I20260709 12:21:28.638732 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:28.658001 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:28.659363 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:28.702981 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.62:41417
I20260709 12:21:28.703094  5872 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.62:41417 every 8 connection(s)
I20260709 12:21:28.707370  5873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:28.721911  5873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508: Bootstrap starting.
I20260709 12:21:28.727690  5873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:28.732688  5873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508: No bootstrap required, opened a new log
I20260709 12:21:28.735348  5873 raft_consensus.cc:359] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f551b9d3330a42fd9561b58021624508" member_type: VOTER }
I20260709 12:21:28.735795  5873 raft_consensus.cc:385] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:28.736042  5873 raft_consensus.cc:740] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f551b9d3330a42fd9561b58021624508, State: Initialized, Role: FOLLOWER
I20260709 12:21:28.736658  5873 consensus_queue.cc:260] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f551b9d3330a42fd9561b58021624508" member_type: VOTER }
I20260709 12:21:28.737159  5873 raft_consensus.cc:399] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:28.737389  5873 raft_consensus.cc:493] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:28.737640  5873 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:28.744078  5873 raft_consensus.cc:515] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f551b9d3330a42fd9561b58021624508" member_type: VOTER }
I20260709 12:21:28.744663  5873 leader_election.cc:304] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f551b9d3330a42fd9561b58021624508; no voters: 
I20260709 12:21:28.745944  5873 leader_election.cc:290] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:28.746326  5876 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:28.747812  5876 raft_consensus.cc:697] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 1 LEADER]: Becoming Leader. State: Replica: f551b9d3330a42fd9561b58021624508, State: Running, Role: LEADER
I20260709 12:21:28.748550  5876 consensus_queue.cc:237] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f551b9d3330a42fd9561b58021624508" member_type: VOTER }
I20260709 12:21:28.749037  5873 sys_catalog.cc:565] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:28.753517  5878 sys_catalog.cc:455] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f551b9d3330a42fd9561b58021624508. Latest consensus state: current_term: 1 leader_uuid: "f551b9d3330a42fd9561b58021624508" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f551b9d3330a42fd9561b58021624508" member_type: VOTER } }
I20260709 12:21:28.754410  5878 sys_catalog.cc:458] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:28.755743  5877 sys_catalog.cc:455] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f551b9d3330a42fd9561b58021624508" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f551b9d3330a42fd9561b58021624508" member_type: VOTER } }
I20260709 12:21:28.756472  5877 sys_catalog.cc:458] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:28.756510  5883 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:28.762116  5883 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:28.767726 27116 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:28.772073  5883 catalog_manager.cc:1383] Generated new cluster ID: eb220c66c2f546a4b3f9e349fbbc17ac
I20260709 12:21:28.772367  5883 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:28.791724  5883 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:28.793208  5883 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:28.811132  5883 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508: Generated new TSK 0
I20260709 12:21:28.811792  5883 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:28.835736 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:28.842708  5894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.843808  5895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.845052  5897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:28.845432 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:28.846612 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:28.846827 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:28.846998 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599688846978 us; error 0 us; skew 500 ppm
I20260709 12:21:28.847580 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:28.850149 27116 webserver.cc:533] Webserver started at http://127.26.123.1:35255/ using document root <none> and password file <none>
I20260709 12:21:28.850780 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:28.851207 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:28.851614 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:28.852906 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-0-root/instance:
uuid: "66ea6dd222b444f984b3e44d54ebba65"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-9q7n"
I20260709 12:21:28.858762 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.002s
I20260709 12:21:28.862720  5902 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.863549 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 12:21:28.863857 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "66ea6dd222b444f984b3e44d54ebba65"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-9q7n"
I20260709 12:21:28.864172 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:28.881536 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:28.883004 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:28.884598 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:28.887130 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:28.887352 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.887671 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:28.887849 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.935616 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:39999
I20260709 12:21:28.935730  5964 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:39999 every 8 connection(s)
I20260709 12:21:28.941319 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:28.949237  5969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.950699  5970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:28.954034  5972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:28.954828 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:28.956277  5965 heartbeater.cc:344] Connected to a master server at 127.26.123.62:41417
I20260709 12:21:28.956281 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:28.956875 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:28.956964  5965 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:28.957139 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599688957113 us; error 0 us; skew 500 ppm
I20260709 12:21:28.958122 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:28.958114  5965 heartbeater.cc:507] Master 127.26.123.62:41417 requested a full tablet report, sending...
I20260709 12:21:28.960904  5838 ts_manager.cc:194] Registered new tserver with Master: 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999)
I20260709 12:21:28.961493 27116 webserver.cc:533] Webserver started at http://127.26.123.2:40993/ using document root <none> and password file <none>
I20260709 12:21:28.962162 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:28.962363 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:28.962626 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:28.963187  5838 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34384
I20260709 12:21:28.963925 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-1-root/instance:
uuid: "8d2b1baf654845e4b25f67cb7a66a930"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-9q7n"
I20260709 12:21:28.969623 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 12:21:28.973459  5977 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:28.974365 27116 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:21:28.974705 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-1-root
uuid: "8d2b1baf654845e4b25f67cb7a66a930"
format_stamp: "Formatted at 2026-07-09 12:21:28 on dist-test-slave-9q7n"
I20260709 12:21:28.974997 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:28.994458 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:28.995791 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:28.997407 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:28.999845 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:29.000041 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:29.000331 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:29.000481 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:29.046627 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.2:43061
I20260709 12:21:29.046746  6039 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.2:43061 every 8 connection(s)
I20260709 12:21:29.052306 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:29.060266  6043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:29.062570  6044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:29.066818  6040 heartbeater.cc:344] Connected to a master server at 127.26.123.62:41417
I20260709 12:21:29.067462  6040 heartbeater.cc:461] Registering TS with master...
W20260709 12:21:29.067914  6046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:29.068413 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:29.068437  6040 heartbeater.cc:507] Master 127.26.123.62:41417 requested a full tablet report, sending...
I20260709 12:21:29.069800 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:29.070019 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:29.070209 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599689070187 us; error 0 us; skew 500 ppm
I20260709 12:21:29.070845 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:29.071178  5838 ts_manager.cc:194] Registered new tserver with Master: 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2:43061)
I20260709 12:21:29.073078  5838 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34396
I20260709 12:21:29.073573 27116 webserver.cc:533] Webserver started at http://127.26.123.3:43263/ using document root <none> and password file <none>
I20260709 12:21:29.074236 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:29.074451 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:29.074791 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:29.076510 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-2-root/instance:
uuid: "96ce369c77f44947996bbdc2f48b1f7b"
format_stamp: "Formatted at 2026-07-09 12:21:29 on dist-test-slave-9q7n"
I20260709 12:21:29.081995 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:21:29.086167  6051 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:29.087051 27116 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:21:29.087366 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-2-root
uuid: "96ce369c77f44947996bbdc2f48b1f7b"
format_stamp: "Formatted at 2026-07-09 12:21:29 on dist-test-slave-9q7n"
I20260709 12:21:29.087641 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599329322122-27116-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:29.125231 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:29.126627 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:29.128232 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:29.130970 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:29.131204 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:29.131490 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:29.131652 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:29.180464 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.3:37633
I20260709 12:21:29.180586  6113 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.3:37633 every 8 connection(s)
I20260709 12:21:29.196290  6114 heartbeater.cc:344] Connected to a master server at 127.26.123.62:41417
I20260709 12:21:29.196798  6114 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:29.197651  6114 heartbeater.cc:507] Master 127.26.123.62:41417 requested a full tablet report, sending...
I20260709 12:21:29.200004  5838 ts_manager.cc:194] Registered new tserver with Master: 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:29.200735 27116 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016603166s
I20260709 12:21:29.202055  5838 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34410
I20260709 12:21:29.231575  5838 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34420:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:29.234043  5838 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:29.285259  6005 tablet_service.cc:1511] Processing CreateTablet for tablet f5a3e81eb67043339cedf12add318a38 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=210ab5d1b48847bd9bc5f4020bb0ad1d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:29.285456  6004 tablet_service.cc:1511] Processing CreateTablet for tablet 3bb68b3efbd74f84bd15e8747a28a799 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=210ab5d1b48847bd9bc5f4020bb0ad1d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:29.287070  6005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a3e81eb67043339cedf12add318a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.288933  6004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bb68b3efbd74f84bd15e8747a28a799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.296550  5930 tablet_service.cc:1511] Processing CreateTablet for tablet f5a3e81eb67043339cedf12add318a38 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=210ab5d1b48847bd9bc5f4020bb0ad1d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:29.297500  5929 tablet_service.cc:1511] Processing CreateTablet for tablet 3bb68b3efbd74f84bd15e8747a28a799 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=210ab5d1b48847bd9bc5f4020bb0ad1d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:29.298471  5930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a3e81eb67043339cedf12add318a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.298311  6079 tablet_service.cc:1511] Processing CreateTablet for tablet f5a3e81eb67043339cedf12add318a38 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=210ab5d1b48847bd9bc5f4020bb0ad1d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:29.299249  5929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bb68b3efbd74f84bd15e8747a28a799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.299787  6079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a3e81eb67043339cedf12add318a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.302402  6078 tablet_service.cc:1511] Processing CreateTablet for tablet 3bb68b3efbd74f84bd15e8747a28a799 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=210ab5d1b48847bd9bc5f4020bb0ad1d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:29.303987  6078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bb68b3efbd74f84bd15e8747a28a799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:29.343703  6129 tablet_bootstrap.cc:492] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930: Bootstrap starting.
I20260709 12:21:29.350770  6130 tablet_bootstrap.cc:492] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65: Bootstrap starting.
I20260709 12:21:29.353785  6129 tablet_bootstrap.cc:654] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.359434  6130 tablet_bootstrap.cc:654] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.360785  6131 tablet_bootstrap.cc:492] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b: Bootstrap starting.
I20260709 12:21:29.361634  6129 tablet_bootstrap.cc:492] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930: No bootstrap required, opened a new log
I20260709 12:21:29.362378  6129 ts_tablet_manager.cc:1403] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 12:21:29.366432  6129 raft_consensus.cc:359] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.367414  6129 raft_consensus.cc:385] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.367762  6129 raft_consensus.cc:740] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d2b1baf654845e4b25f67cb7a66a930, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.368932  6131 tablet_bootstrap.cc:654] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.368686  6129 consensus_queue.cc:260] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.370725  6130 tablet_bootstrap.cc:492] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65: No bootstrap required, opened a new log
I20260709 12:21:29.371411  6130 ts_tablet_manager.cc:1403] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.015s
I20260709 12:21:29.371721  6040 heartbeater.cc:499] Master 127.26.123.62:41417 was elected leader, sending a full tablet report...
I20260709 12:21:29.374716  6129 ts_tablet_manager.cc:1434] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:21:29.374851  6130 raft_consensus.cc:359] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.375785  6130 raft_consensus.cc:385] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.375999  6129 tablet_bootstrap.cc:492] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930: Bootstrap starting.
I20260709 12:21:29.376135  6130 raft_consensus.cc:740] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ea6dd222b444f984b3e44d54ebba65, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.377564  6131 tablet_bootstrap.cc:492] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b: No bootstrap required, opened a new log
I20260709 12:21:29.377378  6130 consensus_queue.cc:260] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.378360  6131 ts_tablet_manager.cc:1403] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 12:21:29.383436  6131 raft_consensus.cc:359] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.384449  6131 raft_consensus.cc:385] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.384848  6131 raft_consensus.cc:740] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ce369c77f44947996bbdc2f48b1f7b, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.385428  6129 tablet_bootstrap.cc:654] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.385780  6131 consensus_queue.cc:260] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.387092  5965 heartbeater.cc:499] Master 127.26.123.62:41417 was elected leader, sending a full tablet report...
I20260709 12:21:29.387449  6134 raft_consensus.cc:493] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:29.388044  6134 raft_consensus.cc:515] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.388801  6130 ts_tablet_manager.cc:1434] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65: Time spent starting tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 12:21:29.390331  6130 tablet_bootstrap.cc:492] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65: Bootstrap starting.
I20260709 12:21:29.395557  6134 leader_election.cc:290] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999), 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:29.405017  6130 tablet_bootstrap.cc:654] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.416591  6114 heartbeater.cc:499] Master 127.26.123.62:41417 was elected leader, sending a full tablet report...
I20260709 12:21:29.417616  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ea6dd222b444f984b3e44d54ebba65" is_pre_election: true
I20260709 12:21:29.418503  6129 tablet_bootstrap.cc:492] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930: No bootstrap required, opened a new log
I20260709 12:21:29.419153  6129 ts_tablet_manager.cc:1403] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.012s
W20260709 12:21:29.419972  5981 leader_election.cc:343] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:29.422768  6129 raft_consensus.cc:359] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.423740  6129 raft_consensus.cc:385] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.423770  6130 tablet_bootstrap.cc:492] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65: No bootstrap required, opened a new log
I20260709 12:21:29.424180  6129 raft_consensus.cc:740] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d2b1baf654845e4b25f67cb7a66a930, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.424670  6130 ts_tablet_manager.cc:1403] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.009s
I20260709 12:21:29.425078  6129 consensus_queue.cc:260] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.427551  6129 ts_tablet_manager.cc:1434] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 12:21:29.425971  6131 ts_tablet_manager.cc:1434] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.007s
I20260709 12:21:29.428696  6130 raft_consensus.cc:359] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.429612  6130 raft_consensus.cc:385] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.429535  6089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b" is_pre_election: true
I20260709 12:21:29.430125  6130 raft_consensus.cc:740] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ea6dd222b444f984b3e44d54ebba65, State: Initialized, Role: FOLLOWER
W20260709 12:21:29.431478  5981 leader_election.cc:343] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:29.431170  6130 consensus_queue.cc:260] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.432021  5981 leader_election.cc:304] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d2b1baf654845e4b25f67cb7a66a930; no voters: 66ea6dd222b444f984b3e44d54ebba65, 96ce369c77f44947996bbdc2f48b1f7b
I20260709 12:21:29.432919  6134 raft_consensus.cc:2749] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:29.435592  6130 ts_tablet_manager.cc:1434] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 12:21:29.436034  6131 tablet_bootstrap.cc:492] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b: Bootstrap starting.
I20260709 12:21:29.444286  6131 tablet_bootstrap.cc:654] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:29.450069  6131 tablet_bootstrap.cc:492] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b: No bootstrap required, opened a new log
I20260709 12:21:29.450583  6131 ts_tablet_manager.cc:1403] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:21:29.453222  6131 raft_consensus.cc:359] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.453838  6131 raft_consensus.cc:385] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:29.454082  6131 raft_consensus.cc:740] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ce369c77f44947996bbdc2f48b1f7b, State: Initialized, Role: FOLLOWER
I20260709 12:21:29.454800  6131 consensus_queue.cc:260] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.456899  6131 ts_tablet_manager.cc:1434] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:29.531198  6134 raft_consensus.cc:493] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:29.531939  6134 raft_consensus.cc:515] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.534230  6134 leader_election.cc:290] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999), 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:29.535162  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb68b3efbd74f84bd15e8747a28a799" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ea6dd222b444f984b3e44d54ebba65" is_pre_election: true
I20260709 12:21:29.535490  6089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb68b3efbd74f84bd15e8747a28a799" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b" is_pre_election: true
I20260709 12:21:29.535939  5940 raft_consensus.cc:2468] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d2b1baf654845e4b25f67cb7a66a930 in term 0.
I20260709 12:21:29.536160  6089 raft_consensus.cc:2468] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d2b1baf654845e4b25f67cb7a66a930 in term 0.
I20260709 12:21:29.537058  5981 leader_election.cc:304] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ea6dd222b444f984b3e44d54ebba65, 8d2b1baf654845e4b25f67cb7a66a930; no voters: 
I20260709 12:21:29.537813  6134 raft_consensus.cc:2804] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:29.538122  6134 raft_consensus.cc:493] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:29.538391  6134 raft_consensus.cc:3060] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:29.545888  6134 raft_consensus.cc:515] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.547801  6134 leader_election.cc:290] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 election: Requested vote from peers 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999), 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:29.549263  6089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb68b3efbd74f84bd15e8747a28a799" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b"
I20260709 12:21:29.549225  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bb68b3efbd74f84bd15e8747a28a799" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ea6dd222b444f984b3e44d54ebba65"
I20260709 12:21:29.550261  6089 raft_consensus.cc:3060] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:29.550323  5940 raft_consensus.cc:3060] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:29.558568  5940 raft_consensus.cc:2468] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d2b1baf654845e4b25f67cb7a66a930 in term 1.
I20260709 12:21:29.558571  6089 raft_consensus.cc:2468] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d2b1baf654845e4b25f67cb7a66a930 in term 1.
I20260709 12:21:29.560181  5981 leader_election.cc:304] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ea6dd222b444f984b3e44d54ebba65, 8d2b1baf654845e4b25f67cb7a66a930; no voters: 
I20260709 12:21:29.560971  6134 raft_consensus.cc:2804] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:29.562219  6134 raft_consensus.cc:697] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 LEADER]: Becoming Leader. State: Replica: 8d2b1baf654845e4b25f67cb7a66a930, State: Running, Role: LEADER
I20260709 12:21:29.563140  6134 consensus_queue.cc:237] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.571476  5837 catalog_manager.cc:5697] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "8d2b1baf654845e4b25f67cb7a66a930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:29.761854  5886 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:29.877808  6139 raft_consensus.cc:493] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:29.878451  6139 raft_consensus.cc:515] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.880694  6139 leader_election.cc:290] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2:43061), 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999)
I20260709 12:21:29.893249  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38" candidate_uuid: "96ce369c77f44947996bbdc2f48b1f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ea6dd222b444f984b3e44d54ebba65" is_pre_election: true
I20260709 12:21:29.893360  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38" candidate_uuid: "96ce369c77f44947996bbdc2f48b1f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d2b1baf654845e4b25f67cb7a66a930" is_pre_election: true
I20260709 12:21:29.894172  5940 raft_consensus.cc:2468] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96ce369c77f44947996bbdc2f48b1f7b in term 0.
I20260709 12:21:29.894254  6015 raft_consensus.cc:2468] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96ce369c77f44947996bbdc2f48b1f7b in term 0.
I20260709 12:21:29.895455  6055 leader_election.cc:304] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ea6dd222b444f984b3e44d54ebba65, 96ce369c77f44947996bbdc2f48b1f7b; no voters: 
I20260709 12:21:29.896250  6139 raft_consensus.cc:2804] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:29.896559  6139 raft_consensus.cc:493] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:29.896838  6139 raft_consensus.cc:3060] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:29.904663  6139 raft_consensus.cc:515] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.906623  6139 leader_election.cc:290] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [CANDIDATE]: Term 1 election: Requested vote from peers 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2:43061), 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999)
I20260709 12:21:29.907560  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38" candidate_uuid: "96ce369c77f44947996bbdc2f48b1f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d2b1baf654845e4b25f67cb7a66a930"
I20260709 12:21:29.907747  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38" candidate_uuid: "96ce369c77f44947996bbdc2f48b1f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ea6dd222b444f984b3e44d54ebba65"
I20260709 12:21:29.908195  6015 raft_consensus.cc:3060] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:29.908325  5940 raft_consensus.cc:3060] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:29.915189  6015 raft_consensus.cc:2468] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ce369c77f44947996bbdc2f48b1f7b in term 1.
I20260709 12:21:29.915176  5940 raft_consensus.cc:2468] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ce369c77f44947996bbdc2f48b1f7b in term 1.
I20260709 12:21:29.916846  6055 leader_election.cc:304] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ea6dd222b444f984b3e44d54ebba65, 96ce369c77f44947996bbdc2f48b1f7b; no voters: 
I20260709 12:21:29.917865  6139 raft_consensus.cc:2804] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:29.918920  6139 raft_consensus.cc:697] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 LEADER]: Becoming Leader. State: Replica: 96ce369c77f44947996bbdc2f48b1f7b, State: Running, Role: LEADER
I20260709 12:21:29.919770  6139 consensus_queue.cc:237] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:29.927487  5837 catalog_manager.cc:5697] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3). New cstate: current_term: 1 leader_uuid: "96ce369c77f44947996bbdc2f48b1f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:30.006640  6134 consensus_queue.cc:1048] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:30.019454  6142 consensus_queue.cc:1048] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:30.057390  5838 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34428:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:30.059705  5838 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:30.086413  5929 tablet_service.cc:1511] Processing CreateTablet for tablet b2d1d081affe461e97b19e2106f69bae (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ec649460e4db41aabcfd09eb54bbfefb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:30.087867  5929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d1d081affe461e97b19e2106f69bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.088016  6004 tablet_service.cc:1511] Processing CreateTablet for tablet b2d1d081affe461e97b19e2106f69bae (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ec649460e4db41aabcfd09eb54bbfefb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:30.089574  6004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d1d081affe461e97b19e2106f69bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.090601  6078 tablet_service.cc:1511] Processing CreateTablet for tablet b2d1d081affe461e97b19e2106f69bae (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ec649460e4db41aabcfd09eb54bbfefb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:21:30.091988  5930 tablet_service.cc:1511] Processing CreateTablet for tablet a567ec217d484cc99b9739c3f89531b3 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ec649460e4db41aabcfd09eb54bbfefb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:30.092159  6078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d1d081affe461e97b19e2106f69bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.093434  5930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a567ec217d484cc99b9739c3f89531b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.094617  6005 tablet_service.cc:1511] Processing CreateTablet for tablet a567ec217d484cc99b9739c3f89531b3 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ec649460e4db41aabcfd09eb54bbfefb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:30.096062  6005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a567ec217d484cc99b9739c3f89531b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.097091  6079 tablet_service.cc:1511] Processing CreateTablet for tablet a567ec217d484cc99b9739c3f89531b3 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=ec649460e4db41aabcfd09eb54bbfefb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:21:30.098564  6079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a567ec217d484cc99b9739c3f89531b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:30.123095  6163 tablet_bootstrap.cc:492] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65: Bootstrap starting.
I20260709 12:21:30.128376  6162 tablet_bootstrap.cc:492] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b: Bootstrap starting.
I20260709 12:21:30.134919  6163 tablet_bootstrap.cc:654] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.146214  6164 tablet_bootstrap.cc:492] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930: Bootstrap starting.
I20260709 12:21:30.148485  6162 tablet_bootstrap.cc:654] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.152035  6163 tablet_bootstrap.cc:492] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65: No bootstrap required, opened a new log
I20260709 12:21:30.152665  6163 ts_tablet_manager.cc:1403] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.009s
I20260709 12:21:30.156275  6164 tablet_bootstrap.cc:654] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.156451  6162 tablet_bootstrap.cc:492] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b: No bootstrap required, opened a new log
I20260709 12:21:30.157148  6162 ts_tablet_manager.cc:1403] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.011s
I20260709 12:21:30.156672  6163 raft_consensus.cc:359] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.157603  6163 raft_consensus.cc:385] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.158013  6163 raft_consensus.cc:740] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ea6dd222b444f984b3e44d54ebba65, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.158951  6163 consensus_queue.cc:260] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.161357  6162 raft_consensus.cc:359] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.162559  6162 raft_consensus.cc:385] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.162992  6163 ts_tablet_manager.cc:1434] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:21:30.162995  6162 raft_consensus.cc:740] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ce369c77f44947996bbdc2f48b1f7b, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.164317  6164 tablet_bootstrap.cc:492] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930: No bootstrap required, opened a new log
I20260709 12:21:30.164695  6163 tablet_bootstrap.cc:492] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65: Bootstrap starting.
I20260709 12:21:30.164424  6162 consensus_queue.cc:260] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.165120  6164 ts_tablet_manager.cc:1403] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:21:30.168802  6164 raft_consensus.cc:359] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.169642  6164 raft_consensus.cc:385] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.169761  6162 ts_tablet_manager.cc:1434] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 12:21:30.170046  6164 raft_consensus.cc:740] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d2b1baf654845e4b25f67cb7a66a930, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.170454  6153 raft_consensus.cc:493] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.171037  6153 raft_consensus.cc:515] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.171175  6164 consensus_queue.cc:260] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.172914  6163 tablet_bootstrap.cc:654] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.173861  6164 ts_tablet_manager.cc:1434] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:21:30.175238  6164 tablet_bootstrap.cc:492] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930: Bootstrap starting.
I20260709 12:21:30.171243  6162 tablet_bootstrap.cc:492] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b: Bootstrap starting.
I20260709 12:21:30.184291  6162 tablet_bootstrap.cc:654] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.184921  6153 leader_election.cc:290] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2:43061), 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:30.190117  6164 tablet_bootstrap.cc:654] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:30.198843  6162 tablet_bootstrap.cc:492] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b: No bootstrap required, opened a new log
I20260709 12:21:30.199514  6162 ts_tablet_manager.cc:1403] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.003s
I20260709 12:21:30.203027  6162 raft_consensus.cc:359] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.203833  6162 raft_consensus.cc:385] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.204144  6162 raft_consensus.cc:740] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ce369c77f44947996bbdc2f48b1f7b, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.205091  6162 consensus_queue.cc:260] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.213267  6162 ts_tablet_manager.cc:1434] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:21:30.216874  6164 tablet_bootstrap.cc:492] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930: No bootstrap required, opened a new log
I20260709 12:21:30.217595  6164 ts_tablet_manager.cc:1403] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.017s
I20260709 12:21:30.221062  6164 raft_consensus.cc:359] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.221889  6164 raft_consensus.cc:385] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.222223  6164 raft_consensus.cc:740] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d2b1baf654845e4b25f67cb7a66a930, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.223165  6164 consensus_queue.cc:260] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.227111  6163 tablet_bootstrap.cc:492] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65: No bootstrap required, opened a new log
I20260709 12:21:30.227967  6163 ts_tablet_manager.cc:1403] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65: Time spent bootstrapping tablet: real 0.064s	user 0.021s	sys 0.041s
I20260709 12:21:30.229056  6164 ts_tablet_manager.cc:1434] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.005s
I20260709 12:21:30.232170  6163 raft_consensus.cc:359] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.233317  6163 raft_consensus.cc:385] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:30.233678  6163 raft_consensus.cc:740] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ea6dd222b444f984b3e44d54ebba65, State: Initialized, Role: FOLLOWER
I20260709 12:21:30.234666  6163 consensus_queue.cc:260] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.237454  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a567ec217d484cc99b9739c3f89531b3" candidate_uuid: "66ea6dd222b444f984b3e44d54ebba65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d2b1baf654845e4b25f67cb7a66a930" is_pre_election: true
I20260709 12:21:30.238427  6015 raft_consensus.cc:2468] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ea6dd222b444f984b3e44d54ebba65 in term 0.
I20260709 12:21:30.237587  6163 ts_tablet_manager.cc:1434] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:21:30.240252  5906 leader_election.cc:304] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ea6dd222b444f984b3e44d54ebba65, 8d2b1baf654845e4b25f67cb7a66a930; no voters: 
I20260709 12:21:30.241261  6153 raft_consensus.cc:2804] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.241688  6153 raft_consensus.cc:493] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.242081  6153 raft_consensus.cc:3060] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.250535  6089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a567ec217d484cc99b9739c3f89531b3" candidate_uuid: "66ea6dd222b444f984b3e44d54ebba65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b" is_pre_election: true
I20260709 12:21:30.251283  6089 raft_consensus.cc:2468] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ea6dd222b444f984b3e44d54ebba65 in term 0.
I20260709 12:21:30.251577  6153 raft_consensus.cc:515] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.253630  6153 leader_election.cc:290] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [CANDIDATE]: Term 1 election: Requested vote from peers 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2:43061), 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:30.254657  6015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a567ec217d484cc99b9739c3f89531b3" candidate_uuid: "66ea6dd222b444f984b3e44d54ebba65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d2b1baf654845e4b25f67cb7a66a930"
I20260709 12:21:30.254968  6089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a567ec217d484cc99b9739c3f89531b3" candidate_uuid: "66ea6dd222b444f984b3e44d54ebba65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b"
I20260709 12:21:30.255293  6015 raft_consensus.cc:3060] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.255578  6089 raft_consensus.cc:3060] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.262602  6015 raft_consensus.cc:2468] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ea6dd222b444f984b3e44d54ebba65 in term 1.
I20260709 12:21:30.262748  6089 raft_consensus.cc:2468] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ea6dd222b444f984b3e44d54ebba65 in term 1.
I20260709 12:21:30.263860  5906 leader_election.cc:304] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ea6dd222b444f984b3e44d54ebba65, 8d2b1baf654845e4b25f67cb7a66a930; no voters: 
I20260709 12:21:30.264632  6153 raft_consensus.cc:2804] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:30.265815  6153 raft_consensus.cc:697] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 LEADER]: Becoming Leader. State: Replica: 66ea6dd222b444f984b3e44d54ebba65, State: Running, Role: LEADER
I20260709 12:21:30.266499  6142 raft_consensus.cc:493] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:30.266675  6153 consensus_queue.cc:237] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.267140  6142 raft_consensus.cc:515] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.269097  6142 leader_election.cc:290] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999), 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:30.270016  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ea6dd222b444f984b3e44d54ebba65" is_pre_election: true
I20260709 12:21:30.270296  6089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b" is_pre_election: true
I20260709 12:21:30.270884  5940 raft_consensus.cc:2468] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d2b1baf654845e4b25f67cb7a66a930 in term 0.
I20260709 12:21:30.271193  6089 raft_consensus.cc:2468] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d2b1baf654845e4b25f67cb7a66a930 in term 0.
I20260709 12:21:30.272562  5981 leader_election.cc:304] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d2b1baf654845e4b25f67cb7a66a930, 96ce369c77f44947996bbdc2f48b1f7b; no voters: 
I20260709 12:21:30.273414  6142 raft_consensus.cc:2804] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:30.273787  6142 raft_consensus.cc:493] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:30.274134  6142 raft_consensus.cc:3060] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.275908  5836 catalog_manager.cc:5697] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "66ea6dd222b444f984b3e44d54ebba65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:30.282145  6142 raft_consensus.cc:515] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.284132  6142 leader_election.cc:290] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 election: Requested vote from peers 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999), 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:30.285223  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ea6dd222b444f984b3e44d54ebba65"
I20260709 12:21:30.285418  6089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae" candidate_uuid: "8d2b1baf654845e4b25f67cb7a66a930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b"
I20260709 12:21:30.286070  5940 raft_consensus.cc:3060] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.286178  6089 raft_consensus.cc:3060] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:30.294966  6089 raft_consensus.cc:2468] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d2b1baf654845e4b25f67cb7a66a930 in term 1.
I20260709 12:21:30.294970  5940 raft_consensus.cc:2468] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d2b1baf654845e4b25f67cb7a66a930 in term 1.
I20260709 12:21:30.296345  5981 leader_election.cc:304] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ea6dd222b444f984b3e44d54ebba65, 8d2b1baf654845e4b25f67cb7a66a930; no voters: 
I20260709 12:21:30.297125  6142 raft_consensus.cc:2804] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:30.297660  6142 raft_consensus.cc:697] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 LEADER]: Becoming Leader. State: Replica: 8d2b1baf654845e4b25f67cb7a66a930, State: Running, Role: LEADER
I20260709 12:21:30.298439  6142 consensus_queue.cc:237] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.307144  5836 catalog_manager.cc:5697] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "8d2b1baf654845e4b25f67cb7a66a930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:30.347388  6089 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38"
dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b"
mode: GRACEFUL
new_leader_uuid: "66ea6dd222b444f984b3e44d54ebba65"
 from {username='slave'} at 127.0.0.1:45096
I20260709 12:21:30.348003  6089 raft_consensus.cc:606] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 LEADER]: Received request to transfer leadership to TS 66ea6dd222b444f984b3e44d54ebba65
I20260709 12:21:30.366657  6014 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae"
dest_uuid: "8d2b1baf654845e4b25f67cb7a66a930"
mode: GRACEFUL
new_leader_uuid: "66ea6dd222b444f984b3e44d54ebba65"
 from {username='slave'} at 127.0.0.1:37686
I20260709 12:21:30.367398  6014 raft_consensus.cc:606] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 LEADER]: Received request to transfer leadership to TS 66ea6dd222b444f984b3e44d54ebba65
I20260709 12:21:30.368343  6139 consensus_queue.cc:1048] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:30.369369  6147 consensus_queue.cc:1048] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:30.382064  6147 raft_consensus.cc:993] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b: : Instructing follower 66ea6dd222b444f984b3e44d54ebba65 to start an election
I20260709 12:21:30.383579  6147 raft_consensus.cc:1081] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 LEADER]: Signalling peer 66ea6dd222b444f984b3e44d54ebba65 to start an election
I20260709 12:21:30.391976  5940 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38"
dest_uuid: "66ea6dd222b444f984b3e44d54ebba65"
 from {username='slave'} at 127.0.0.1:59140
I20260709 12:21:30.393265  5940 raft_consensus.cc:493] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:30.393780  5940 raft_consensus.cc:3060] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.404007  5940 raft_consensus.cc:515] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.405144  5939 raft_consensus.cc:1240] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 2 FOLLOWER]: Rejecting Update request from peer 96ce369c77f44947996bbdc2f48b1f7b for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:30.407032  5940 leader_election.cc:290] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [CANDIDATE]: Term 2 election: Requested vote from peers 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2:43061), 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:30.407956  6014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38" candidate_uuid: "66ea6dd222b444f984b3e44d54ebba65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d2b1baf654845e4b25f67cb7a66a930"
I20260709 12:21:30.408340  6089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a3e81eb67043339cedf12add318a38" candidate_uuid: "66ea6dd222b444f984b3e44d54ebba65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b"
I20260709 12:21:30.409034  6089 raft_consensus.cc:3055] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:30.409440  6180 consensus_queue.cc:1059] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:30.409031  6014 raft_consensus.cc:3060] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.409567  6089 raft_consensus.cc:740] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96ce369c77f44947996bbdc2f48b1f7b, State: Running, Role: LEADER
I20260709 12:21:30.411224  6089 consensus_queue.cc:260] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.412421  6089 raft_consensus.cc:3060] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.420069  6089 raft_consensus.cc:2468] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ea6dd222b444f984b3e44d54ebba65 in term 2.
I20260709 12:21:30.420069  6014 raft_consensus.cc:2468] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ea6dd222b444f984b3e44d54ebba65 in term 2.
I20260709 12:21:30.421355  5906 leader_election.cc:304] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ea6dd222b444f984b3e44d54ebba65, 96ce369c77f44947996bbdc2f48b1f7b; no voters: 
I20260709 12:21:30.422194  6153 raft_consensus.cc:2804] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:30.422724  6153 raft_consensus.cc:697] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 2 LEADER]: Becoming Leader. State: Replica: 66ea6dd222b444f984b3e44d54ebba65, State: Running, Role: LEADER
I20260709 12:21:30.423597  6153 consensus_queue.cc:237] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.430980  5835 catalog_manager.cc:5697] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 reported cstate change: term changed from 1 to 2, leader changed from 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3) to 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "66ea6dd222b444f984b3e44d54ebba65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:30.656620  6153 consensus_queue.cc:1048] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:30.670861  6153 consensus_queue.cc:1048] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:30.819581  6181 consensus_queue.cc:1048] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:30.831873  6151 consensus_queue.cc:1048] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:30.840289  6186 raft_consensus.cc:993] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930: : Instructing follower 66ea6dd222b444f984b3e44d54ebba65 to start an election
I20260709 12:21:30.840926  6151 raft_consensus.cc:1081] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 LEADER]: Signalling peer 66ea6dd222b444f984b3e44d54ebba65 to start an election
I20260709 12:21:30.842907  5940 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae"
dest_uuid: "66ea6dd222b444f984b3e44d54ebba65"
 from {username='slave'} at 127.0.0.1:59130
I20260709 12:21:30.843469  5940 raft_consensus.cc:493] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:30.843789  5940 raft_consensus.cc:3060] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.853163  5940 raft_consensus.cc:515] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.855250  5940 leader_election.cc:290] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [CANDIDATE]: Term 2 election: Requested vote from peers 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2:43061), 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3:37633)
I20260709 12:21:30.856001  6014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae" candidate_uuid: "66ea6dd222b444f984b3e44d54ebba65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d2b1baf654845e4b25f67cb7a66a930"
I20260709 12:21:30.856770  6014 raft_consensus.cc:3055] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:30.856853  6089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae" candidate_uuid: "66ea6dd222b444f984b3e44d54ebba65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b"
I20260709 12:21:30.857406  6014 raft_consensus.cc:740] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d2b1baf654845e4b25f67cb7a66a930, State: Running, Role: LEADER
I20260709 12:21:30.857671  6089 raft_consensus.cc:3060] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.858513  6014 consensus_queue.cc:260] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.860018  6014 raft_consensus.cc:3060] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.867487  6089 raft_consensus.cc:2468] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ea6dd222b444f984b3e44d54ebba65 in term 2.
I20260709 12:21:30.868314  6014 raft_consensus.cc:2468] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ea6dd222b444f984b3e44d54ebba65 in term 2.
I20260709 12:21:30.868916  5906 leader_election.cc:304] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ea6dd222b444f984b3e44d54ebba65, 96ce369c77f44947996bbdc2f48b1f7b; no voters: 
I20260709 12:21:30.869850  6153 raft_consensus.cc:2804] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:30.870378  6153 raft_consensus.cc:697] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 2 LEADER]: Becoming Leader. State: Replica: 66ea6dd222b444f984b3e44d54ebba65, State: Running, Role: LEADER
I20260709 12:21:30.871232  6153 consensus_queue.cc:237] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:30.877961  5835 catalog_manager.cc:5697] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 reported cstate change: term changed from 1 to 2, leader changed from 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2) to 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "66ea6dd222b444f984b3e44d54ebba65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:30.884945  6089 raft_consensus.cc:1275] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 2 FOLLOWER]: Refusing update from remote peer 66ea6dd222b444f984b3e44d54ebba65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:30.886598  6153 consensus_queue.cc:1048] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:30.908344  6014 raft_consensus.cc:1275] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 2 FOLLOWER]: Refusing update from remote peer 66ea6dd222b444f984b3e44d54ebba65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:30.910022  6188 consensus_queue.cc:1048] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:31.148285 27116 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:21:31.150002 27116 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:21:31.150486 27116 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:21:31.169553  5940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae"
dest_uuid: "66ea6dd222b444f984b3e44d54ebba65"
mode: GRACEFUL
new_leader_uuid: "96ce369c77f44947996bbdc2f48b1f7b"
 from {username='slave'} at 127.0.0.1:59142
I20260709 12:21:31.170202  5940 raft_consensus.cc:606] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 2 LEADER]: Received request to transfer leadership to TS 96ce369c77f44947996bbdc2f48b1f7b
I20260709 12:21:31.171523 27116 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:21:31.323009  6089 raft_consensus.cc:1275] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 2 FOLLOWER]: Refusing update from remote peer 66ea6dd222b444f984b3e44d54ebba65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:31.324369  6188 consensus_queue.cc:1048] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:31.330291  6174 raft_consensus.cc:993] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65: : Instructing follower 96ce369c77f44947996bbdc2f48b1f7b to start an election
I20260709 12:21:31.330847  6153 raft_consensus.cc:1081] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 2 LEADER]: Signalling peer 96ce369c77f44947996bbdc2f48b1f7b to start an election
I20260709 12:21:31.332737  6088 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae"
dest_uuid: "96ce369c77f44947996bbdc2f48b1f7b"
 from {username='slave'} at 127.0.0.1:45094
I20260709 12:21:31.333371  6088 raft_consensus.cc:493] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:31.333709  6088 raft_consensus.cc:3060] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.336437  6014 raft_consensus.cc:1275] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 2 FOLLOWER]: Refusing update from remote peer 66ea6dd222b444f984b3e44d54ebba65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:31.338007  6174 consensus_queue.cc:1048] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:31.343048  6088 raft_consensus.cc:515] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:31.344292  6089 raft_consensus.cc:1240] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 3 FOLLOWER]: Rejecting Update request from peer 66ea6dd222b444f984b3e44d54ebba65 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:31.345595  6088 leader_election.cc:290] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [CANDIDATE]: Term 3 election: Requested vote from peers 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1:39999), 8d2b1baf654845e4b25f67cb7a66a930 (127.26.123.2:43061)
I20260709 12:21:31.346339  6174 consensus_queue.cc:1059] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:31.347332  6014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae" candidate_uuid: "96ce369c77f44947996bbdc2f48b1f7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8d2b1baf654845e4b25f67cb7a66a930"
I20260709 12:21:31.347431  5940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d1d081affe461e97b19e2106f69bae" candidate_uuid: "96ce369c77f44947996bbdc2f48b1f7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66ea6dd222b444f984b3e44d54ebba65"
I20260709 12:21:31.348192  6014 raft_consensus.cc:3060] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.348362  5940 raft_consensus.cc:3055] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:31.348763  5940 raft_consensus.cc:740] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 66ea6dd222b444f984b3e44d54ebba65, State: Running, Role: LEADER
I20260709 12:21:31.349591  5940 consensus_queue.cc:260] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:31.350816  5940 raft_consensus.cc:3060] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.356634  6014 raft_consensus.cc:2468] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ce369c77f44947996bbdc2f48b1f7b in term 3.
I20260709 12:21:31.358029  6055 leader_election.cc:304] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d2b1baf654845e4b25f67cb7a66a930, 96ce369c77f44947996bbdc2f48b1f7b; no voters: 
I20260709 12:21:31.358819  5940 raft_consensus.cc:2468] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ce369c77f44947996bbdc2f48b1f7b in term 3.
I20260709 12:21:31.358999  6147 raft_consensus.cc:2804] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:21:31.359562  6153 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:21:31.361466  6147 raft_consensus.cc:697] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 3 LEADER]: Becoming Leader. State: Replica: 96ce369c77f44947996bbdc2f48b1f7b, State: Running, Role: LEADER
I20260709 12:21:31.362398  6147 consensus_queue.cc:237] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } }
I20260709 12:21:31.369185  5837 catalog_manager.cc:5697] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b reported cstate change: term changed from 2 to 3, leader changed from 66ea6dd222b444f984b3e44d54ebba65 (127.26.123.1) to 96ce369c77f44947996bbdc2f48b1f7b (127.26.123.3). New cstate: current_term: 3 leader_uuid: "96ce369c77f44947996bbdc2f48b1f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ea6dd222b444f984b3e44d54ebba65" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d2b1baf654845e4b25f67cb7a66a930" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 43061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ce369c77f44947996bbdc2f48b1f7b" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 37633 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:31.387295 27116 tablet_server.cc:179] TabletServer@127.26.123.1:0 shutting down...
I20260709 12:21:31.406091 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:31.406872 27116 tablet_replica.cc:333] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65: stopping tablet replica
I20260709 12:21:31.407513 27116 raft_consensus.cc:2243] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.407998 27116 raft_consensus.cc:2272] T b2d1d081affe461e97b19e2106f69bae P 66ea6dd222b444f984b3e44d54ebba65 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.410562 27116 tablet_replica.cc:333] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65: stopping tablet replica
I20260709 12:21:31.411099 27116 raft_consensus.cc:2243] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.411911 27116 raft_consensus.cc:2272] T a567ec217d484cc99b9739c3f89531b3 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.414565 27116 tablet_replica.cc:333] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65: stopping tablet replica
I20260709 12:21:31.415079 27116 raft_consensus.cc:2243] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.415885 27116 raft_consensus.cc:2272] T f5a3e81eb67043339cedf12add318a38 P 66ea6dd222b444f984b3e44d54ebba65 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.418243 27116 tablet_replica.cc:333] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65: stopping tablet replica
I20260709 12:21:31.418700 27116 raft_consensus.cc:2243] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.419073 27116 raft_consensus.cc:2272] T 3bb68b3efbd74f84bd15e8747a28a799 P 66ea6dd222b444f984b3e44d54ebba65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.467797 27116 tablet_server.cc:196] TabletServer@127.26.123.1:0 shutdown complete.
I20260709 12:21:31.497310 27116 tablet_server.cc:179] TabletServer@127.26.123.2:0 shutting down...
I20260709 12:21:31.520530 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:31.521600 27116 tablet_replica.cc:333] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930: stopping tablet replica
I20260709 12:21:31.522764 27116 raft_consensus.cc:2243] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.523550 27116 raft_consensus.cc:2272] T a567ec217d484cc99b9739c3f89531b3 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.527607 27116 tablet_replica.cc:333] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930: stopping tablet replica
I20260709 12:21:31.528308 27116 raft_consensus.cc:2243] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.529197 27116 raft_consensus.cc:2272] T b2d1d081affe461e97b19e2106f69bae P 8d2b1baf654845e4b25f67cb7a66a930 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.532534 27116 tablet_replica.cc:333] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930: stopping tablet replica
I20260709 12:21:31.533171 27116 raft_consensus.cc:2243] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.534883 27116 raft_consensus.cc:2272] T 3bb68b3efbd74f84bd15e8747a28a799 P 8d2b1baf654845e4b25f67cb7a66a930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.538448 27116 tablet_replica.cc:333] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930: stopping tablet replica
I20260709 12:21:31.539191 27116 raft_consensus.cc:2243] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.539904 27116 raft_consensus.cc:2272] T f5a3e81eb67043339cedf12add318a38 P 8d2b1baf654845e4b25f67cb7a66a930 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.568531 27116 tablet_server.cc:196] TabletServer@127.26.123.2:0 shutdown complete.
I20260709 12:21:31.592386 27116 tablet_server.cc:179] TabletServer@127.26.123.3:0 shutting down...
I20260709 12:21:31.615680 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:31.616569 27116 tablet_replica.cc:333] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b: stopping tablet replica
I20260709 12:21:31.617403 27116 raft_consensus.cc:2243] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.618654 27116 pending_rounds.cc:70] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b: Trying to abort 2 pending ops.
I20260709 12:21:31.618878 27116 pending_rounds.cc:77] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305624335828254720 op_type: NO_OP noop_request { }
I20260709 12:21:31.619354 27116 raft_consensus.cc:2889] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:21:31.619589 27116 pending_rounds.cc:77] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305624333816782848 op_type: NO_OP noop_request { }
I20260709 12:21:31.619817 27116 raft_consensus.cc:2889] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:21:31.620133 27116 raft_consensus.cc:2272] T b2d1d081affe461e97b19e2106f69bae P 96ce369c77f44947996bbdc2f48b1f7b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.624159 27116 tablet_replica.cc:333] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b: stopping tablet replica
I20260709 12:21:31.624799 27116 raft_consensus.cc:2243] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.625427 27116 raft_consensus.cc:2272] T a567ec217d484cc99b9739c3f89531b3 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.628971 27116 tablet_replica.cc:333] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b: stopping tablet replica
I20260709 12:21:31.629616 27116 raft_consensus.cc:2243] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.630404 27116 raft_consensus.cc:2272] T f5a3e81eb67043339cedf12add318a38 P 96ce369c77f44947996bbdc2f48b1f7b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.633536 27116 tablet_replica.cc:333] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b: stopping tablet replica
I20260709 12:21:31.634104 27116 raft_consensus.cc:2243] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:31.634747 27116 raft_consensus.cc:2272] T 3bb68b3efbd74f84bd15e8747a28a799 P 96ce369c77f44947996bbdc2f48b1f7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.660813 27116 tablet_server.cc:196] TabletServer@127.26.123.3:0 shutdown complete.
I20260709 12:21:31.683310 27116 master.cc:562] Master@127.26.123.62:41417 shutting down...
I20260709 12:21:31.702068 27116 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:31.702922 27116 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:31.703261 27116 tablet_replica.cc:333] T 00000000000000000000000000000000 P f551b9d3330a42fd9561b58021624508: stopping tablet replica
I20260709 12:21:31.723788 27116 master.cc:584] Master@127.26.123.62:41417 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3155 ms)
[----------] 6 tests from LeaderRebalancerTest (362267 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:21:31.768535 27116 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.123.62:39929
I20260709 12:21:31.769822 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:31.776062  6193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:31.776126  6194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:31.776909  6196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:31.778625 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:31.779498 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:31.779681 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:31.779795 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599691779784 us; error 0 us; skew 500 ppm
I20260709 12:21:31.780335 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:31.782784 27116 webserver.cc:533] Webserver started at http://127.26.123.62:37997/ using document root <none> and password file <none>
I20260709 12:21:31.783238 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:31.783403 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:31.783633 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:31.784806 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/master-0-root/instance:
uuid: "fe410dacadf94b808c24e45361038fbb"
format_stamp: "Formatted at 2026-07-09 12:21:31 on dist-test-slave-9q7n"
I20260709 12:21:31.789311 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:31.792994  6201 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:31.793753 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:31.794034 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/master-0-root
uuid: "fe410dacadf94b808c24e45361038fbb"
format_stamp: "Formatted at 2026-07-09 12:21:31 on dist-test-slave-9q7n"
I20260709 12:21:31.794315 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:31.828780 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:31.830112 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:31.873294 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.62:39929
I20260709 12:21:31.873436  6252 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.62:39929 every 8 connection(s)
I20260709 12:21:31.877681  6253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:31.892102  6253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb: Bootstrap starting.
I20260709 12:21:31.897820  6253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:31.902523  6253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb: No bootstrap required, opened a new log
I20260709 12:21:31.905033  6253 raft_consensus.cc:359] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe410dacadf94b808c24e45361038fbb" member_type: VOTER }
I20260709 12:21:31.905488  6253 raft_consensus.cc:385] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:31.905678  6253 raft_consensus.cc:740] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe410dacadf94b808c24e45361038fbb, State: Initialized, Role: FOLLOWER
I20260709 12:21:31.906231  6253 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe410dacadf94b808c24e45361038fbb" member_type: VOTER }
I20260709 12:21:31.906731  6253 raft_consensus.cc:399] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:31.906931  6253 raft_consensus.cc:493] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:31.907155  6253 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:31.913112  6253 raft_consensus.cc:515] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe410dacadf94b808c24e45361038fbb" member_type: VOTER }
I20260709 12:21:31.913676  6253 leader_election.cc:304] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe410dacadf94b808c24e45361038fbb; no voters: 
I20260709 12:21:31.914855  6253 leader_election.cc:290] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:31.915208  6256 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:31.916576  6256 raft_consensus.cc:697] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 1 LEADER]: Becoming Leader. State: Replica: fe410dacadf94b808c24e45361038fbb, State: Running, Role: LEADER
I20260709 12:21:31.917315  6256 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe410dacadf94b808c24e45361038fbb" member_type: VOTER }
I20260709 12:21:31.917939  6253 sys_catalog.cc:565] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:31.920498  6257 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe410dacadf94b808c24e45361038fbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe410dacadf94b808c24e45361038fbb" member_type: VOTER } }
I20260709 12:21:31.920601  6258 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe410dacadf94b808c24e45361038fbb. Latest consensus state: current_term: 1 leader_uuid: "fe410dacadf94b808c24e45361038fbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe410dacadf94b808c24e45361038fbb" member_type: VOTER } }
I20260709 12:21:31.921379  6258 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:31.921960  6257 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:31.924712  6262 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:31.932407  6262 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:31.933930 27116 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:31.942353  6262 catalog_manager.cc:1383] Generated new cluster ID: ced73a8b739a427c8865c0711158d742
I20260709 12:21:31.942720  6262 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:31.983083  6262 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:31.984513  6262 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:32.001541  6262 catalog_manager.cc:6070] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb: Generated new TSK 0
I20260709 12:21:32.002255  6262 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:32.066113 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:32.072929  6274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:32.073762  6275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:32.075307  6277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:32.076172 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:32.077121 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:32.077318 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:32.077445 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599692077433 us; error 0 us; skew 500 ppm
I20260709 12:21:32.078073 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:32.080499 27116 webserver.cc:533] Webserver started at http://127.26.123.1:35089/ using document root <none> and password file <none>
I20260709 12:21:32.080998 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:32.081158 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:32.081380 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:32.082719 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-0-root/instance:
uuid: "7416dd332944470faa2d346b708a24ad"
format_stamp: "Formatted at 2026-07-09 12:21:32 on dist-test-slave-9q7n"
I20260709 12:21:32.087859 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:21:32.091697  6282 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.092623 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:21:32.092967 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "7416dd332944470faa2d346b708a24ad"
format_stamp: "Formatted at 2026-07-09 12:21:32 on dist-test-slave-9q7n"
I20260709 12:21:32.093253 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:32.113512 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:32.114977 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:32.116676 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:32.119352 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:32.119570 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.119812 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:32.119992 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.168148 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:35523
I20260709 12:21:32.168262  6344 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:35523 every 8 connection(s)
I20260709 12:21:32.173961 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:32.182997  6349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:32.185149  6350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:32.187616 27116 server_base.cc:1061] running on GCE node
W20260709 12:21:32.187726  6352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:32.189301 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:32.189553 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:32.189764 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599692189716 us; error 0 us; skew 500 ppm
I20260709 12:21:32.190047  6345 heartbeater.cc:344] Connected to a master server at 127.26.123.62:39929
I20260709 12:21:32.190397 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:32.190629  6345 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:32.191648  6345 heartbeater.cc:507] Master 127.26.123.62:39929 requested a full tablet report, sending...
I20260709 12:21:32.193696 27116 webserver.cc:533] Webserver started at http://127.26.123.2:45019/ using document root <none> and password file <none>
I20260709 12:21:32.194435 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:32.194625 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:32.194514  6218 ts_manager.cc:194] Registered new tserver with Master: 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523)
I20260709 12:21:32.195197 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:32.196764 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-1-root/instance:
uuid: "b330bead86d943e8bf0a329597ac24f9"
format_stamp: "Formatted at 2026-07-09 12:21:32 on dist-test-slave-9q7n"
I20260709 12:21:32.197575  6218 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39074
I20260709 12:21:32.202240 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:21:32.206588  6357 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.207456 27116 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:21:32.207799 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-1-root
uuid: "b330bead86d943e8bf0a329597ac24f9"
format_stamp: "Formatted at 2026-07-09 12:21:32 on dist-test-slave-9q7n"
I20260709 12:21:32.208181 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:32.245448 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:32.246902 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:32.248559 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:32.251143 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:32.251372 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.251617 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:32.251775 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.300947 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.2:45611
I20260709 12:21:32.301085  6419 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.2:45611 every 8 connection(s)
I20260709 12:21:32.306830 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:32.316005  6424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:32.316373  6423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:32.319774  6420 heartbeater.cc:344] Connected to a master server at 127.26.123.62:39929
I20260709 12:21:32.320281  6420 heartbeater.cc:461] Registering TS with master...
W20260709 12:21:32.320413  6426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:32.320463 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:32.321321  6420 heartbeater.cc:507] Master 127.26.123.62:39929 requested a full tablet report, sending...
I20260709 12:21:32.321945 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:32.322216 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:32.322422 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599692322401 us; error 0 us; skew 500 ppm
I20260709 12:21:32.323323 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:32.323767  6218 ts_manager.cc:194] Registered new tserver with Master: b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:32.325628  6218 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39082
I20260709 12:21:32.326201 27116 webserver.cc:533] Webserver started at http://127.26.123.3:39389/ using document root <none> and password file <none>
I20260709 12:21:32.326802 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:32.327013 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:32.327279 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:32.328589 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-2-root/instance:
uuid: "ec26b183414a47ca8e7e3d185016670d"
format_stamp: "Formatted at 2026-07-09 12:21:32 on dist-test-slave-9q7n"
I20260709 12:21:32.333601 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.007s
I20260709 12:21:32.337464  6431 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.338294 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:32.338622 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-2-root
uuid: "ec26b183414a47ca8e7e3d185016670d"
format_stamp: "Formatted at 2026-07-09 12:21:32 on dist-test-slave-9q7n"
I20260709 12:21:32.338979 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599329322122-27116-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:32.364535 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:32.365923 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:32.367551 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:32.370064 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:32.370282 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.370512 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:32.370666 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:32.418135 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.3:44755
I20260709 12:21:32.418257  6493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.3:44755 every 8 connection(s)
I20260709 12:21:32.433013  6494 heartbeater.cc:344] Connected to a master server at 127.26.123.62:39929
I20260709 12:21:32.433485  6494 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:32.434365  6494 heartbeater.cc:507] Master 127.26.123.62:39929 requested a full tablet report, sending...
I20260709 12:21:32.436349  6218 ts_manager.cc:194] Registered new tserver with Master: ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:32.436589 27116 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014808296s
I20260709 12:21:32.437886  6218 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39090
I20260709 12:21:32.464110  6218 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39092:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:32.467311  6218 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:32.551092  6310 tablet_service.cc:1511] Processing CreateTablet for tablet d120b13ba1044b03a9ed88a94a704041 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:32.551640  6308 tablet_service.cc:1511] Processing CreateTablet for tablet 81b89db80b78448c9a4dedcb9a3dfef0 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:32.552834  6310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d120b13ba1044b03a9ed88a94a704041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.554486  6309 tablet_service.cc:1511] Processing CreateTablet for tablet 1545406360df433fa90dee76690705a0 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:32.556066  6309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1545406360df433fa90dee76690705a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.556958  6385 tablet_service.cc:1511] Processing CreateTablet for tablet d120b13ba1044b03a9ed88a94a704041 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:32.558547  6385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d120b13ba1044b03a9ed88a94a704041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.558688  6308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b89db80b78448c9a4dedcb9a3dfef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.563165  6384 tablet_service.cc:1511] Processing CreateTablet for tablet 1545406360df433fa90dee76690705a0 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:32.566359  6383 tablet_service.cc:1511] Processing CreateTablet for tablet 81b89db80b78448c9a4dedcb9a3dfef0 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:32.569105  6382 tablet_service.cc:1511] Processing CreateTablet for tablet ea3bf4631ebe421981c7020476d222b0 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:32.571826  6307 tablet_service.cc:1511] Processing CreateTablet for tablet ea3bf4631ebe421981c7020476d222b0 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:32.573510  6307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3bf4631ebe421981c7020476d222b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.593544  6378 tablet_service.cc:1511] Processing CreateTablet for tablet 9ce53ec9b61046068b0564b54bdc9624 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:32.575011  6306 tablet_service.cc:1511] Processing CreateTablet for tablet 8f396467f79b438cb11b5bccf557ff0d (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:32.595923  6306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f396467f79b438cb11b5bccf557ff0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.599458  6459 tablet_service.cc:1511] Processing CreateTablet for tablet d120b13ba1044b03a9ed88a94a704041 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:32.601146  6459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d120b13ba1044b03a9ed88a94a704041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.603085  6458 tablet_service.cc:1511] Processing CreateTablet for tablet 1545406360df433fa90dee76690705a0 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:32.604094  6308 tablet_service.cc:1511] Processing CreateTablet for tablet e707cafcda8a4012953a99b8aed9340f (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:32.604744  6458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1545406360df433fa90dee76690705a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.605507  6308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e707cafcda8a4012953a99b8aed9340f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.606961  6457 tablet_service.cc:1511] Processing CreateTablet for tablet 81b89db80b78448c9a4dedcb9a3dfef0 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:32.608400  6457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b89db80b78448c9a4dedcb9a3dfef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.610514  6456 tablet_service.cc:1511] Processing CreateTablet for tablet ea3bf4631ebe421981c7020476d222b0 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:32.612273  6451 tablet_service.cc:1511] Processing CreateTablet for tablet e707cafcda8a4012953a99b8aed9340f (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:32.613579  6455 tablet_service.cc:1511] Processing CreateTablet for tablet 8f396467f79b438cb11b5bccf557ff0d (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:32.616209  6454 tablet_service.cc:1511] Processing CreateTablet for tablet 445b5ae598f64f36a672a41bb779b96f (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:32.619113  6453 tablet_service.cc:1511] Processing CreateTablet for tablet 18d6b432d7dc44e9a89513234e129bd7 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:32.622331  6452 tablet_service.cc:1511] Processing CreateTablet for tablet 9ce53ec9b61046068b0564b54bdc9624 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:32.625229  6377 tablet_service.cc:1511] Processing CreateTablet for tablet e707cafcda8a4012953a99b8aed9340f (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:32.577028  6305 tablet_service.cc:1511] Processing CreateTablet for tablet 445b5ae598f64f36a672a41bb779b96f (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:32.629300  6305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 445b5ae598f64f36a672a41bb779b96f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.579092  6380 tablet_service.cc:1511] Processing CreateTablet for tablet 445b5ae598f64f36a672a41bb779b96f (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:32.581112  6304 tablet_service.cc:1511] Processing CreateTablet for tablet 18d6b432d7dc44e9a89513234e129bd7 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:32.635742  6304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18d6b432d7dc44e9a89513234e129bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.583112  6379 tablet_service.cc:1511] Processing CreateTablet for tablet 18d6b432d7dc44e9a89513234e129bd7 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:32.585115  6381 tablet_service.cc:1511] Processing CreateTablet for tablet 8f396467f79b438cb11b5bccf557ff0d (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:32.612272  6456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3bf4631ebe421981c7020476d222b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.641333  6451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e707cafcda8a4012953a99b8aed9340f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.644023  6455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f396467f79b438cb11b5bccf557ff0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.646486  6454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 445b5ae598f64f36a672a41bb779b96f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.651880  6453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18d6b432d7dc44e9a89513234e129bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.657788  6452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce53ec9b61046068b0564b54bdc9624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.564860  6384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1545406360df433fa90dee76690705a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.590451  6303 tablet_service.cc:1511] Processing CreateTablet for tablet 9ce53ec9b61046068b0564b54bdc9624 (DEFAULT_TABLE table=test-workload [id=995b47b0b18746c8ae8a3f6c77ccda26]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:32.659377  6303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce53ec9b61046068b0564b54bdc9624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.661942  6381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f396467f79b438cb11b5bccf557ff0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.664063  6378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce53ec9b61046068b0564b54bdc9624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.666123  6380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 445b5ae598f64f36a672a41bb779b96f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.666846  6382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3bf4631ebe421981c7020476d222b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.669446  6377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e707cafcda8a4012953a99b8aed9340f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.671590  6383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b89db80b78448c9a4dedcb9a3dfef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.673910  6379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18d6b432d7dc44e9a89513234e129bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:32.718698  6509 tablet_bootstrap.cc:492] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:32.762317  6510 tablet_bootstrap.cc:492] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:32.781103  6509 tablet_bootstrap.cc:654] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.791237  6511 tablet_bootstrap.cc:492] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:32.807878  6510 tablet_bootstrap.cc:654] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.837421  6511 tablet_bootstrap.cc:654] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.847636  6510 tablet_bootstrap.cc:492] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:32.848270  6510 ts_tablet_manager.cc:1403] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.086s	user 0.009s	sys 0.014s
I20260709 12:21:32.851854  6510 raft_consensus.cc:359] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.852149  6509 tablet_bootstrap.cc:492] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:32.852478  6510 raft_consensus.cc:385] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.852820  6510 raft_consensus.cc:740] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.852888  6509 ts_tablet_manager.cc:1403] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.135s	user 0.020s	sys 0.016s
I20260709 12:21:32.853878  6510 consensus_queue.cc:260] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.856101  6511 tablet_bootstrap.cc:492] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:32.856441  6494 heartbeater.cc:499] Master 127.26.123.62:39929 was elected leader, sending a full tablet report...
I20260709 12:21:32.856729  6511 ts_tablet_manager.cc:1403] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.066s	user 0.007s	sys 0.006s
I20260709 12:21:32.856820  6510 ts_tablet_manager.cc:1434] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:32.857864  6510 tablet_bootstrap.cc:492] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:32.860067  6511 raft_consensus.cc:359] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:32.860814  6511 raft_consensus.cc:385] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.861146  6511 raft_consensus.cc:740] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.861078  6509 raft_consensus.cc:359] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.862088  6509 raft_consensus.cc:385] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.862424  6509 raft_consensus.cc:740] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.862375  6511 consensus_queue.cc:260] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:32.863595  6509 consensus_queue.cc:260] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.864369  6510 tablet_bootstrap.cc:654] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.870491  6509 ts_tablet_manager.cc:1434] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.003s
I20260709 12:21:32.871546  6345 heartbeater.cc:499] Master 127.26.123.62:39929 was elected leader, sending a full tablet report...
I20260709 12:21:32.883773  6509 tablet_bootstrap.cc:492] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:32.891885  6509 tablet_bootstrap.cc:654] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.893405  6420 heartbeater.cc:499] Master 127.26.123.62:39929 was elected leader, sending a full tablet report...
I20260709 12:21:32.893987  6511 ts_tablet_manager.cc:1434] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.037s	user 0.017s	sys 0.022s
I20260709 12:21:32.895123  6511 tablet_bootstrap.cc:492] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:32.900079  6510 tablet_bootstrap.cc:492] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:32.900745  6510 ts_tablet_manager.cc:1403] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.043s	user 0.023s	sys 0.019s
I20260709 12:21:32.904356  6510 raft_consensus.cc:359] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.904913  6511 tablet_bootstrap.cc:654] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.905334  6510 raft_consensus.cc:385] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.905814  6510 raft_consensus.cc:740] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.906831  6510 consensus_queue.cc:260] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.911510  6510 ts_tablet_manager.cc:1434] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:21:32.912859  6510 tablet_bootstrap.cc:492] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:32.920318  6510 tablet_bootstrap.cc:654] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.925410  6511 tablet_bootstrap.cc:492] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:32.926039  6511 ts_tablet_manager.cc:1403] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.015s
I20260709 12:21:32.927338  6510 tablet_bootstrap.cc:492] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:32.927927  6510 ts_tablet_manager.cc:1403] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.008s
W20260709 12:21:32.928615  6266 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:32.928841  6509 tablet_bootstrap.cc:492] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:32.929483  6509 ts_tablet_manager.cc:1403] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.004s
I20260709 12:21:32.931123  6510 raft_consensus.cc:359] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.932066  6510 raft_consensus.cc:385] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.931550  6511 raft_consensus.cc:359] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.932482  6510 raft_consensus.cc:740] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.932693  6511 raft_consensus.cc:385] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.933197  6511 raft_consensus.cc:740] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.933211  6509 raft_consensus.cc:359] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.933593  6510 consensus_queue.cc:260] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.934180  6509 raft_consensus.cc:385] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.934588  6509 raft_consensus.cc:740] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.934291  6511 consensus_queue.cc:260] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.935546  6509 consensus_queue.cc:260] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.937616  6511 ts_tablet_manager.cc:1434] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:21:32.939265  6511 tablet_bootstrap.cc:492] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:32.939410  6515 raft_consensus.cc:493] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.939957  6515 raft_consensus.cc:515] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.942039  6510 ts_tablet_manager.cc:1434] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:21:32.943166  6510 tablet_bootstrap.cc:492] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:32.946060  6515 leader_election.cc:290] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:32.946359  6509 ts_tablet_manager.cc:1434] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:21:32.947913  6511 tablet_bootstrap.cc:654] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.948254  6509 tablet_bootstrap.cc:492] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:32.949707  6510 tablet_bootstrap.cc:654] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.960721  6511 tablet_bootstrap.cc:492] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:32.961463  6511 ts_tablet_manager.cc:1403] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.005s
I20260709 12:21:32.964623  6515 raft_consensus.cc:493] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.965157  6511 raft_consensus.cc:359] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.965229  6515 raft_consensus.cc:515] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.967904  6515 leader_election.cc:290] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:32.968530  6511 raft_consensus.cc:385] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.968986  6511 raft_consensus.cc:740] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.970335  6511 consensus_queue.cc:260] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.973280  6511 ts_tablet_manager.cc:1434] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:21:32.974112  6509 tablet_bootstrap.cc:654] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.974695  6511 tablet_bootstrap.cc:492] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:32.983551  6511 tablet_bootstrap.cc:654] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:32.989037  6509 tablet_bootstrap.cc:492] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:32.989893  6509 ts_tablet_manager.cc:1403] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.004s
I20260709 12:21:32.993688  6509 raft_consensus.cc:359] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.994782  6509 raft_consensus.cc:385] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:32.995151  6509 raft_consensus.cc:740] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:32.996476  6509 consensus_queue.cc:260] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:32.997313  6511 tablet_bootstrap.cc:492] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:32.997902  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d6b432d7dc44e9a89513234e129bd7" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:32.998235  6511 ts_tablet_manager.cc:1403] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
W20260709 12:21:33.000373  6433 leader_election.cc:343] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.001899  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445b5ae598f64f36a672a41bb779b96f" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.002329  6511 raft_consensus.cc:359] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
W20260709 12:21:33.004722  6433 leader_election.cc:343] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.005082  6511 raft_consensus.cc:385] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.007233  6511 raft_consensus.cc:740] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.007928  6509 ts_tablet_manager.cc:1434] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.005s
I20260709 12:21:33.009359  6509 tablet_bootstrap.cc:492] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:33.011513  6510 tablet_bootstrap.cc:492] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:33.012324  6510 ts_tablet_manager.cc:1403] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.069s	user 0.044s	sys 0.017s
I20260709 12:21:33.015861  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d6b432d7dc44e9a89513234e129bd7" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.008232  6511 consensus_queue.cc:260] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.016752  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445b5ae598f64f36a672a41bb779b96f" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.016894  6510 raft_consensus.cc:359] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.018025  6510 raft_consensus.cc:385] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.018409  6510 raft_consensus.cc:740] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
W20260709 12:21:33.018510  6433 leader_election.cc:343] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.019171  6511 ts_tablet_manager.cc:1434] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:21:33.019089  6433 leader_election.cc:304] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:33.019726  6510 consensus_queue.cc:260] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.020771  6511 tablet_bootstrap.cc:492] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:33.022801  6509 tablet_bootstrap.cc:654] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:33.020866  6433 leader_election.cc:343] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.026368  6433 leader_election.cc:304] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:33.020669  6515 raft_consensus.cc:2749] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.030527  6511 tablet_bootstrap.cc:654] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.030840  6515 raft_consensus.cc:2749] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.042999  6511 tablet_bootstrap.cc:492] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:33.043821  6511 ts_tablet_manager.cc:1403] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.008s
I20260709 12:21:33.039057  6510 ts_tablet_manager.cc:1434] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.026s	user 0.013s	sys 0.004s
I20260709 12:21:33.046106  6510 tablet_bootstrap.cc:492] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:33.046810  6509 tablet_bootstrap.cc:492] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:33.047452  6509 ts_tablet_manager.cc:1403] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.002s
I20260709 12:21:33.047770  6511 raft_consensus.cc:359] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.048806  6511 raft_consensus.cc:385] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.049134  6511 raft_consensus.cc:740] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.050034  6511 consensus_queue.cc:260] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.051321  6509 raft_consensus.cc:359] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:33.052230  6509 raft_consensus.cc:385] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.052541  6509 raft_consensus.cc:740] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.052786  6511 ts_tablet_manager.cc:1434] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:21:33.054133  6511 tablet_bootstrap.cc:492] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:33.053597  6509 consensus_queue.cc:260] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:33.054324  6510 tablet_bootstrap.cc:654] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.062952  6511 tablet_bootstrap.cc:654] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.065783  6509 ts_tablet_manager.cc:1434] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:21:33.074011  6511 tablet_bootstrap.cc:492] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:33.074205  6510 tablet_bootstrap.cc:492] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:33.074867  6511 ts_tablet_manager.cc:1403] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.005s
I20260709 12:21:33.075026  6509 tablet_bootstrap.cc:492] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:33.074936  6510 ts_tablet_manager.cc:1403] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.029s	user 0.020s	sys 0.005s
I20260709 12:21:33.079025  6511 raft_consensus.cc:359] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.079207  6510 raft_consensus.cc:359] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.079928  6511 raft_consensus.cc:385] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.080034  6510 raft_consensus.cc:385] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.080318  6511 raft_consensus.cc:740] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.080457  6510 raft_consensus.cc:740] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.081399  6510 consensus_queue.cc:260] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.081440  6511 consensus_queue.cc:260] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.082221  6509 tablet_bootstrap.cc:654] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.084161  6510 ts_tablet_manager.cc:1434] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:21:33.085522  6510 tablet_bootstrap.cc:492] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:33.087910  6511 ts_tablet_manager.cc:1434] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:21:33.089222  6511 tablet_bootstrap.cc:492] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:33.094266  6510 tablet_bootstrap.cc:654] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.095818  6516 raft_consensus.cc:493] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.096715  6516 raft_consensus.cc:515] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.103291  6516 leader_election.cc:290] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.105149  6516 raft_consensus.cc:493] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.106050  6510 tablet_bootstrap.cc:492] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:33.105962  6516 raft_consensus.cc:515] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:33.106899  6510 ts_tablet_manager.cc:1403] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 12:21:33.109067  6516 leader_election.cc:290] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.110973  6510 raft_consensus.cc:359] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.109539  6511 tablet_bootstrap.cc:654] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.112026  6510 raft_consensus.cc:385] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.112447  6510 raft_consensus.cc:740] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.113636  6510 consensus_queue.cc:260] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.114428  6509 tablet_bootstrap.cc:492] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:33.115379  6509 ts_tablet_manager.cc:1403] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.007s
I20260709 12:21:33.118981  6510 ts_tablet_manager.cc:1434] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:21:33.119313  6509 raft_consensus.cc:359] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.120429  6509 raft_consensus.cc:385] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.120590  6510 tablet_bootstrap.cc:492] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:33.120837  6509 raft_consensus.cc:740] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.122152  6509 consensus_queue.cc:260] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.128257  6509 ts_tablet_manager.cc:1434] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:21:33.129671  6509 tablet_bootstrap.cc:492] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:33.129660  6510 tablet_bootstrap.cc:654] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.132853  6529 raft_consensus.cc:493] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.133502  6529 raft_consensus.cc:515] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.136083  6529 leader_election.cc:290] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:33.136648  6529 raft_consensus.cc:493] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.137130  6529 raft_consensus.cc:515] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.137575  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1545406360df433fa90dee76690705a0" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.138397  6509 tablet_bootstrap.cc:654] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.139691  6529 leader_election.cc:290] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:33.140661  6319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e707cafcda8a4012953a99b8aed9340f" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.141561  6319 raft_consensus.cc:2468] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:33.143740  6433 leader_election.cc:304] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:33.144886  6529 raft_consensus.cc:2804] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.145325  6529 raft_consensus.cc:493] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.145717  6529 raft_consensus.cc:3060] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.146148  6511 tablet_bootstrap.cc:492] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:33.146932  6511 ts_tablet_manager.cc:1403] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.009s
I20260709 12:21:33.138361  6320 raft_consensus.cc:2468] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:33.151510  6516 raft_consensus.cc:493] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.152452  6509 tablet_bootstrap.cc:492] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:33.152132  6516 raft_consensus.cc:515] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.153597  6433 leader_election.cc:304] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:33.154803  6516 leader_election.cc:290] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.154903  6528 raft_consensus.cc:2804] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.155413  6511 raft_consensus.cc:359] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.156082  6528 raft_consensus.cc:493] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.156718  6528 raft_consensus.cc:3060] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.156050  6509 ts_tablet_manager.cc:1403] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.027s	user 0.004s	sys 0.017s
I20260709 12:21:33.157174  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1545406360df433fa90dee76690705a0" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.158087  6394 raft_consensus.cc:2468] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:33.160579  6517 raft_consensus.cc:493] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.161201  6517 raft_consensus.cc:515] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:33.161365  6509 raft_consensus.cc:359] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.162392  6509 raft_consensus.cc:385] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.162772  6509 raft_consensus.cc:740] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.163678  6509 consensus_queue.cc:260] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.156257  6511 raft_consensus.cc:385] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.164770  6511 raft_consensus.cc:740] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.165692  6511 consensus_queue.cc:260] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.168350  6511 ts_tablet_manager.cc:1434] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.021s	user 0.002s	sys 0.005s
I20260709 12:21:33.168588  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.169461  6469 raft_consensus.cc:2468] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:33.169575  6511 tablet_bootstrap.cc:492] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:33.171068  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.172755  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.173617  6467 raft_consensus.cc:2468] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:33.177220  6358 leader_election.cc:304] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
W20260709 12:21:33.178454  6358 leader_election.cc:343] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.179235  6358 leader_election.cc:304] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:33.179451  6511 tablet_bootstrap.cc:654] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.169886  6517 leader_election.cc:290] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:33.179888  6516 raft_consensus.cc:2804] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.180375  6516 raft_consensus.cc:493] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.180721  6516 raft_consensus.cc:3060] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.190989  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e707cafcda8a4012953a99b8aed9340f" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.191931  6395 raft_consensus.cc:2468] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:33.193195  6510 tablet_bootstrap.cc:492] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:33.194406  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.195277  6320 raft_consensus.cc:2468] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:33.196655  6516 raft_consensus.cc:515] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.200732  6510 ts_tablet_manager.cc:1403] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.080s	user 0.025s	sys 0.004s
I20260709 12:21:33.204381  6510 raft_consensus.cc:359] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.205170  6510 raft_consensus.cc:385] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.206622  6528 raft_consensus.cc:515] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.210129  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1545406360df433fa90dee76690705a0" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:33.210794  6320 raft_consensus.cc:3060] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.210573  6545 raft_consensus.cc:2804] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.215070  6545 raft_consensus.cc:493] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.215451  6545 raft_consensus.cc:3060] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.216926  6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.218788  6528 leader_election.cc:290] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:33.219038  6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:33.219676  6318 raft_consensus.cc:3060] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.221819  6319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.222661  6319 raft_consensus.cc:2468] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:33.205487  6510 raft_consensus.cc:740] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.236271  6510 consensus_queue.cc:260] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.250029  6510 ts_tablet_manager.cc:1434] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.049s	user 0.011s	sys 0.000s
I20260709 12:21:33.251303  6510 tablet_bootstrap.cc:492] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
W20260709 12:21:33.262090  6359 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.1:35523, user_credentials={real_user=slave}} blocked reactor thread for 46782.8us
I20260709 12:21:33.263383  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:33.263639  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1545406360df433fa90dee76690705a0" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:33.264163  6467 raft_consensus.cc:3060] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.264252  6395 raft_consensus.cc:3060] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.273895  6516 leader_election.cc:290] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
W20260709 12:21:33.274168  6359 leader_election.cc:343] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.275122  6467 raft_consensus.cc:2468] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:33.275346  6359 leader_election.cc:304] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:33.276674  6358 leader_election.cc:304] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:33.280546  6516 raft_consensus.cc:2804] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.280972  6516 raft_consensus.cc:493] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.281301  6516 raft_consensus.cc:3060] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.291488  6395 raft_consensus.cc:2468] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:33.293056  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
W20260709 12:21:33.295198  6283 leader_election.cc:343] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.297591  6433 leader_election.cc:304] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:33.298800  6528 raft_consensus.cc:2804] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.299353  6528 raft_consensus.cc:697] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:33.300279  6528 consensus_queue.cc:237] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.304072  6320 raft_consensus.cc:2468] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:33.293545  6546 raft_consensus.cc:2804] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.312391  6318 raft_consensus.cc:2468] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:33.316918  6529 raft_consensus.cc:515] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.306085  6545 raft_consensus.cc:515] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.324067  6516 raft_consensus.cc:515] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:33.324874  6318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:33.327349  6509 ts_tablet_manager.cc:1434] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.169s	user 0.044s	sys 0.092s
I20260709 12:21:33.327966  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:33.328344  6216 catalog_manager.cc:5697] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 0 to 1, leader changed from <none> to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.329895  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
W20260709 12:21:33.330090  6358 leader_election.cc:343] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.330847  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:33.331521  6320 raft_consensus.cc:3060] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.332597  6511 tablet_bootstrap.cc:492] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:33.333421  6511 ts_tablet_manager.cc:1403] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.164s	user 0.051s	sys 0.091s
I20260709 12:21:33.330533  6467 raft_consensus.cc:3060] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.333091  6509 tablet_bootstrap.cc:492] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:33.334225  6319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e707cafcda8a4012953a99b8aed9340f" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:33.338599  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.340790  6319 raft_consensus.cc:3060] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.341038  6395 raft_consensus.cc:2393] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7416dd332944470faa2d346b708a24ad in current term 1: Already voted for candidate b330bead86d943e8bf0a329597ac24f9 in this term.
I20260709 12:21:33.343055  6284 leader_election.cc:304] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7416dd332944470faa2d346b708a24ad; no voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:33.343678  6511 raft_consensus.cc:359] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.344692  6511 raft_consensus.cc:385] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.334677  6546 raft_consensus.cc:697] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:33.346562  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e707cafcda8a4012953a99b8aed9340f" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:33.345041  6511 raft_consensus.cc:740] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.333310  6545 leader_election.cc:290] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.333645  6516 leader_election.cc:290] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
W20260709 12:21:33.349064  6359 leader_election.cc:343] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.348939  6511 consensus_queue.cc:260] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.346745  6546 consensus_queue.cc:237] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.349601  6395 raft_consensus.cc:3060] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.351269  6509 tablet_bootstrap.cc:654] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.358253  6467 raft_consensus.cc:2468] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:33.359594  6319 raft_consensus.cc:2468] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:33.360116  6358 leader_election.cc:304] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad
I20260709 12:21:33.347349  6529 leader_election.cc:290] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:33.361456  6510 tablet_bootstrap.cc:654] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.361202  6433 leader_election.cc:304] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:33.362545  6529 raft_consensus.cc:2804] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.363391  6529 raft_consensus.cc:697] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:33.364413  6529 consensus_queue.cc:237] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.364980  6548 raft_consensus.cc:2804] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.365567  6545 raft_consensus.cc:493] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.366034  6548 raft_consensus.cc:697] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:33.366485  6545 raft_consensus.cc:515] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.354683  6320 raft_consensus.cc:2468] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:33.367581  6542 raft_consensus.cc:2749] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.368425  6359 leader_election.cc:304] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:33.369488  6511 ts_tablet_manager.cc:1434] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:21:33.370483  6516 raft_consensus.cc:2804] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.370692  6511 tablet_bootstrap.cc:492] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:33.370956  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445b5ae598f64f36a672a41bb779b96f" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.367061  6548 consensus_queue.cc:237] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.370980  6545 leader_election.cc:290] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.371387  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445b5ae598f64f36a672a41bb779b96f" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.372584  6467 raft_consensus.cc:2468] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:33.373880  6358 leader_election.cc:304] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
W20260709 12:21:33.374718  6359 leader_election.cc:343] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.374878  6545 raft_consensus.cc:2804] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.375289  6545 raft_consensus.cc:493] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.375654  6545 raft_consensus.cc:3060] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.378702  6542 raft_consensus.cc:493] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.379354  6542 raft_consensus.cc:515] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.388540  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b89db80b78448c9a4dedcb9a3dfef0" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.388792  6542 leader_election.cc:290] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
W20260709 12:21:33.391512  6284 leader_election.cc:343] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.392637  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b89db80b78448c9a4dedcb9a3dfef0" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.393491  6467 raft_consensus.cc:2468] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 0.
I20260709 12:21:33.394279  6510 tablet_bootstrap.cc:492] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:33.394914  6510 ts_tablet_manager.cc:1403] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.144s	user 0.010s	sys 0.012s
I20260709 12:21:33.395617  6283 leader_election.cc:304] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:33.396926  6542 raft_consensus.cc:2804] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.397300  6542 raft_consensus.cc:493] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.397665  6542 raft_consensus.cc:3060] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.398531  6510 raft_consensus.cc:359] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:33.399350  6510 raft_consensus.cc:385] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.399631  6510 raft_consensus.cc:740] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.399863  6546 raft_consensus.cc:493] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.395807  6516 raft_consensus.cc:697] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:33.400478  6546 raft_consensus.cc:515] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.400482  6510 consensus_queue.cc:260] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:33.401803  6516 consensus_queue.cc:237] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:33.403486  6510 ts_tablet_manager.cc:1434] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:33.400935  6511 tablet_bootstrap.cc:654] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.404374  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d6b432d7dc44e9a89513234e129bd7" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.405797  6546 leader_election.cc:290] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.394774  6509 tablet_bootstrap.cc:492] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:33.406829  6509 ts_tablet_manager.cc:1403] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.074s	user 0.020s	sys 0.000s
I20260709 12:21:33.407156  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d6b432d7dc44e9a89513234e129bd7" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.408051  6467 raft_consensus.cc:2468] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
W20260709 12:21:33.407051  6359 leader_election.cc:343] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.409866  6358 leader_election.cc:304] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad
I20260709 12:21:33.410693  6510 tablet_bootstrap.cc:492] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:33.411051  6546 raft_consensus.cc:2804] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.411510  6546 raft_consensus.cc:493] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.411859  6546 raft_consensus.cc:3060] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.410542  6509 raft_consensus.cc:359] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.364116  6395 raft_consensus.cc:2468] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:33.413645  6509 raft_consensus.cc:385] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.414059  6509 raft_consensus.cc:740] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.415094  6509 consensus_queue.cc:260] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.418972  6511 tablet_bootstrap.cc:492] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:33.419006  6510 tablet_bootstrap.cc:654] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.419835  6511 ts_tablet_manager.cc:1403] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.003s
I20260709 12:21:33.422173  6509 ts_tablet_manager.cc:1434] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:21:33.423544  6511 raft_consensus.cc:359] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.424515  6511 raft_consensus.cc:385] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.424875  6511 raft_consensus.cc:740] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.426033  6511 consensus_queue.cc:260] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.432796  6511 ts_tablet_manager.cc:1434] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:21:33.432937  6510 tablet_bootstrap.cc:492] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:33.436481  6510 ts_tablet_manager.cc:1403] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.001s
I20260709 12:21:33.436394  6217 catalog_manager.cc:5697] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.439211  6510 raft_consensus.cc:359] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.439839  6510 raft_consensus.cc:385] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.440050  6510 raft_consensus.cc:740] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.440802  6510 consensus_queue.cc:260] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.423508  6509 tablet_bootstrap.cc:492] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:33.441234  6542 raft_consensus.cc:515] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.444891  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b89db80b78448c9a4dedcb9a3dfef0" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:33.444893  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b89db80b78448c9a4dedcb9a3dfef0" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:33.445603  6395 raft_consensus.cc:3060] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.445878  6467 raft_consensus.cc:3060] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.449287  6509 tablet_bootstrap.cc:654] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.439028  6545 raft_consensus.cc:515] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.451742  6545 leader_election.cc:290] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.453011  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445b5ae598f64f36a672a41bb779b96f" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:33.453589  6510 ts_tablet_manager.cc:1434] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:21:33.453838  6468 raft_consensus.cc:3060] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.456051  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445b5ae598f64f36a672a41bb779b96f" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:33.456418  6467 raft_consensus.cc:2468] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 1.
I20260709 12:21:33.458186  6283 leader_election.cc:304] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
W20260709 12:21:33.459081  6359 leader_election.cc:343] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.459945  6517 raft_consensus.cc:2804] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.465174  6545 raft_consensus.cc:493] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.465575  6542 leader_election.cc:290] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 election: Requested vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.465850  6517 raft_consensus.cc:697] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 1 LEADER]: Becoming Leader. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:33.465888  6545 raft_consensus.cc:515] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.466888  6517 consensus_queue.cc:237] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.468086  6468 raft_consensus.cc:2468] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:33.469496  6358 leader_election.cc:304] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad
I20260709 12:21:33.469666  6545 leader_election.cc:290] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.470600  6545 raft_consensus.cc:2804] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.471216  6545 raft_consensus.cc:697] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:33.472276  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f396467f79b438cb11b5bccf557ff0d" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.470841  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f396467f79b438cb11b5bccf557ff0d" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.472321  6545 consensus_queue.cc:237] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.473280  6467 raft_consensus.cc:2468] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:33.473693  6320 raft_consensus.cc:2468] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:33.465288  6395 raft_consensus.cc:2468] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 1.
I20260709 12:21:33.475069  6358 leader_election.cc:304] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:33.476148  6516 raft_consensus.cc:2804] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:33.476557  6516 raft_consensus.cc:493] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.476955  6516 raft_consensus.cc:3060] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.432571  6546 raft_consensus.cc:515] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.465389  6218 catalog_manager.cc:5697] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 0 to 1, leader changed from <none> to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.488469  6516 raft_consensus.cc:515] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.492812  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f396467f79b438cb11b5bccf557ff0d" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:33.493602  6467 raft_consensus.cc:3060] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.499953  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d6b432d7dc44e9a89513234e129bd7" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:33.505921  6516 leader_election.cc:290] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.505307  6467 raft_consensus.cc:2468] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:33.507954  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f396467f79b438cb11b5bccf557ff0d" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:33.508905  6320 raft_consensus.cc:3060] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.509059  6358 leader_election.cc:304] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:33.511173  6516 raft_consensus.cc:2804] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.512127  6516 raft_consensus.cc:697] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:33.513317  6516 consensus_queue.cc:237] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
W20260709 12:21:33.530409  6359 leader_election.cc:343] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:33.532133  6217 catalog_manager.cc:5697] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 7416dd332944470faa2d346b708a24ad (127.26.123.1). New cstate: current_term: 1 leader_uuid: "7416dd332944470faa2d346b708a24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.538930  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d6b432d7dc44e9a89513234e129bd7" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:33.539884  6467 raft_consensus.cc:3060] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.549913  6546 leader_election.cc:290] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.553434  6467 raft_consensus.cc:2468] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:33.557188  6320 raft_consensus.cc:2468] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:33.560696  6509 tablet_bootstrap.cc:492] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:33.560634  6358 leader_election.cc:304] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad
I20260709 12:21:33.561403  6509 ts_tablet_manager.cc:1403] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.138s	user 0.062s	sys 0.046s
I20260709 12:21:33.562682  6545 raft_consensus.cc:2804] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:33.563364  6545 raft_consensus.cc:697] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:33.564306  6545 consensus_queue.cc:237] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.564340  6509 raft_consensus.cc:359] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.565330  6509 raft_consensus.cc:385] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.565649  6509 raft_consensus.cc:740] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.566450  6509 consensus_queue.cc:260] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.568815  6509 ts_tablet_manager.cc:1434] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:33.570103  6509 tablet_bootstrap.cc:492] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:33.569921  6217 catalog_manager.cc:5697] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:33.572398  6217 catalog_manager.cc:5697] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.574681  6217 catalog_manager.cc:5697] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.579182  6509 tablet_bootstrap.cc:654] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:33.584400  6509 tablet_bootstrap.cc:492] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:33.584887  6509 ts_tablet_manager.cc:1403] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 12:21:33.587450  6509 raft_consensus.cc:359] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.587989  6509 raft_consensus.cc:385] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:33.588191  6509 raft_consensus.cc:740] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:33.588827  6509 consensus_queue.cc:260] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.591120  6509 ts_tablet_manager.cc:1434] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:21:33.597685  6215 catalog_manager.cc:5697] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.599253  6215 catalog_manager.cc:5697] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:33.647235  6517 raft_consensus.cc:493] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.647859  6517 raft_consensus.cc:515] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.649997  6517 leader_election.cc:290] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.650784  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.651760  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.652627  6467 raft_consensus.cc:2393] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7416dd332944470faa2d346b708a24ad in current term 1: Already voted for candidate b330bead86d943e8bf0a329597ac24f9 in this term.
I20260709 12:21:33.653720  6283 leader_election.cc:304] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7416dd332944470faa2d346b708a24ad; no voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:33.654582  6517 raft_consensus.cc:3060] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.664515  6517 raft_consensus.cc:2749] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.668586  6395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041"
dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
mode: GRACEFUL
new_leader_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:42160
I20260709 12:21:33.669323  6395 raft_consensus.cc:606] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Received request to transfer leadership to TS 7416dd332944470faa2d346b708a24ad
I20260709 12:21:33.680953  6517 raft_consensus.cc:493] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.681653  6517 raft_consensus.cc:515] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.684620  6467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1545406360df433fa90dee76690705a0"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
mode: GRACEFUL
new_leader_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:45786
I20260709 12:21:33.684818  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d6b432d7dc44e9a89513234e129bd7" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.685352  6467 raft_consensus.cc:606] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Received request to transfer leadership to TS 7416dd332944470faa2d346b708a24ad
I20260709 12:21:33.686440  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18d6b432d7dc44e9a89513234e129bd7" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.686910  6517 leader_election.cc:290] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.687372  6468 raft_consensus.cc:2393] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7416dd332944470faa2d346b708a24ad in current term 1: Already voted for candidate b330bead86d943e8bf0a329597ac24f9 in this term.
I20260709 12:21:33.688454  6283 leader_election.cc:304] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7416dd332944470faa2d346b708a24ad; no voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:33.689180  6517 raft_consensus.cc:3060] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.694792  6395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0"
dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
mode: GRACEFUL
new_leader_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:42160
I20260709 12:21:33.695621  6395 raft_consensus.cc:606] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Received request to transfer leadership to TS 7416dd332944470faa2d346b708a24ad
I20260709 12:21:33.695761  6517 raft_consensus.cc:2749] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.794819  6545 consensus_queue.cc:1048] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.807978  6545 consensus_queue.cc:1048] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.811345  6529 consensus_queue.cc:1048] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.832056  6528 consensus_queue.cc:1048] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.836118  6529 consensus_queue.cc:1048] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.838048  6529 raft_consensus.cc:493] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.838732  6529 raft_consensus.cc:515] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:33.842425  6319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:33.843545  6319 raft_consensus.cc:2393] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec26b183414a47ca8e7e3d185016670d in current term 1: Already voted for candidate b330bead86d943e8bf0a329597ac24f9 in this term.
I20260709 12:21:33.846395  6529 leader_election.cc:290] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:33.847738  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.849692  6433 leader_election.cc:304] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:33.850687  6529 raft_consensus.cc:3060] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.856469  6528 consensus_queue.cc:1048] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.861014  6529 raft_consensus.cc:2749] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.867884  6529 raft_consensus.cc:993] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d: : Instructing follower 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:33.869432  6529 raft_consensus.cc:1081] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Signalling peer 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:33.879251  6319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1545406360df433fa90dee76690705a0"
dest_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:53124
I20260709 12:21:33.880877  6319 raft_consensus.cc:493] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:33.881345  6319 raft_consensus.cc:3060] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.885529  6546 consensus_queue.cc:1048] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.889286  6517 raft_consensus.cc:493] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.889210  6546 consensus_queue.cc:1048] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.889995  6517 raft_consensus.cc:515] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.893572  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445b5ae598f64f36a672a41bb779b96f" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:33.894513  6319 raft_consensus.cc:515] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.897154  6517 leader_election.cc:290] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.898834  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "445b5ae598f64f36a672a41bb779b96f" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:33.899820  6467 raft_consensus.cc:2393] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7416dd332944470faa2d346b708a24ad in current term 1: Already voted for candidate b330bead86d943e8bf0a329597ac24f9 in this term.
I20260709 12:21:33.901126  6283 leader_election.cc:304] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7416dd332944470faa2d346b708a24ad; no voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:33.902819  6517 raft_consensus.cc:3060] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:33.903640  6546 consensus_queue.cc:1048] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.916942  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1545406360df433fa90dee76690705a0" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:33.917865  6395 raft_consensus.cc:3060] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.924196  6517 raft_consensus.cc:2749] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:33.928117  6467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1545406360df433fa90dee76690705a0" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:33.941926  6319 leader_election.cc:290] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Requested vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:33.944830  6545 consensus_queue.cc:1048] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.949801  6467 raft_consensus.cc:3055] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:33.953397  6467 raft_consensus.cc:740] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:33.955153  6467 consensus_queue.cc:260] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.956216  6554 consensus_queue.cc:1048] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.956653  6467 raft_consensus.cc:3060] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.962829  6395 raft_consensus.cc:2468] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:33.964480  6545 consensus_queue.cc:1048] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:33.974099  6516 consensus_queue.cc:1048] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:33.982686  6284 leader_election.cc:304] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: 
I20260709 12:21:33.983763  6517 raft_consensus.cc:2804] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:33.984275  6517 raft_consensus.cc:697] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Becoming Leader. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:33.985169  6517 consensus_queue.cc:237] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:33.994572  6545 raft_consensus.cc:993] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9: : Instructing follower 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:33.995249  6545 raft_consensus.cc:1081] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Signalling peer 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:34.001286  6317 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0"
dest_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:53138
I20260709 12:21:34.002074  6317 raft_consensus.cc:493] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:34.002494  6317 raft_consensus.cc:3060] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.007812  6545 consensus_queue.cc:1048] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:34.010834  6467 raft_consensus.cc:2468] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:34.028748  6317 raft_consensus.cc:515] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:34.031561  6317 leader_election.cc:290] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Requested vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:34.033090  6542 consensus_queue.cc:1048] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:34.033298  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:34.034271  6468 raft_consensus.cc:3060] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.035018  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:34.035740  6395 raft_consensus.cc:3055] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:34.036104  6395 raft_consensus.cc:740] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:34.041462  6542 consensus_queue.cc:1048] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:34.043720  6516 consensus_queue.cc:1048] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:34.044725  6468 raft_consensus.cc:2468] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:34.045699  6395 consensus_queue.cc:260] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:34.046363  6283 leader_election.cc:304] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:34.047338  6395 raft_consensus.cc:3060] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.047648  6517 raft_consensus.cc:2804] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:34.050810  6217 catalog_manager.cc:5697] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad reported cstate change: term changed from 1 to 2, leader changed from ec26b183414a47ca8e7e3d185016670d (127.26.123.3) to 7416dd332944470faa2d346b708a24ad (127.26.123.1). New cstate: current_term: 2 leader_uuid: "7416dd332944470faa2d346b708a24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:34.057610  6517 raft_consensus.cc:697] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Becoming Leader. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:34.059101  6517 consensus_queue.cc:237] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:34.066939  6395 raft_consensus.cc:2468] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:34.067481  6584 raft_consensus.cc:993] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9: : Instructing follower 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:34.068084  6545 raft_consensus.cc:1081] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Signalling peer 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:34.070927  6317 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041"
dest_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:53138
I20260709 12:21:34.071717  6317 raft_consensus.cc:493] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:34.072197  6317 raft_consensus.cc:3060] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.076638  6545 consensus_queue.cc:1048] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:34.087298  6215 catalog_manager.cc:5697] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad reported cstate change: term changed from 1 to 2, leader changed from b330bead86d943e8bf0a329597ac24f9 (127.26.123.2) to 7416dd332944470faa2d346b708a24ad (127.26.123.1). New cstate: current_term: 2 leader_uuid: "7416dd332944470faa2d346b708a24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:34.103801  6317 raft_consensus.cc:515] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:34.107674  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:34.108477  6469 raft_consensus.cc:3060] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.114374  6317 leader_election.cc:290] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Requested vote from peers ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:34.121444  6469 raft_consensus.cc:2468] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:34.122298  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d120b13ba1044b03a9ed88a94a704041" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:34.123170  6394 raft_consensus.cc:3055] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:34.123548  6394 raft_consensus.cc:740] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:34.123594  6283 leader_election.cc:304] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:34.124660  6394 consensus_queue.cc:260] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:34.125056  6592 raft_consensus.cc:2804] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:34.125914  6592 raft_consensus.cc:697] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Becoming Leader. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:34.127045  6592 consensus_queue.cc:237] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } }
I20260709 12:21:34.136168  6394 raft_consensus.cc:3060] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.139278  6218 catalog_manager.cc:5697] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad reported cstate change: term changed from 1 to 2, leader changed from b330bead86d943e8bf0a329597ac24f9 (127.26.123.2) to 7416dd332944470faa2d346b708a24ad (127.26.123.1). New cstate: current_term: 2 leader_uuid: "7416dd332944470faa2d346b708a24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:34.155673  6394 raft_consensus.cc:2468] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:34.523679  6468 raft_consensus.cc:1275] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.525017  6594 consensus_queue.cc:1048] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:34.533541  6394 raft_consensus.cc:1275] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.535382  6517 consensus_queue.cc:1048] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:34.545873  6394 raft_consensus.cc:1275] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.547518  6517 consensus_queue.cc:1048] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:34.559522  6468 raft_consensus.cc:1275] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.561604  6592 consensus_queue.cc:1048] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:34.645054  6394 raft_consensus.cc:1275] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.646553  6592 consensus_queue.cc:1048] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:34.655534  6468 raft_consensus.cc:1275] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.657620  6517 consensus_queue.cc:1048] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:36.743466  6215 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39098:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:36.745929  6215 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:36.793128  6304 tablet_service.cc:1511] Processing CreateTablet for tablet e3c293ac71de4d9fa1ec15d9048d47b9 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:36.794349  6380 tablet_service.cc:1511] Processing CreateTablet for tablet e3c293ac71de4d9fa1ec15d9048d47b9 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:36.794461  6304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3c293ac71de4d9fa1ec15d9048d47b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.795802  6380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3c293ac71de4d9fa1ec15d9048d47b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.795965  6455 tablet_service.cc:1511] Processing CreateTablet for tablet e3c293ac71de4d9fa1ec15d9048d47b9 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:36.797278  6455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3c293ac71de4d9fa1ec15d9048d47b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.799978  6305 tablet_service.cc:1511] Processing CreateTablet for tablet 8ba9fa0a822146d68a7da0de412de790 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:36.801334  6305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba9fa0a822146d68a7da0de412de790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.802556  6452 tablet_service.cc:1511] Processing CreateTablet for tablet 8ba9fa0a822146d68a7da0de412de790 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:36.803898  6452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba9fa0a822146d68a7da0de412de790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.807569  6383 tablet_service.cc:1511] Processing CreateTablet for tablet 8ba9fa0a822146d68a7da0de412de790 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:36.809033  6383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba9fa0a822146d68a7da0de412de790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.810838  6459 tablet_service.cc:1511] Processing CreateTablet for tablet 62c124ba608e47d7b3c5f576572ffad5 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:36.812242  6459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c124ba608e47d7b3c5f576572ffad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.815290  6303 tablet_service.cc:1511] Processing CreateTablet for tablet 62c124ba608e47d7b3c5f576572ffad5 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:36.816668  6303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c124ba608e47d7b3c5f576572ffad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.816459  6307 tablet_service.cc:1511] Processing CreateTablet for tablet 28a630c71ca84039b721dcb6343727a9 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:36.817791  6307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a630c71ca84039b721dcb6343727a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.819528  6379 tablet_service.cc:1511] Processing CreateTablet for tablet 62c124ba608e47d7b3c5f576572ffad5 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:36.820458  6306 tablet_service.cc:1511] Processing CreateTablet for tablet c800840c300c4fe5a61e9fcb8bc8dc39 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:36.821787  6306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c800840c300c4fe5a61e9fcb8bc8dc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.822089  6377 tablet_service.cc:1511] Processing CreateTablet for tablet 28a630c71ca84039b721dcb6343727a9 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:36.824707  6457 tablet_service.cc:1511] Processing CreateTablet for tablet 28a630c71ca84039b721dcb6343727a9 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:36.825107  6458 tablet_service.cc:1511] Processing CreateTablet for tablet c800840c300c4fe5a61e9fcb8bc8dc39 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:36.825479  6381 tablet_service.cc:1511] Processing CreateTablet for tablet c800840c300c4fe5a61e9fcb8bc8dc39 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:36.826043  6457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a630c71ca84039b721dcb6343727a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.826751  6458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c800840c300c4fe5a61e9fcb8bc8dc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.830649  6309 tablet_service.cc:1511] Processing CreateTablet for tablet ff02e69900044d57aa73317dc4f6adca (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:36.832114  6309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff02e69900044d57aa73317dc4f6adca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.833045  6456 tablet_service.cc:1511] Processing CreateTablet for tablet ff02e69900044d57aa73317dc4f6adca (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:36.834345  6456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff02e69900044d57aa73317dc4f6adca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.839102  6454 tablet_service.cc:1511] Processing CreateTablet for tablet c1ea1c9340c5421bb9df628838e15b23 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:36.840453  6454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1ea1c9340c5421bb9df628838e15b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.840539  6385 tablet_service.cc:1511] Processing CreateTablet for tablet ff02e69900044d57aa73317dc4f6adca (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:36.842715  6378 tablet_service.cc:1511] Processing CreateTablet for tablet c1ea1c9340c5421bb9df628838e15b23 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:36.844615  6308 tablet_service.cc:1511] Processing CreateTablet for tablet b09d133cc6394c56967281782f9821f5 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:36.846052  6308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b09d133cc6394c56967281782f9821f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.846293  6453 tablet_service.cc:1511] Processing CreateTablet for tablet b09d133cc6394c56967281782f9821f5 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:36.847551  6453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b09d133cc6394c56967281782f9821f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.852082  6382 tablet_service.cc:1511] Processing CreateTablet for tablet b09d133cc6394c56967281782f9821f5 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:36.853135  6302 tablet_service.cc:1511] Processing CreateTablet for tablet 1ed251d0273a4101bb69a3dc7e9541b8 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:36.844736  6310 tablet_service.cc:1511] Processing CreateTablet for tablet c1ea1c9340c5421bb9df628838e15b23 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:36.854502  6302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ed251d0273a4101bb69a3dc7e9541b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.820835  6379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c124ba608e47d7b3c5f576572ffad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.855335  6378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1ea1c9340c5421bb9df628838e15b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.857026  6385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff02e69900044d57aa73317dc4f6adca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.854305  6384 tablet_service.cc:1511] Processing CreateTablet for tablet 1ed251d0273a4101bb69a3dc7e9541b8 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:36.858568  6384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ed251d0273a4101bb69a3dc7e9541b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.859139  6382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b09d133cc6394c56967281782f9821f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.859823  6310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1ea1c9340c5421bb9df628838e15b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.861820  6377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28a630c71ca84039b721dcb6343727a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.866035  6381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c800840c300c4fe5a61e9fcb8bc8dc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.877522  6451 tablet_service.cc:1511] Processing CreateTablet for tablet 1ed251d0273a4101bb69a3dc7e9541b8 (DEFAULT_TABLE table=soft_deleted_table [id=77771ffead724122be126c5e184c825e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:36.879719  6451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ed251d0273a4101bb69a3dc7e9541b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:36.905599  6614 tablet_bootstrap.cc:492] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:36.933712  6614 tablet_bootstrap.cc:654] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:36.947237  6615 tablet_bootstrap.cc:492] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:36.987534  6615 tablet_bootstrap.cc:654] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.004706  6614 tablet_bootstrap.cc:492] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:37.005507  6614 ts_tablet_manager.cc:1403] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.016s
I20260709 12:21:37.009205  6614 raft_consensus.cc:359] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.010118  6614 raft_consensus.cc:385] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.010430  6614 raft_consensus.cc:740] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.011358  6614 consensus_queue.cc:260] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
W20260709 12:21:37.013559  6266 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:37.016986  6614 ts_tablet_manager.cc:1434] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:21:37.018571  6614 tablet_bootstrap.cc:492] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:37.026592  6614 tablet_bootstrap.cc:654] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.048550  6618 tablet_bootstrap.cc:492] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:37.052969  6615 tablet_bootstrap.cc:492] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:37.053673  6615 ts_tablet_manager.cc:1403] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.107s	user 0.042s	sys 0.017s
I20260709 12:21:37.054036  6614 tablet_bootstrap.cc:492] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:37.054711  6614 ts_tablet_manager.cc:1403] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.015s
I20260709 12:21:37.055552  6618 tablet_bootstrap.cc:654] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.057200  6615 raft_consensus.cc:359] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.058091  6615 raft_consensus.cc:385] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.057911  6614 raft_consensus.cc:359] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.058445  6615 raft_consensus.cc:740] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.058753  6614 raft_consensus.cc:385] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.059078  6614 raft_consensus.cc:740] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.059356  6615 consensus_queue.cc:260] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.059900  6614 consensus_queue.cc:260] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.062067  6615 ts_tablet_manager.cc:1434] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:37.062449  6614 ts_tablet_manager.cc:1434] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:37.063385  6615 tablet_bootstrap.cc:492] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:37.063480  6614 tablet_bootstrap.cc:492] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:37.071658  6614 tablet_bootstrap.cc:654] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.071659  6615 tablet_bootstrap.cc:654] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.077450  6618 tablet_bootstrap.cc:492] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:37.078105  6618 ts_tablet_manager.cc:1403] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.000s
I20260709 12:21:37.080590  6618 raft_consensus.cc:359] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.081214  6618 raft_consensus.cc:385] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.081465  6618 raft_consensus.cc:740] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.082154  6618 consensus_queue.cc:260] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.084805  6614 tablet_bootstrap.cc:492] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:37.085453  6614 ts_tablet_manager.cc:1403] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.009s
I20260709 12:21:37.085472  6618 ts_tablet_manager.cc:1434] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 12:21:37.086222  6615 tablet_bootstrap.cc:492] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:37.086792  6615 ts_tablet_manager.cc:1403] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 12:21:37.087082  6618 tablet_bootstrap.cc:492] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:37.089030  6614 raft_consensus.cc:359] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.089856  6614 raft_consensus.cc:385] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.090207  6614 raft_consensus.cc:740] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.089994  6615 raft_consensus.cc:359] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.090782  6615 raft_consensus.cc:385] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.091105  6615 raft_consensus.cc:740] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.091104  6614 consensus_queue.cc:260] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.092259  6615 consensus_queue.cc:260] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.093094  6597 raft_consensus.cc:493] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.093828  6614 ts_tablet_manager.cc:1434] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:37.093631  6597 raft_consensus.cc:515] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.094826  6615 ts_tablet_manager.cc:1434] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:37.095093  6614 tablet_bootstrap.cc:492] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:37.095957  6615 tablet_bootstrap.cc:492] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:37.096383  6618 tablet_bootstrap.cc:654] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.099131  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:37.099941  6469 raft_consensus.cc:2468] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 0.
I20260709 12:21:37.101105  6597 leader_election.cc:290] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.101812  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:37.103765  6615 tablet_bootstrap.cc:654] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.103931  6614 tablet_bootstrap.cc:654] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.103554  6283 leader_election.cc:304] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
W20260709 12:21:37.105726  6284 leader_election.cc:343] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.133471  6597 raft_consensus.cc:2804] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.134248  6597 raft_consensus.cc:493] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.134738  6597 raft_consensus.cc:3060] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.159909  6597 raft_consensus.cc:515] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.165513  6618 tablet_bootstrap.cc:492] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:37.165948  6614 tablet_bootstrap.cc:492] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:37.166276  6618 ts_tablet_manager.cc:1403] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.079s	user 0.036s	sys 0.038s
I20260709 12:21:37.166560  6614 ts_tablet_manager.cc:1403] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.072s	user 0.042s	sys 0.027s
I20260709 12:21:37.166132  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
W20260709 12:21:37.168398  6284 leader_election.cc:343] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.166472  6615 tablet_bootstrap.cc:492] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:37.168829  6603 raft_consensus.cc:493] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.169474  6615 ts_tablet_manager.cc:1403] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.074s	user 0.019s	sys 0.037s
I20260709 12:21:37.169433  6603 raft_consensus.cc:515] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.170328  6614 raft_consensus.cc:359] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.171119  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:37.171293  6597 leader_election.cc:290] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 election: Requested vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.171821  6469 raft_consensus.cc:3060] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.171948  6603 leader_election.cc:290] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.173609  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.174918  6618 raft_consensus.cc:359] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
W20260709 12:21:37.175381  6433 leader_election.cc:343] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.175822  6618 raft_consensus.cc:385] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.176219  6618 raft_consensus.cc:740] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.176378  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:37.177188  6618 consensus_queue.cc:260] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.173121  6615 raft_consensus.cc:359] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.178278  6615 raft_consensus.cc:385] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.178598  6615 raft_consensus.cc:740] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.179431  6615 consensus_queue.cc:260] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.171114  6614 raft_consensus.cc:385] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.180251  6614 raft_consensus.cc:740] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.180483  6394 raft_consensus.cc:2468] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:37.181252  6614 consensus_queue.cc:260] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.183702  6615 ts_tablet_manager.cc:1434] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:21:37.183820  6614 ts_tablet_manager.cc:1434] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:21:37.185207  6615 tablet_bootstrap.cc:492] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:37.185213  6614 tablet_bootstrap.cc:492] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:37.180150  6618 ts_tablet_manager.cc:1434] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:21:37.193017  6615 tablet_bootstrap.cc:654] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.192986  6614 tablet_bootstrap.cc:654] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.190974  6618 tablet_bootstrap.cc:492] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:37.202178  6433 leader_election.cc:304] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad
I20260709 12:21:37.204084  6469 raft_consensus.cc:2468] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 1.
I20260709 12:21:37.206041  6615 tablet_bootstrap.cc:492] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:37.206041  6614 tablet_bootstrap.cc:492] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:37.206851  6614 ts_tablet_manager.cc:1403] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 12:21:37.206851  6615 ts_tablet_manager.cc:1403] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.000s
I20260709 12:21:37.206845  6283 leader_election.cc:304] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:37.208662  6597 raft_consensus.cc:2804] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.210592  6614 raft_consensus.cc:359] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.211493  6614 raft_consensus.cc:385] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.211791  6614 raft_consensus.cc:740] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.212628  6614 consensus_queue.cc:260] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.221550  6632 raft_consensus.cc:493] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.222200  6632 raft_consensus.cc:515] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.224582  6615 raft_consensus.cc:359] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.226452  6614 ts_tablet_manager.cc:1434] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 12:21:37.224627  6632 leader_election.cc:290] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.227667  6615 raft_consensus.cc:385] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.228374  6615 raft_consensus.cc:740] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.228721  6614 tablet_bootstrap.cc:492] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:37.229691  6615 consensus_queue.cc:260] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.229959  6618 tablet_bootstrap.cc:654] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.233386  6615 ts_tablet_manager.cc:1434] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:21:37.233969  6632 raft_consensus.cc:2804] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.234424  6632 raft_consensus.cc:493] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.234678  6615 tablet_bootstrap.cc:492] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:37.234784  6632 raft_consensus.cc:3060] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.250116  6546 raft_consensus.cc:493] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.250881  6546 raft_consensus.cc:515] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.241353  6614 tablet_bootstrap.cc:654] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.255663  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.258427  6618 tablet_bootstrap.cc:492] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:37.259061  6618 ts_tablet_manager.cc:1403] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.068s	user 0.009s	sys 0.007s
I20260709 12:21:37.260151  6632 raft_consensus.cc:515] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
W20260709 12:21:37.262293  6359 leader_election.cc:343] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.264992  6632 leader_election.cc:290] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.264346  6618 raft_consensus.cc:359] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.265506  6618 raft_consensus.cc:385] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.265868  6618 raft_consensus.cc:740] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.268126  6615 tablet_bootstrap.cc:654] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.272122  6546 leader_election.cc:290] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.272269  6618 consensus_queue.cc:260] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.275691  6618 ts_tablet_manager.cc:1434] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 12:21:37.276978  6618 tablet_bootstrap.cc:492] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:37.285419  6615 tablet_bootstrap.cc:492] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:37.285446  6614 tablet_bootstrap.cc:492] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:37.286281  6615 ts_tablet_manager.cc:1403] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.005s
I20260709 12:21:37.289202  6614 ts_tablet_manager.cc:1403] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.061s	user 0.019s	sys 0.014s
I20260709 12:21:37.297125  6614 raft_consensus.cc:359] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
W20260709 12:21:37.300660  6433 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.123.2:45611, user_credentials={real_user=slave}} blocked reactor thread for 98642us
I20260709 12:21:37.303256  6618 tablet_bootstrap.cc:654] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.299839  6614 raft_consensus.cc:385] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.303678  6614 raft_consensus.cc:740] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.303399  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba9fa0a822146d68a7da0de412de790" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.304340  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba9fa0a822146d68a7da0de412de790" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:37.304448  6320 raft_consensus.cc:2468] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:37.304577  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:37.305251  6394 raft_consensus.cc:2468] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:37.302371  6615 raft_consensus.cc:359] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.306274  6615 raft_consensus.cc:385] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.306635  6615 raft_consensus.cc:740] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.307650  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:37.307533  6615 consensus_queue.cc:260] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.304574  6614 consensus_queue.cc:260] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.308450  6394 raft_consensus.cc:3060] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.298738  6597 raft_consensus.cc:697] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 1 LEADER]: Becoming Leader. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:37.309831  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:37.310678  6615 ts_tablet_manager.cc:1434] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:21:37.310364  6597 consensus_queue.cc:237] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.311928  6615 tablet_bootstrap.cc:492] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
W20260709 12:21:37.313597  6358 leader_election.cc:343] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.314234  6358 leader_election.cc:304] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9; no voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:37.315768  6433 leader_election.cc:304] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:37.319058  6630 raft_consensus.cc:2804] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.319525  6630 raft_consensus.cc:493] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.317800  6614 ts_tablet_manager.cc:1434] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 12:21:37.319819  6626 raft_consensus.cc:2749] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.320108  6394 raft_consensus.cc:2468] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
W20260709 12:21:37.320806  6433 leader_election.cc:343] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.319897  6630 raft_consensus.cc:3060] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.322275  6614 tablet_bootstrap.cc:492] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:37.327209  6618 tablet_bootstrap.cc:492] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:37.327961  6618 ts_tablet_manager.cc:1403] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.001s
I20260709 12:21:37.330920  6433 leader_election.cc:304] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad
I20260709 12:21:37.332088  6632 raft_consensus.cc:493] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.332697  6632 raft_consensus.cc:515] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.335114  6632 leader_election.cc:290] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.335805  6632 raft_consensus.cc:2804] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.335979  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c800840c300c4fe5a61e9fcb8bc8dc39" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.336736  6632 raft_consensus.cc:697] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:37.337949  6632 consensus_queue.cc:237] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
W20260709 12:21:37.338915  6433 leader_election.cc:343] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.340000  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c800840c300c4fe5a61e9fcb8bc8dc39" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
W20260709 12:21:37.341841  6433 leader_election.cc:343] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.342370  6433 leader_election.cc:304] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:37.343622  6603 raft_consensus.cc:2749] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.344041  6614 tablet_bootstrap.cc:654] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.348100  6618 raft_consensus.cc:359] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.349007  6618 raft_consensus.cc:385] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.349334  6618 raft_consensus.cc:740] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.350195  6618 consensus_queue.cc:260] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.354614  6618 ts_tablet_manager.cc:1434] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.004s
I20260709 12:21:37.355857  6618 tablet_bootstrap.cc:492] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:37.362046  6614 tablet_bootstrap.cc:492] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:37.364358  6618 tablet_bootstrap.cc:654] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.365190  6614 ts_tablet_manager.cc:1403] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.009s
I20260709 12:21:37.363749  6630 raft_consensus.cc:515] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.372663  6615 tablet_bootstrap.cc:654] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.375483  6630 leader_election.cc:290] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.378383  6614 raft_consensus.cc:359] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.380767  6614 raft_consensus.cc:385] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.375782  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba9fa0a822146d68a7da0de412de790" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:37.381767  6614 raft_consensus.cc:740] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.381398  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba9fa0a822146d68a7da0de412de790" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:37.382882  6394 raft_consensus.cc:3060] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.381965  6320 raft_consensus.cc:3060] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.383814  6614 consensus_queue.cc:260] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.390407  6614 ts_tablet_manager.cc:1434] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 12:21:37.391538  6630 raft_consensus.cc:493] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.392247  6630 raft_consensus.cc:515] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.393527  6394 raft_consensus.cc:2468] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:37.394131  6614 tablet_bootstrap.cc:492] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:37.396087  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1ea1c9340c5421bb9df628838e15b23" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.396885  6317 raft_consensus.cc:2468] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:37.398674  6433 leader_election.cc:304] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:37.399689  6433 leader_election.cc:304] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:37.400529  6603 raft_consensus.cc:2804] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.400943  6603 raft_consensus.cc:493] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.401294  6603 raft_consensus.cc:3060] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.403546  6627 raft_consensus.cc:2804] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.405169  6630 leader_election.cc:290] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.410513  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1ea1c9340c5421bb9df628838e15b23" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
W20260709 12:21:37.413023  6433 leader_election.cc:343] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.417179  6215 catalog_manager.cc:5697] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 0 to 1, leader changed from <none> to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.423017  6614 tablet_bootstrap.cc:654] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.423545  6615 tablet_bootstrap.cc:492] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:37.424173  6615 ts_tablet_manager.cc:1403] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.113s	user 0.015s	sys 0.006s
I20260709 12:21:37.425457  6618 tablet_bootstrap.cc:492] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:37.426259  6618 ts_tablet_manager.cc:1403] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.071s	user 0.016s	sys 0.006s
I20260709 12:21:37.426613  6597 raft_consensus.cc:493] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.427332  6597 raft_consensus.cc:515] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.430457  6597 leader_election.cc:290] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.431811  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a630c71ca84039b721dcb6343727a9" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:37.433198  6627 raft_consensus.cc:697] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
W20260709 12:21:37.433636  6283 leader_election.cc:343] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.434285  6627 consensus_queue.cc:237] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.431157  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a630c71ca84039b721dcb6343727a9" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:37.435842  6394 raft_consensus.cc:2468] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 0.
I20260709 12:21:37.437175  6284 leader_election.cc:304] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:37.438194  6597 raft_consensus.cc:2804] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.438674  6597 raft_consensus.cc:493] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.439257  6597 raft_consensus.cc:3060] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.437000  6618 raft_consensus.cc:359] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.450023  6618 raft_consensus.cc:385] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.450050  6597 raft_consensus.cc:515] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.451561  6320 raft_consensus.cc:2468] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:37.435338  6615 raft_consensus.cc:359] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.452950  6615 raft_consensus.cc:385] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.453299  6615 raft_consensus.cc:740] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.454232  6615 consensus_queue.cc:260] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.454733  6597 leader_election.cc:290] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 election: Requested vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.450481  6618 raft_consensus.cc:740] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.456074  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a630c71ca84039b721dcb6343727a9" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:37.456693  6394 raft_consensus.cc:3060] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.457047  6618 consensus_queue.cc:260] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.457643  6615 ts_tablet_manager.cc:1434] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.003s
I20260709 12:21:37.459725  6618 ts_tablet_manager.cc:1434] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.003s
I20260709 12:21:37.460886  6618 tablet_bootstrap.cc:492] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:37.462684  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a630c71ca84039b721dcb6343727a9" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:37.466475  6615 tablet_bootstrap.cc:492] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:37.445335  6603 raft_consensus.cc:515] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.469139  6618 tablet_bootstrap.cc:654] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:37.469830  6283 leader_election.cc:343] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:37.470407  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1ea1c9340c5421bb9df628838e15b23" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:37.471246  6320 raft_consensus.cc:3060] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.475042  6615 tablet_bootstrap.cc:654] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.475901  6394 raft_consensus.cc:2468] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 1.
I20260709 12:21:37.477531  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1ea1c9340c5421bb9df628838e15b23" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:37.478364  6395 raft_consensus.cc:3060] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.477545  6284 leader_election.cc:304] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:37.483440  6597 raft_consensus.cc:2804] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.484112  6597 raft_consensus.cc:697] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 1 LEADER]: Becoming Leader. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:37.483623  6216 catalog_manager.cc:5697] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 7416dd332944470faa2d346b708a24ad (127.26.123.1). New cstate: current_term: 1 leader_uuid: "7416dd332944470faa2d346b708a24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.485049  6597 consensus_queue.cc:237] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.477451  6603 leader_election.cc:290] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.503847  6395 raft_consensus.cc:2468] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:37.509104  6614 tablet_bootstrap.cc:492] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:37.509299  6433 leader_election.cc:304] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:37.510119  6614 ts_tablet_manager.cc:1403] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.117s	user 0.020s	sys 0.036s
I20260709 12:21:37.510906  6603 raft_consensus.cc:2804] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.511875  6603 raft_consensus.cc:697] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:37.512950  6603 consensus_queue.cc:237] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.513760  6614 raft_consensus.cc:359] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.514844  6614 raft_consensus.cc:385] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.515193  6614 raft_consensus.cc:740] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.516197  6614 consensus_queue.cc:260] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.518409  6614 ts_tablet_manager.cc:1434] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:21:37.520542  6614 tablet_bootstrap.cc:492] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d: Bootstrap starting.
I20260709 12:21:37.522738  6320 raft_consensus.cc:2468] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:37.541054  6614 tablet_bootstrap.cc:654] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.548830  6218 catalog_manager.cc:5697] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 0 to 1, leader changed from <none> to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.556475  6618 tablet_bootstrap.cc:492] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:37.556475  6614 tablet_bootstrap.cc:492] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d: No bootstrap required, opened a new log
I20260709 12:21:37.557667  6614 ts_tablet_manager.cc:1403] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.006s
I20260709 12:21:37.557729  6618 ts_tablet_manager.cc:1403] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.097s	user 0.037s	sys 0.042s
I20260709 12:21:37.562754  6614 raft_consensus.cc:359] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.563493  6618 raft_consensus.cc:359] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.564519  6618 raft_consensus.cc:385] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.564824  6618 raft_consensus.cc:740] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.563606  6614 raft_consensus.cc:385] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.565296  6614 raft_consensus.cc:740] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.566102  6618 consensus_queue.cc:260] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.566145  6614 consensus_queue.cc:260] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.566695  6615 tablet_bootstrap.cc:492] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:37.569190  6618 ts_tablet_manager.cc:1434] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:21:37.569212  6615 ts_tablet_manager.cc:1403] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.103s	user 0.009s	sys 0.013s
I20260709 12:21:37.570024  6614 ts_tablet_manager.cc:1434] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:21:37.570531  6618 tablet_bootstrap.cc:492] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:37.573038  6216 catalog_manager.cc:5697] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 7416dd332944470faa2d346b708a24ad (127.26.123.1). New cstate: current_term: 1 leader_uuid: "7416dd332944470faa2d346b708a24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.573148  6615 raft_consensus.cc:359] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.575802  6615 raft_consensus.cc:385] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.576215  6615 raft_consensus.cc:740] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.577157  6615 consensus_queue.cc:260] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.577639  6618 tablet_bootstrap.cc:654] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.580188  6615 ts_tablet_manager.cc:1434] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:21:37.582654  6615 tablet_bootstrap.cc:492] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:37.584599  6618 tablet_bootstrap.cc:492] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:37.585243  6618 ts_tablet_manager.cc:1403] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.005s
I20260709 12:21:37.588840  6618 raft_consensus.cc:359] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.589789  6618 raft_consensus.cc:385] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.589689  6218 catalog_manager.cc:5697] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 0 to 1, leader changed from <none> to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.591341  6615 tablet_bootstrap.cc:654] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.591442  6618 raft_consensus.cc:740] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.598335  6618 consensus_queue.cc:260] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.599928  6615 tablet_bootstrap.cc:492] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:37.600749  6615 ts_tablet_manager.cc:1403] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 12:21:37.600943  6618 ts_tablet_manager.cc:1434] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:21:37.602324  6618 tablet_bootstrap.cc:492] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:37.604498  6615 raft_consensus.cc:359] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.605357  6615 raft_consensus.cc:385] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.605360  6603 raft_consensus.cc:493] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.605772  6615 raft_consensus.cc:740] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.606647  6603 raft_consensus.cc:515] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.607034  6615 consensus_queue.cc:260] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.609406  6615 ts_tablet_manager.cc:1434] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:21:37.610389  6615 tablet_bootstrap.cc:492] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9: Bootstrap starting.
I20260709 12:21:37.610556  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a630c71ca84039b721dcb6343727a9" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.609944  6618 tablet_bootstrap.cc:654] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.615110  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28a630c71ca84039b721dcb6343727a9" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:37.616333  6615 tablet_bootstrap.cc:654] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.616653  6395 raft_consensus.cc:2393] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec26b183414a47ca8e7e3d185016670d in current term 1: Already voted for candidate 7416dd332944470faa2d346b708a24ad in this term.
I20260709 12:21:37.618525  6433 leader_election.cc:304] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec26b183414a47ca8e7e3d185016670d; no voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:37.619555  6627 raft_consensus.cc:3060] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.627633  6603 leader_election.cc:290] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.636646  6627 raft_consensus.cc:2749] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.644981  6626 raft_consensus.cc:493] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.645716  6626 raft_consensus.cc:515] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.646220  6615 tablet_bootstrap.cc:492] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9: No bootstrap required, opened a new log
I20260709 12:21:37.646847  6615 ts_tablet_manager.cc:1403] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.016s
I20260709 12:21:37.647465  6618 tablet_bootstrap.cc:492] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:37.648021  6618 ts_tablet_manager.cc:1403] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.018s
I20260709 12:21:37.649051  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.649585  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:37.649924  6615 raft_consensus.cc:359] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.650494  6469 raft_consensus.cc:2393] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b330bead86d943e8bf0a329597ac24f9 in current term 1: Already voted for candidate 7416dd332944470faa2d346b708a24ad in this term.
I20260709 12:21:37.650705  6615 raft_consensus.cc:385] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.651147  6615 raft_consensus.cc:740] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.651154  6618 raft_consensus.cc:359] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.651968  6618 raft_consensus.cc:385] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.651880  6358 leader_election.cc:304] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9; no voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:37.652563  6618 raft_consensus.cc:740] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.651968  6615 consensus_queue.cc:260] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.653426  6546 raft_consensus.cc:3060] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.653367  6618 consensus_queue.cc:260] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.656014  6618 ts_tablet_manager.cc:1434] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:37.657199  6618 tablet_bootstrap.cc:492] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad: Bootstrap starting.
I20260709 12:21:37.648097  6626 leader_election.cc:290] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.664287  6618 tablet_bootstrap.cc:654] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:37.672820  6618 tablet_bootstrap.cc:492] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad: No bootstrap required, opened a new log
I20260709 12:21:37.673466  6618 ts_tablet_manager.cc:1403] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 12:21:37.676662  6618 raft_consensus.cc:359] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.677373  6618 raft_consensus.cc:385] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:37.677649  6618 raft_consensus.cc:740] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Initialized, Role: FOLLOWER
I20260709 12:21:37.678015  6546 raft_consensus.cc:2749] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:37.678596  6618 consensus_queue.cc:260] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.679708  6615 ts_tablet_manager.cc:1434] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9: Time spent starting tablet: real 0.033s	user 0.012s	sys 0.019s
I20260709 12:21:37.680707  6618 ts_tablet_manager.cc:1434] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:21:37.751034  6320 raft_consensus.cc:3060] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.760013  6627 consensus_queue.cc:1048] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:37.772928  6603 consensus_queue.cc:1048] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.780771  6660 raft_consensus.cc:493] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.781419  6660 raft_consensus.cc:515] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.783625  6603 raft_consensus.cc:493] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.785531  6603 raft_consensus.cc:515] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.786660  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c124ba608e47d7b3c5f576572ffad5" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:37.787523  6469 raft_consensus.cc:2468] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:37.784631  6660 leader_election.cc:290] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.788163  6603 leader_election.cc:290] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.788647  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c124ba608e47d7b3c5f576572ffad5" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.789258  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff02e69900044d57aa73317dc4f6adca" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.789546  6320 raft_consensus.cc:2468] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:37.790059  6317 raft_consensus.cc:2468] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:37.789324  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff02e69900044d57aa73317dc4f6adca" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:37.791177  6359 leader_election.cc:304] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: 
I20260709 12:21:37.791509  6433 leader_election.cc:304] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:37.792629  6660 raft_consensus.cc:2804] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.793141  6660 raft_consensus.cc:493] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.793529  6660 raft_consensus.cc:3060] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.792627  6394 raft_consensus.cc:2468] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:37.796196  6597 consensus_queue.cc:1048] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.805042  6660 raft_consensus.cc:515] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.807811  6660 leader_election.cc:290] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.792934  6603 raft_consensus.cc:2804] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.808758  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c124ba608e47d7b3c5f576572ffad5" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:37.809168  6603 raft_consensus.cc:493] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.808957  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c124ba608e47d7b3c5f576572ffad5" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:37.809723  6317 raft_consensus.cc:3060] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.809782  6603 raft_consensus.cc:3060] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.809938  6468 raft_consensus.cc:3060] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.819300  6317 raft_consensus.cc:2468] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:37.819802  6603 raft_consensus.cc:515] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.820643  6359 leader_election.cc:304] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: 
I20260709 12:21:37.821602  6660 raft_consensus.cc:2804] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.819334  6468 raft_consensus.cc:2468] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:37.822283  6660 raft_consensus.cc:697] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:37.823282  6660 consensus_queue.cc:237] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.828370  6603 leader_election.cc:290] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.829241  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff02e69900044d57aa73317dc4f6adca" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:37.829471  6394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff02e69900044d57aa73317dc4f6adca" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:37.830087  6320 raft_consensus.cc:3060] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.830245  6394 raft_consensus.cc:3060] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.840605  6320 raft_consensus.cc:2468] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:37.840744  6218 catalog_manager.cc:5697] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.842056  6433 leader_election.cc:304] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:37.840616  6636 consensus_queue.cc:1048] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:37.843034  6603 raft_consensus.cc:2804] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.843550  6603 raft_consensus.cc:697] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:37.844450  6603 consensus_queue.cc:237] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.845961  6394 raft_consensus.cc:2468] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:37.862355  6218 catalog_manager.cc:5697] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 0 to 1, leader changed from <none> to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.863873  6603 consensus_queue.cc:1048] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.872916  6546 raft_consensus.cc:493] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.873464  6546 raft_consensus.cc:515] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.875754  6546 leader_election.cc:290] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.877768  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c800840c300c4fe5a61e9fcb8bc8dc39" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.878691  6320 raft_consensus.cc:2468] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:37.879993  6359 leader_election.cc:304] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: 
I20260709 12:21:37.877897  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c800840c300c4fe5a61e9fcb8bc8dc39" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d" is_pre_election: true
I20260709 12:21:37.881248  6468 raft_consensus.cc:2468] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 0.
I20260709 12:21:37.881784  6645 consensus_queue.cc:1048] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:37.882594  6546 raft_consensus.cc:2804] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.883105  6546 raft_consensus.cc:493] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.883514  6546 raft_consensus.cc:3060] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.891091  6603 raft_consensus.cc:493] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.891743  6603 raft_consensus.cc:515] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.892663  6546 raft_consensus.cc:515] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.894074  6603 leader_election.cc:290] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.895367  6546 leader_election.cc:290] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:37.896134  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c800840c300c4fe5a61e9fcb8bc8dc39" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:37.896797  6317 raft_consensus.cc:3060] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.897120  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c800840c300c4fe5a61e9fcb8bc8dc39" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:37.897636  6468 raft_consensus.cc:3060] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.899312  6393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9" is_pre_election: true
I20260709 12:21:37.900010  6393 raft_consensus.cc:2468] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:37.901391  6433 leader_election.cc:304] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:37.903641  6603 raft_consensus.cc:2804] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:37.904078  6603 raft_consensus.cc:493] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.904471  6603 raft_consensus.cc:3060] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.905364  6468 raft_consensus.cc:2468] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:37.906688  6358 leader_election.cc:304] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:37.906750  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad" is_pre_election: true
I20260709 12:21:37.907575  6320 raft_consensus.cc:2468] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 0.
I20260709 12:21:37.907639  6546 raft_consensus.cc:2804] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.908411  6546 raft_consensus.cc:697] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:37.909189  6317 raft_consensus.cc:2468] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 1.
I20260709 12:21:37.909312  6546 consensus_queue.cc:237] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.914987  6603 raft_consensus.cc:515] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.917354  6603 leader_election.cc:290] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:37.919075  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:37.919315  6393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:37.919876  6317 raft_consensus.cc:3060] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.919934  6218 catalog_manager.cc:5697] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:37.920006  6393 raft_consensus.cc:3060] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:37.927449  6393 raft_consensus.cc:2468] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:37.928457  6317 raft_consensus.cc:2468] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 1.
I20260709 12:21:37.928610  6433 leader_election.cc:304] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:37.930158  6603 raft_consensus.cc:2804] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:37.931226  6603 raft_consensus.cc:697] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:37.931998  6603 consensus_queue.cc:237] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:37.940855  6218 catalog_manager.cc:5697] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 0 to 1, leader changed from <none> to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:37.959223  6468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
mode: GRACEFUL
new_leader_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:45786
I20260709 12:21:37.959756  6468 raft_consensus.cc:606] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Received request to transfer leadership to TS 7416dd332944470faa2d346b708a24ad
I20260709 12:21:37.964563  6468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ba9fa0a822146d68a7da0de412de790"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
mode: GRACEFUL
new_leader_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:45786
I20260709 12:21:37.965080  6468 raft_consensus.cc:606] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Received request to transfer leadership to TS 7416dd332944470faa2d346b708a24ad
I20260709 12:21:37.969820  6393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62c124ba608e47d7b3c5f576572ffad5"
dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
mode: GRACEFUL
new_leader_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:42160
I20260709 12:21:37.970337  6393 raft_consensus.cc:606] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Received request to transfer leadership to TS 7416dd332944470faa2d346b708a24ad
I20260709 12:21:37.980159  6468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff02e69900044d57aa73317dc4f6adca"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
mode: GRACEFUL
new_leader_uuid: "b330bead86d943e8bf0a329597ac24f9"
 from {username='slave'} at 127.0.0.1:45786
I20260709 12:21:37.980748  6468 raft_consensus.cc:606] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Received request to transfer leadership to TS b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:37.986121  6468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1ea1c9340c5421bb9df628838e15b23"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
mode: GRACEFUL
new_leader_uuid: "b330bead86d943e8bf0a329597ac24f9"
 from {username='slave'} at 127.0.0.1:45786
I20260709 12:21:37.986675  6468 raft_consensus.cc:606] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Received request to transfer leadership to TS b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:37.991955  6468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b09d133cc6394c56967281782f9821f5"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
mode: GRACEFUL
new_leader_uuid: "b330bead86d943e8bf0a329597ac24f9"
 from {username='slave'} at 127.0.0.1:45786
I20260709 12:21:37.992554  6468 raft_consensus.cc:606] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Received request to transfer leadership to TS b330bead86d943e8bf0a329597ac24f9
I20260709 12:21:38.006770  6636 consensus_queue.cc:1048] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.008665  6317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8"
dest_uuid: "7416dd332944470faa2d346b708a24ad"
mode: GRACEFUL
new_leader_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:53142
I20260709 12:21:38.009462  6317 raft_consensus.cc:606] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 1 LEADER]: Received request to transfer leadership to TS ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:38.029253  6679 consensus_queue.cc:1048] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.103029  6603 consensus_queue.cc:1048] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.114708  6627 consensus_queue.cc:1048] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.122824  6603 raft_consensus.cc:993] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d: : Instructing follower b330bead86d943e8bf0a329597ac24f9 to start an election
I20260709 12:21:38.123394  6627 raft_consensus.cc:1081] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Signalling peer b330bead86d943e8bf0a329597ac24f9 to start an election
I20260709 12:21:38.125526  6393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1ea1c9340c5421bb9df628838e15b23"
dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
 from {username='slave'} at 127.0.0.1:42150
I20260709 12:21:38.126175  6393 raft_consensus.cc:493] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:38.126528  6393 raft_consensus.cc:3060] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.134727  6393 raft_consensus.cc:515] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.136893  6393 leader_election.cc:290] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 2 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:38.138635  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1ea1c9340c5421bb9df628838e15b23" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:38.138657  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1ea1c9340c5421bb9df628838e15b23" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:38.139598  6317 raft_consensus.cc:3060] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.140100  6469 raft_consensus.cc:3055] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:38.140415  6469 raft_consensus.cc:740] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:38.141253  6469 consensus_queue.cc:260] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.142449  6469 raft_consensus.cc:3060] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.148569  6317 raft_consensus.cc:2468] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 2.
I20260709 12:21:38.149008  6469 raft_consensus.cc:2468] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 2.
I20260709 12:21:38.150115  6359 leader_election.cc:304] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: 
I20260709 12:21:38.151022  6660 raft_consensus.cc:2804] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.151537  6660 raft_consensus.cc:697] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 2 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:38.152345  6660 consensus_queue.cc:237] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.159548  6218 catalog_manager.cc:5697] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 1 to 2, leader changed from ec26b183414a47ca8e7e3d185016670d (127.26.123.3) to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.219676  6660 consensus_queue.cc:1048] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.226692  6546 raft_consensus.cc:993] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9: : Instructing follower 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:38.227245  6626 raft_consensus.cc:1081] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Signalling peer 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:38.232118  6546 consensus_queue.cc:1048] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.233332  6317 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62c124ba608e47d7b3c5f576572ffad5"
dest_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:53138
I20260709 12:21:38.234134  6317 raft_consensus.cc:493] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:38.234618  6317 raft_consensus.cc:3060] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.243522  6627 consensus_queue.cc:1048] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.244449  6317 raft_consensus.cc:515] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.246544  6317 leader_election.cc:290] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Requested vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:38.247306  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c124ba608e47d7b3c5f576572ffad5" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:38.247632  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c124ba608e47d7b3c5f576572ffad5" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:38.248041  6395 raft_consensus.cc:3055] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:38.248246  6469 raft_consensus.cc:3060] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.248462  6395 raft_consensus.cc:740] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:38.249481  6395 consensus_queue.cc:260] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.250929  6395 raft_consensus.cc:3060] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.252537  6603 raft_consensus.cc:993] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d: : Instructing follower b330bead86d943e8bf0a329597ac24f9 to start an election
I20260709 12:21:38.253162  6603 raft_consensus.cc:1081] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Signalling peer b330bead86d943e8bf0a329597ac24f9 to start an election
I20260709 12:21:38.258348  6469 raft_consensus.cc:2468] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:38.258776  6393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff02e69900044d57aa73317dc4f6adca"
dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
 from {username='slave'} at 127.0.0.1:42150
I20260709 12:21:38.259898  6393 raft_consensus.cc:493] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:38.260316  6393 raft_consensus.cc:3060] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.263972  6645 consensus_queue.cc:1048] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.266947  6395 raft_consensus.cc:2468] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:38.268000  6283 leader_election.cc:304] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:38.268939  6679 raft_consensus.cc:2804] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.269538  6679 raft_consensus.cc:697] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Becoming Leader. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:38.270543  6679 consensus_queue.cc:237] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.271736  6393 raft_consensus.cc:515] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.274814  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff02e69900044d57aa73317dc4f6adca" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:38.275436  6320 raft_consensus.cc:3060] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.275734  6393 leader_election.cc:290] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 2 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:38.278683  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff02e69900044d57aa73317dc4f6adca" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:38.279325  6469 raft_consensus.cc:3055] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:38.279661  6469 raft_consensus.cc:740] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:38.280524  6469 consensus_queue.cc:260] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.282043  6469 raft_consensus.cc:3060] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.285380  6320 raft_consensus.cc:2468] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 2.
I20260709 12:21:38.290143  6359 leader_election.cc:304] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: 
I20260709 12:21:38.291116  6626 raft_consensus.cc:2804] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.291643  6626 raft_consensus.cc:697] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 2 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:38.291779  6469 raft_consensus.cc:2468] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 2.
I20260709 12:21:38.292838  6626 consensus_queue.cc:237] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.295060  6216 catalog_manager.cc:5697] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad reported cstate change: term changed from 1 to 2, leader changed from b330bead86d943e8bf0a329597ac24f9 (127.26.123.2) to 7416dd332944470faa2d346b708a24ad (127.26.123.1). New cstate: current_term: 2 leader_uuid: "7416dd332944470faa2d346b708a24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.306651  6215 catalog_manager.cc:5697] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 1 to 2, leader changed from ec26b183414a47ca8e7e3d185016670d (127.26.123.3) to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.390091  6632 raft_consensus.cc:993] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d: : Instructing follower 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:38.390748  6632 raft_consensus.cc:1081] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Signalling peer 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:38.392692  6320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9"
dest_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:53124
I20260709 12:21:38.393240  6320 raft_consensus.cc:493] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:38.393538  6320 raft_consensus.cc:3060] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.400193  6320 raft_consensus.cc:515] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.402197  6320 leader_election.cc:290] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Requested vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:38.402662  6636 raft_consensus.cc:993] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad: : Instructing follower ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:38.403218  6636 raft_consensus.cc:1081] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 1 LEADER]: Signalling peer ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:38.403339  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:38.403383  6393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:38.404160  6469 raft_consensus.cc:3055] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:38.404213  6393 raft_consensus.cc:3060] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.404649  6469 raft_consensus.cc:740] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:38.405162  6468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:45780
I20260709 12:21:38.405450  6469 consensus_queue.cc:260] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.405853  6468 raft_consensus.cc:493] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:38.406299  6468 raft_consensus.cc:3060] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.406687  6469 raft_consensus.cc:3060] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.412948  6469 raft_consensus.cc:2468] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:38.413370  6393 raft_consensus.cc:2468] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:38.414132  6283 leader_election.cc:304] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:38.414541  6468 raft_consensus.cc:515] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.415023  6636 raft_consensus.cc:2804] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.415520  6636 raft_consensus.cc:697] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Becoming Leader. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:38.416330  6636 consensus_queue.cc:237] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.416786  6468 leader_election.cc:290] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 2 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:38.417572  6320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:38.418195  6393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ed251d0273a4101bb69a3dc7e9541b8" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:38.418253  6320 raft_consensus.cc:3055] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:38.418848  6393 raft_consensus.cc:3060] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.418840  6320 raft_consensus.cc:740] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:38.419938  6320 consensus_queue.cc:260] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.421255  6320 raft_consensus.cc:3060] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.426857  6393 raft_consensus.cc:2468] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 2.
I20260709 12:21:38.428510  6433 leader_election.cc:304] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:38.428902  6320 raft_consensus.cc:2468] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 2.
I20260709 12:21:38.428267  6218 catalog_manager.cc:5697] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad reported cstate change: term changed from 1 to 2, leader changed from ec26b183414a47ca8e7e3d185016670d (127.26.123.3) to 7416dd332944470faa2d346b708a24ad (127.26.123.1). New cstate: current_term: 2 leader_uuid: "7416dd332944470faa2d346b708a24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.429543  6632 raft_consensus.cc:2804] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.430140  6632 raft_consensus.cc:697] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 2 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:38.431030  6632 consensus_queue.cc:237] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.440742  6645 consensus_queue.cc:1048] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.441659  6215 catalog_manager.cc:5697] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 1 to 2, leader changed from 7416dd332944470faa2d346b708a24ad (127.26.123.1) to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 2 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:38.446946  6626 consensus_queue.cc:1048] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:38.461144  6645 consensus_queue.cc:1048] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.463048  6546 consensus_queue.cc:1048] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:38.477984  6603 raft_consensus.cc:993] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d: : Instructing follower 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:38.478515  6603 raft_consensus.cc:1081] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Signalling peer 7416dd332944470faa2d346b708a24ad to start an election
I20260709 12:21:38.487964  6632 raft_consensus.cc:993] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d: : Instructing follower b330bead86d943e8bf0a329597ac24f9 to start an election
I20260709 12:21:38.488441  6632 raft_consensus.cc:1081] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Signalling peer b330bead86d943e8bf0a329597ac24f9 to start an election
I20260709 12:21:38.492509  6395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b09d133cc6394c56967281782f9821f5"
dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
 from {username='slave'} at 127.0.0.1:42150
I20260709 12:21:38.493106  6395 raft_consensus.cc:493] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:38.493443  6395 raft_consensus.cc:3060] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.494621  6320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ba9fa0a822146d68a7da0de412de790"
dest_uuid: "7416dd332944470faa2d346b708a24ad"
 from {username='slave'} at 127.0.0.1:53124
I20260709 12:21:38.495256  6320 raft_consensus.cc:493] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:38.495615  6320 raft_consensus.cc:3060] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.501070  6395 raft_consensus.cc:515] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.502607  6320 raft_consensus.cc:515] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.503683  6395 leader_election.cc:290] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 2 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:38.504316  6468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:38.504944  6468 raft_consensus.cc:3055] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:38.505280  6468 raft_consensus.cc:740] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:38.505295  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba9fa0a822146d68a7da0de412de790" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:38.505796  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "b330bead86d943e8bf0a329597ac24f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:38.506047  6395 raft_consensus.cc:3060] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.506433  6317 raft_consensus.cc:3060] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.506271  6468 consensus_queue.cc:260] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.507735  6468 raft_consensus.cc:3060] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.513183  6395 raft_consensus.cc:2468] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:38.514447  6320 leader_election.cc:290] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Requested vote from peers b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611), ec26b183414a47ca8e7e3d185016670d (127.26.123.3:44755)
I20260709 12:21:38.514423  6284 leader_election.cc:304] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, b330bead86d943e8bf0a329597ac24f9; no voters: 
I20260709 12:21:38.515540  6679 raft_consensus.cc:2804] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.515669  6469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba9fa0a822146d68a7da0de412de790" candidate_uuid: "7416dd332944470faa2d346b708a24ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
I20260709 12:21:38.516064  6679 raft_consensus.cc:697] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Becoming Leader. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:38.516115  6468 raft_consensus.cc:2468] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 2.
I20260709 12:21:38.516363  6469 raft_consensus.cc:3055] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:38.516770  6469 raft_consensus.cc:740] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:38.516999  6679 consensus_queue.cc:237] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.517467  6317 raft_consensus.cc:2468] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b330bead86d943e8bf0a329597ac24f9 in term 2.
I20260709 12:21:38.517622  6469 consensus_queue.cc:260] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.519251  6469 raft_consensus.cc:3060] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.524847  6469 raft_consensus.cc:2468] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7416dd332944470faa2d346b708a24ad in term 2.
I20260709 12:21:38.517580  6358 leader_election.cc:304] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:38.525074  6214 catalog_manager.cc:5697] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad reported cstate change: term changed from 1 to 2, leader changed from ec26b183414a47ca8e7e3d185016670d (127.26.123.3) to 7416dd332944470faa2d346b708a24ad (127.26.123.1). New cstate: current_term: 2 leader_uuid: "7416dd332944470faa2d346b708a24ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.526466  6626 raft_consensus.cc:2804] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.526957  6626 raft_consensus.cc:697] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 2 LEADER]: Becoming Leader. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:38.527804  6626 consensus_queue.cc:237] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:38.535949  6217 catalog_manager.cc:5697] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 reported cstate change: term changed from 1 to 2, leader changed from ec26b183414a47ca8e7e3d185016670d (127.26.123.3) to b330bead86d943e8bf0a329597ac24f9 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "b330bead86d943e8bf0a329597ac24f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:38.597570  6317 raft_consensus.cc:1275] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Refusing update from remote peer b330bead86d943e8bf0a329597ac24f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.598938  6626 consensus_queue.cc:1048] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.606732  6469 raft_consensus.cc:1275] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Refusing update from remote peer b330bead86d943e8bf0a329597ac24f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.609273  6660 consensus_queue.cc:1048] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:38.690189  6395 raft_consensus.cc:1275] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.691713  6636 consensus_queue.cc:1048] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.700616  6469 raft_consensus.cc:1275] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.702085  6636 consensus_queue.cc:1048] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.713205  6317 raft_consensus.cc:1275] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Refusing update from remote peer b330bead86d943e8bf0a329597ac24f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.714493  6546 consensus_queue.cc:1048] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.724622  6469 raft_consensus.cc:1275] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Refusing update from remote peer b330bead86d943e8bf0a329597ac24f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.726828  6626 consensus_queue.cc:1048] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.837873  6395 raft_consensus.cc:1275] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.839095  6636 consensus_queue.cc:1048] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.848428  6469 raft_consensus.cc:1275] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.849864  6679 consensus_queue.cc:1048] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.958539  6317 raft_consensus.cc:1275] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.960007  6627 consensus_queue.cc:1048] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.968987  6395 raft_consensus.cc:1275] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.971495  6603 consensus_queue.cc:1048] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:38.997124  6317 raft_consensus.cc:1275] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Refusing update from remote peer b330bead86d943e8bf0a329597ac24f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:38.998631  6691 consensus_queue.cc:1048] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:39.006626  6469 raft_consensus.cc:1275] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Refusing update from remote peer b330bead86d943e8bf0a329597ac24f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:39.008623  6546 consensus_queue.cc:1048] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:39.107702  6395 raft_consensus.cc:1275] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:39.109247  6679 consensus_queue.cc:1048] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:39.118952  6469 raft_consensus.cc:1275] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Refusing update from remote peer 7416dd332944470faa2d346b708a24ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:39.120810  6693 consensus_queue.cc:1048] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:41.013260  6215 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39098:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:21:41.036676  6215 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39098:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:21:41.088047 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:41.424046  6756 tablet.cc:1724] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:41.508505  6421 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:c800840c300c4fe5a61e9fcb8bc8dc39)
W20260709 12:21:41.495465  6284 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:45611, user_credentials={real_user=slave}} blocked reactor thread for 43650.3us
W20260709 12:21:41.544284  6757 log.cc:927] Time spent T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:21:41.555277  6421 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:c800840c300c4fe5a61e9fcb8bc8dc39)
I20260709 12:21:41.557821  6762 tablet.cc:1724] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:41.563409  6421 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:c800840c300c4fe5a61e9fcb8bc8dc39)
I20260709 12:21:41.575026  6770 tablet.cc:1724] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.599887  6768 tablet.cc:1724] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.601356  6766 tablet.cc:1724] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.568753  6760 tablet.cc:1724] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.544374  6761 tablet.cc:1724] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.548480  6758 tablet.cc:1724] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.614409  6775 tablet.cc:1724] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:41.648111  6359 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:35523, user_credentials={real_user=slave}} blocked reactor thread for 57256.5us
I20260709 12:21:41.676532  6776 tablet.cc:1724] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:41.642253  6284 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:45611, user_credentials={real_user=slave}} blocked reactor thread for 58557.1us
I20260709 12:21:41.697348  6395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b09d133cc6394c56967281782f9821f5"
dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
mode: GRACEFUL
new_leader_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:42170
I20260709 12:21:41.698722  6395 raft_consensus.cc:606] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 2 LEADER]: Received request to transfer leadership to TS ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:41.725579  6771 tablet.cc:1724] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.735996  6758 tablet.cc:1724] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.737036  6768 tablet.cc:1724] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.612915  6756 tablet.cc:1724] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.613641  6772 tablet.cc:1724] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:21:41.816061  6358 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.3:44755, user_credentials={real_user=slave}} blocked reactor thread for 56518.5us
W20260709 12:21:41.823067  6777 log.cc:927] Time spent T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:21:41.847620  6760 tablet.cc:1724] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.863179  6759 raft_consensus.cc:993] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9: : Instructing follower ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:41.863852  6759 raft_consensus.cc:1081] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 2 LEADER]: Signalling peer ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:41.867648  6768 tablet.cc:1724] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.872272  6468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b09d133cc6394c56967281782f9821f5"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:45768
I20260709 12:21:41.873180  6468 raft_consensus.cc:493] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:41.873659  6468 raft_consensus.cc:3060] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:41.857282  6766 tablet.cc:1724] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.885531  6775 tablet.cc:1724] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.891574  6770 tablet.cc:1724] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.906061  6758 tablet.cc:1724] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.926815  6768 tablet.cc:1724] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.927800  6766 tablet.cc:1724] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.940440  6760 tablet.cc:1724] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.929507  6756 tablet.cc:1724] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.945135  6468 raft_consensus.cc:515] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:41.970651  6317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9"
dest_uuid: "7416dd332944470faa2d346b708a24ad"
mode: GRACEFUL
new_leader_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:53152
I20260709 12:21:41.971504  6317 raft_consensus.cc:606] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Received request to transfer leadership to TS ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:41.977942 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:21:41.980517  6771 tablet.cc:1724] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:41.981313 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:41.995379  6468 leader_election.cc:290] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 3 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:41.997454  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:41.998454  6395 raft_consensus.cc:3055] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:41.998764  6395 raft_consensus.cc:740] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:41.999752  6395 consensus_queue.cc:260] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.001235  6395 raft_consensus.cc:3060] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.007992  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09d133cc6394c56967281782f9821f5" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:42.008275  6391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624"
dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
mode: GRACEFUL
new_leader_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:42170
I20260709 12:21:42.008890  6317 raft_consensus.cc:3060] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.009109  6391 raft_consensus.cc:606] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Received request to transfer leadership to TS ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:42.015332  6316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0"
dest_uuid: "7416dd332944470faa2d346b708a24ad"
mode: GRACEFUL
new_leader_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:53152
I20260709 12:21:42.016113  6316 raft_consensus.cc:606] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Received request to transfer leadership to TS ec26b183414a47ca8e7e3d185016670d
I20260709 12:21:42.018329 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:42.019187 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:42.020622  6317 raft_consensus.cc:2468] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 3.
I20260709 12:21:42.022600  6433 leader_election.cc:304] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7416dd332944470faa2d346b708a24ad, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:42.023808  6603 raft_consensus.cc:2804] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:41.981829  6761 tablet.cc:1724] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:21:42.031437  6774 raft_consensus.cc:993] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9: : Instructing follower ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:42.034667  6774 raft_consensus.cc:1081] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Signalling peer ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:42.035837  6395 raft_consensus.cc:2468] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 3.
I20260709 12:21:42.037024  6468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:45768
I20260709 12:21:42.038026  6468 raft_consensus.cc:493] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:42.038506  6468 raft_consensus.cc:3060] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.042925  6603 raft_consensus.cc:697] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 3 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:42.044512  6603 consensus_queue.cc:237] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.054786  6216 catalog_manager.cc:6242] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb: soft_deleted_table [id=77771ffead724122be126c5e184c825e] alter complete (version 1)
I20260709 12:21:42.057026  6468 raft_consensus.cc:515] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.063176  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:42.063978  6395 raft_consensus.cc:3055] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:42.064694  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ce53ec9b61046068b0564b54bdc9624" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:42.064864  6395 raft_consensus.cc:740] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b330bead86d943e8bf0a329597ac24f9, State: Running, Role: LEADER
I20260709 12:21:42.065498  6317 raft_consensus.cc:3060] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.066150  6395 consensus_queue.cc:260] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.064355  6468 leader_election.cc:290] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 2 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:42.068156  6395 raft_consensus.cc:3060] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.076702  6395 raft_consensus.cc:2468] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 2.
I20260709 12:21:42.078123  6433 leader_election.cc:304] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:42.079205  6603 raft_consensus.cc:2804] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:42.079706  6603 raft_consensus.cc:697] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 2 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:42.080786  6603 consensus_queue.cc:237] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.084061  6214 catalog_manager.cc:5697] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 2 to 3, leader changed from b330bead86d943e8bf0a329597ac24f9 (127.26.123.2) to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 3 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:42.094543  6317 raft_consensus.cc:2468] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 2.
I20260709 12:21:42.109241  6214 catalog_manager.cc:5697] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 1 to 2, leader changed from b330bead86d943e8bf0a329597ac24f9 (127.26.123.2) to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 2 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:42.159261  6769 raft_consensus.cc:993] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad: : Instructing follower ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:42.159893  6763 raft_consensus.cc:1081] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Signalling peer ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:42.161550  6468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:45780
I20260709 12:21:42.162106  6468 raft_consensus.cc:493] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:42.162387  6468 raft_consensus.cc:3060] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.168443  6468 raft_consensus.cc:515] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.170168  6468 leader_election.cc:290] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 3 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:42.171072  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:42.171345  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3c293ac71de4d9fa1ec15d9048d47b9" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:42.171718  6317 raft_consensus.cc:3055] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:42.171900  6395 raft_consensus.cc:3060] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.172040  6317 raft_consensus.cc:740] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:42.172802  6317 consensus_queue.cc:260] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.174234  6317 raft_consensus.cc:3060] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.179256  6395 raft_consensus.cc:2468] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 3.
I20260709 12:21:42.180635  6433 leader_election.cc:304] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:42.180823  6317 raft_consensus.cc:2468] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 3.
I20260709 12:21:42.181489  6603 raft_consensus.cc:2804] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:42.182127  6603 raft_consensus.cc:697] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 3 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:42.183252  6603 consensus_queue.cc:237] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.192960  6214 catalog_manager.cc:5697] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 2 to 3, leader changed from 7416dd332944470faa2d346b708a24ad (127.26.123.1) to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 3 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:42.327971  6765 raft_consensus.cc:993] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad: : Instructing follower ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:42.328425  6769 raft_consensus.cc:1081] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Signalling peer ec26b183414a47ca8e7e3d185016670d to start an election
I20260709 12:21:42.330050  6468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0"
dest_uuid: "ec26b183414a47ca8e7e3d185016670d"
 from {username='slave'} at 127.0.0.1:45780
I20260709 12:21:42.330523  6468 raft_consensus.cc:493] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:42.330787  6468 raft_consensus.cc:3060] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.336561  6468 raft_consensus.cc:515] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.338142  6468 leader_election.cc:290] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 3 election: Requested vote from peers 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523), b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611)
I20260709 12:21:42.339082  6317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7416dd332944470faa2d346b708a24ad"
I20260709 12:21:42.339334  6395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3bf4631ebe421981c7020476d222b0" candidate_uuid: "ec26b183414a47ca8e7e3d185016670d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b330bead86d943e8bf0a329597ac24f9"
I20260709 12:21:42.339797  6317 raft_consensus.cc:3055] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:42.339856  6395 raft_consensus.cc:3060] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.340229  6317 raft_consensus.cc:740] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7416dd332944470faa2d346b708a24ad, State: Running, Role: LEADER
I20260709 12:21:42.341102  6317 consensus_queue.cc:260] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.342542  6317 raft_consensus.cc:3060] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.347800  6395 raft_consensus.cc:2468] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 3.
I20260709 12:21:42.349084  6433 leader_election.cc:304] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b330bead86d943e8bf0a329597ac24f9, ec26b183414a47ca8e7e3d185016670d; no voters: 
I20260709 12:21:42.350015  6603 raft_consensus.cc:2804] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:42.350234  6317 raft_consensus.cc:2468] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec26b183414a47ca8e7e3d185016670d in term 3.
I20260709 12:21:42.350564  6603 raft_consensus.cc:697] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 3 LEADER]: Becoming Leader. State: Replica: ec26b183414a47ca8e7e3d185016670d, State: Running, Role: LEADER
I20260709 12:21:42.351645  6603 consensus_queue.cc:237] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } }
I20260709 12:21:42.361434  6214 catalog_manager.cc:5697] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d reported cstate change: term changed from 2 to 3, leader changed from 7416dd332944470faa2d346b708a24ad (127.26.123.1) to ec26b183414a47ca8e7e3d185016670d (127.26.123.3). New cstate: current_term: 3 leader_uuid: "ec26b183414a47ca8e7e3d185016670d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec26b183414a47ca8e7e3d185016670d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 44755 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:42.427713  6395 raft_consensus.cc:1275] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:42.429019  6603 consensus_queue.cc:1048] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:42.438562  6317 raft_consensus.cc:1275] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:42.440629  6786 consensus_queue.cc:1048] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:42.522505  6317 raft_consensus.cc:1275] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.523780  6786 consensus_queue.cc:1048] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.533802  6395 raft_consensus.cc:1275] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:42.535741  6603 consensus_queue.cc:1048] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:42.754654  6391 raft_consensus.cc:1275] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:42.755905  6781 consensus_queue.cc:1048] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:42.764012  6317 raft_consensus.cc:1275] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:42.765793  6603 consensus_queue.cc:1048] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:42.816061  6391 raft_consensus.cc:1275] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:42.817404  6603 consensus_queue.cc:1048] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b330bead86d943e8bf0a329597ac24f9" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:42.827251  6317 raft_consensus.cc:1275] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Refusing update from remote peer ec26b183414a47ca8e7e3d185016670d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:42.830624  6786 consensus_queue.cc:1048] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7416dd332944470faa2d346b708a24ad" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:43.020756 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:43.026065 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:43.026547 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:43.032274 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:43.032775 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:44.033944 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:44.039222 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:44.039698 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:44.044581 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:44.045029 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:45.046190 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:45.051543 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:45.052042 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:45.056838 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:45.057296 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:46.058511 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:46.065522 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:46.066072 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:46.071115 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:46.071664 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:47.073005 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:47.080410 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:47.080956 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:47.087750 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:47.088254 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:48.089377 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:48.096276 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:48.096797 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:48.101719 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:48.102203 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:49.103310 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:49.110028 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:49.110582 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:49.116513 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:49.117221 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:50.118432 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:50.123687 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:50.124086 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:50.128965 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:50.129333 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:51.131343 27116 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:21:51.142088 27116 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:21:51.142902 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:51.154608 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:51.155956 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:52.322537 27116 tablet_server.cc:179] TabletServer@127.26.123.1:0 shutting down...
W20260709 12:21:52.330663  6433 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 12:21:52.342972  6433 consensus_peers.cc:597] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d -> Peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Couldn't send request to peer 7416dd332944470faa2d346b708a24ad. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:52.371212  6359 consensus_peers.cc:597] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 -> Peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Couldn't send request to peer 7416dd332944470faa2d346b708a24ad. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.403719 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:52.404803 27116 tablet_replica.cc:333] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.406111 27116 raft_consensus.cc:2243] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.407011 27116 raft_consensus.cc:2272] T c800840c300c4fe5a61e9fcb8bc8dc39 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.411499 27116 tablet_replica.cc:333] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.412633 27116 raft_consensus.cc:2243] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.414168 27116 raft_consensus.cc:2272] T 62c124ba608e47d7b3c5f576572ffad5 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.418887 27116 tablet_replica.cc:333] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.419869 27116 raft_consensus.cc:2243] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.420807 27116 raft_consensus.cc:2272] T c1ea1c9340c5421bb9df628838e15b23 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.423851 27116 tablet_replica.cc:333] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.424697 27116 raft_consensus.cc:2243] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.425501 27116 raft_consensus.cc:2272] T b09d133cc6394c56967281782f9821f5 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.428656 27116 tablet_replica.cc:333] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.429549 27116 raft_consensus.cc:2243] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.431092 27116 raft_consensus.cc:2272] T 28a630c71ca84039b721dcb6343727a9 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.435542 27116 tablet_replica.cc:333] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.436547 27116 raft_consensus.cc:2243] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.437527 27116 raft_consensus.cc:2272] T 1ed251d0273a4101bb69a3dc7e9541b8 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.441797 27116 tablet_replica.cc:333] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.442788 27116 raft_consensus.cc:2243] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.443657 27116 raft_consensus.cc:2272] T 9ce53ec9b61046068b0564b54bdc9624 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.447992 27116 tablet_replica.cc:333] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.449067 27116 raft_consensus.cc:2243] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.449841 27116 raft_consensus.cc:2272] T 8f396467f79b438cb11b5bccf557ff0d P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.454316 27116 tablet_replica.cc:333] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.455334 27116 raft_consensus.cc:2243] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.456190 27116 raft_consensus.cc:2272] T 18d6b432d7dc44e9a89513234e129bd7 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.460772 27116 tablet_replica.cc:333] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.461618 27116 raft_consensus.cc:2243] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.462219 27116 raft_consensus.cc:2272] T ea3bf4631ebe421981c7020476d222b0 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.465030 27116 tablet_replica.cc:333] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.465690 27116 raft_consensus.cc:2243] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.466825 27116 raft_consensus.cc:2272] T d120b13ba1044b03a9ed88a94a704041 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.469480 27116 tablet_replica.cc:333] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.470193 27116 raft_consensus.cc:2243] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.470844 27116 raft_consensus.cc:2272] T e3c293ac71de4d9fa1ec15d9048d47b9 P 7416dd332944470faa2d346b708a24ad [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.473557 27116 tablet_replica.cc:333] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.474192 27116 raft_consensus.cc:2243] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.474757 27116 raft_consensus.cc:2272] T e707cafcda8a4012953a99b8aed9340f P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.477602 27116 tablet_replica.cc:333] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.478317 27116 raft_consensus.cc:2243] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.479449 27116 raft_consensus.cc:2272] T 1545406360df433fa90dee76690705a0 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.483632 27116 tablet_replica.cc:333] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.484549 27116 raft_consensus.cc:2243] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.485298 27116 raft_consensus.cc:2272] T 445b5ae598f64f36a672a41bb779b96f P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.488721 27116 tablet_replica.cc:333] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.489327 27116 raft_consensus.cc:2243] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.490492 27116 raft_consensus.cc:2272] T 81b89db80b78448c9a4dedcb9a3dfef0 P 7416dd332944470faa2d346b708a24ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.493340 27116 tablet_replica.cc:333] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.493943 27116 raft_consensus.cc:2243] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.494567 27116 raft_consensus.cc:2272] T ff02e69900044d57aa73317dc4f6adca P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.497582 27116 tablet_replica.cc:333] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad: stopping tablet replica
I20260709 12:21:52.498185 27116 raft_consensus.cc:2243] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.499223 27116 raft_consensus.cc:2272] T 8ba9fa0a822146d68a7da0de412de790 P 7416dd332944470faa2d346b708a24ad [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.552305 27116 tablet_server.cc:196] TabletServer@127.26.123.1:0 shutdown complete.
W20260709 12:21:52.600621  6433 consensus_peers.cc:597] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d -> Peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Couldn't send request to peer 7416dd332944470faa2d346b708a24ad. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:35523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.611620 27116 tablet_server.cc:179] TabletServer@127.26.123.2:0 shutting down...
W20260709 12:21:52.626442  6433 consensus_peers.cc:597] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d -> Peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Couldn't send request to peer 7416dd332944470faa2d346b708a24ad. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:35523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:52.645985  6433 consensus_peers.cc:597] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d -> Peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Couldn't send request to peer 7416dd332944470faa2d346b708a24ad. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:35523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:52.655682  6433 consensus_peers.cc:597] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d -> Peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Couldn't send request to peer 7416dd332944470faa2d346b708a24ad. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:35523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.656208 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:52.657053 27116 tablet_replica.cc:333] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.658164 27116 raft_consensus.cc:2243] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.659276 27116 raft_consensus.cc:2272] T c800840c300c4fe5a61e9fcb8bc8dc39 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.662614 27116 tablet_replica.cc:333] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.663476 27116 raft_consensus.cc:2243] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.664487 27116 raft_consensus.cc:2272] T 1ed251d0273a4101bb69a3dc7e9541b8 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:52.666186  6433 consensus_peers.cc:597] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d -> Peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Couldn't send request to peer b330bead86d943e8bf0a329597ac24f9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.668289 27116 tablet_replica.cc:333] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.669353 27116 raft_consensus.cc:2243] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.670547 27116 raft_consensus.cc:2272] T c1ea1c9340c5421bb9df628838e15b23 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.673629 27116 tablet_replica.cc:333] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.674494 27116 raft_consensus.cc:2243] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.675333 27116 raft_consensus.cc:2272] T 62c124ba608e47d7b3c5f576572ffad5 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.678864 27116 tablet_replica.cc:333] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
W20260709 12:21:52.679467  6359 consensus_peers.cc:597] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 -> Peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Couldn't send request to peer 7416dd332944470faa2d346b708a24ad. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:35523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.679855 27116 raft_consensus.cc:2243] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.681049 27116 raft_consensus.cc:2272] T ff02e69900044d57aa73317dc4f6adca P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.684329 27116 tablet_replica.cc:333] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.685097 27116 raft_consensus.cc:2243] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.685798 27116 raft_consensus.cc:2272] T 8ba9fa0a822146d68a7da0de412de790 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.689316 27116 tablet_replica.cc:333] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.690321 27116 raft_consensus.cc:2243] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.691056 27116 raft_consensus.cc:2272] T 28a630c71ca84039b721dcb6343727a9 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:52.692559  6359 consensus_peers.cc:597] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 -> Peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Couldn't send request to peer 7416dd332944470faa2d346b708a24ad. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:35523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.694054 27116 tablet_replica.cc:333] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.694972 27116 raft_consensus.cc:2243] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.695649 27116 raft_consensus.cc:2272] T b09d133cc6394c56967281782f9821f5 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.698455 27116 tablet_replica.cc:333] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.699096 27116 raft_consensus.cc:2243] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.699689 27116 raft_consensus.cc:2272] T 81b89db80b78448c9a4dedcb9a3dfef0 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.702463 27116 tablet_replica.cc:333] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.703154 27116 raft_consensus.cc:2243] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.703723 27116 raft_consensus.cc:2272] T 1545406360df433fa90dee76690705a0 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.706542 27116 tablet_replica.cc:333] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.707219 27116 raft_consensus.cc:2243] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.708326 27116 raft_consensus.cc:2272] T 445b5ae598f64f36a672a41bb779b96f P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.711602 27116 tablet_replica.cc:333] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.712399 27116 raft_consensus.cc:2243] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.713671 27116 raft_consensus.cc:2272] T 8f396467f79b438cb11b5bccf557ff0d P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:52.715668  6433 consensus_peers.cc:597] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d -> Peer 7416dd332944470faa2d346b708a24ad (127.26.123.1:35523): Couldn't send request to peer 7416dd332944470faa2d346b708a24ad. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:35523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.717847 27116 tablet_replica.cc:333] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
W20260709 12:21:52.718817  6433 consensus_peers.cc:597] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d -> Peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Couldn't send request to peer b330bead86d943e8bf0a329597ac24f9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.718849 27116 raft_consensus.cc:2243] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.720026 27116 raft_consensus.cc:2272] T 9ce53ec9b61046068b0564b54bdc9624 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.723074 27116 tablet_replica.cc:333] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.723811 27116 raft_consensus.cc:2243] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.725080 27116 raft_consensus.cc:2272] T 18d6b432d7dc44e9a89513234e129bd7 P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.728293 27116 tablet_replica.cc:333] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.729115 27116 raft_consensus.cc:2243] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.729913 27116 raft_consensus.cc:2272] T e3c293ac71de4d9fa1ec15d9048d47b9 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.733026 27116 tablet_replica.cc:333] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.733848 27116 raft_consensus.cc:2243] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.734488 27116 raft_consensus.cc:2272] T e707cafcda8a4012953a99b8aed9340f P b330bead86d943e8bf0a329597ac24f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.737119 27116 tablet_replica.cc:333] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.737702 27116 raft_consensus.cc:2243] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.738361 27116 raft_consensus.cc:2272] T ea3bf4631ebe421981c7020476d222b0 P b330bead86d943e8bf0a329597ac24f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.741611 27116 tablet_replica.cc:333] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9: stopping tablet replica
I20260709 12:21:52.742218 27116 raft_consensus.cc:2243] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.742808 27116 raft_consensus.cc:2272] T d120b13ba1044b03a9ed88a94a704041 P b330bead86d943e8bf0a329597ac24f9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:52.768661  6433 consensus_peers.cc:597] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d -> Peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Couldn't send request to peer b330bead86d943e8bf0a329597ac24f9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.793329 27116 tablet_server.cc:196] TabletServer@127.26.123.2:0 shutdown complete.
I20260709 12:21:52.842716 27116 tablet_server.cc:179] TabletServer@127.26.123.3:0 shutting down...
I20260709 12:21:52.882014 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:52.882812 27116 tablet_replica.cc:333] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.883699 27116 raft_consensus.cc:2243] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.884330 27116 raft_consensus.cc:2272] T 28a630c71ca84039b721dcb6343727a9 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.886947 27116 tablet_replica.cc:333] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.887641 27116 raft_consensus.cc:2243] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.888718 27116 raft_consensus.cc:2272] T b09d133cc6394c56967281782f9821f5 P ec26b183414a47ca8e7e3d185016670d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.891360 27116 tablet_replica.cc:333] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.892230 27116 raft_consensus.cc:2243] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.892920 27116 raft_consensus.cc:2272] T 62c124ba608e47d7b3c5f576572ffad5 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.896175 27116 tablet_replica.cc:333] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.896857 27116 raft_consensus.cc:2243] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.897490 27116 raft_consensus.cc:2272] T c800840c300c4fe5a61e9fcb8bc8dc39 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.900038 27116 tablet_replica.cc:333] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.900825 27116 raft_consensus.cc:2243] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.901474 27116 raft_consensus.cc:2272] T c1ea1c9340c5421bb9df628838e15b23 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.904978 27116 tablet_replica.cc:333] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.905781 27116 raft_consensus.cc:2243] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:52.906227  6433 consensus_peers.cc:597] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d -> Peer b330bead86d943e8bf0a329597ac24f9 (127.26.123.2:45611): Couldn't send request to peer b330bead86d943e8bf0a329597ac24f9. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.2:45611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.906533 27116 raft_consensus.cc:2272] T ff02e69900044d57aa73317dc4f6adca P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.909368 27116 tablet_replica.cc:333] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.910087 27116 raft_consensus.cc:2243] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.910677 27116 raft_consensus.cc:2272] T 8ba9fa0a822146d68a7da0de412de790 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.913193 27116 tablet_replica.cc:333] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.913892 27116 raft_consensus.cc:2243] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.914923 27116 raft_consensus.cc:2272] T 1ed251d0273a4101bb69a3dc7e9541b8 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.917521 27116 tablet_replica.cc:333] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.918227 27116 raft_consensus.cc:2243] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.919315 27116 raft_consensus.cc:2272] T e707cafcda8a4012953a99b8aed9340f P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.922222 27116 tablet_replica.cc:333] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.922917 27116 raft_consensus.cc:2243] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.923938 27116 raft_consensus.cc:2272] T e3c293ac71de4d9fa1ec15d9048d47b9 P ec26b183414a47ca8e7e3d185016670d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.926611 27116 tablet_replica.cc:333] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.927255 27116 raft_consensus.cc:2243] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.927822 27116 raft_consensus.cc:2272] T 8f396467f79b438cb11b5bccf557ff0d P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.930460 27116 tablet_replica.cc:333] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.931078 27116 raft_consensus.cc:2243] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.931573 27116 raft_consensus.cc:2272] T 445b5ae598f64f36a672a41bb779b96f P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.934188 27116 tablet_replica.cc:333] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.934790 27116 raft_consensus.cc:2243] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.935317 27116 raft_consensus.cc:2272] T 1545406360df433fa90dee76690705a0 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.937788 27116 tablet_replica.cc:333] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.938376 27116 raft_consensus.cc:2243] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.938916 27116 raft_consensus.cc:2272] T d120b13ba1044b03a9ed88a94a704041 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.941510 27116 tablet_replica.cc:333] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.942123 27116 raft_consensus.cc:2243] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.942628 27116 raft_consensus.cc:2272] T 18d6b432d7dc44e9a89513234e129bd7 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.944904 27116 tablet_replica.cc:333] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.945446 27116 raft_consensus.cc:2243] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.946440 27116 raft_consensus.cc:2272] T ea3bf4631ebe421981c7020476d222b0 P ec26b183414a47ca8e7e3d185016670d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.948830 27116 tablet_replica.cc:333] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.949405 27116 raft_consensus.cc:2243] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:52.950345 27116 raft_consensus.cc:2272] T 9ce53ec9b61046068b0564b54bdc9624 P ec26b183414a47ca8e7e3d185016670d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.952741 27116 tablet_replica.cc:333] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d: stopping tablet replica
I20260709 12:21:52.953263 27116 raft_consensus.cc:2243] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:52.953795 27116 raft_consensus.cc:2272] T 81b89db80b78448c9a4dedcb9a3dfef0 P ec26b183414a47ca8e7e3d185016670d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:52.999152 27116 tablet_server.cc:196] TabletServer@127.26.123.3:0 shutdown complete.
I20260709 12:21:53.043825 27116 master.cc:562] Master@127.26.123.62:39929 shutting down...
I20260709 12:21:53.067278 27116 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:53.067965 27116 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:53.068288 27116 tablet_replica.cc:333] T 00000000000000000000000000000000 P fe410dacadf94b808c24e45361038fbb: stopping tablet replica
I20260709 12:21:53.090265 27116 master.cc:584] Master@127.26.123.62:39929 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21381 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:21:53.150405 27116 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.123.62:45429
I20260709 12:21:53.151686 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:53.158816  6827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:53.158903  6826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:53.159682  6829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:53.160207 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:53.161291 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:53.161484 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:53.161654 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599713161637 us; error 0 us; skew 500 ppm
I20260709 12:21:53.162253 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:53.164822 27116 webserver.cc:533] Webserver started at http://127.26.123.62:43091/ using document root <none> and password file <none>
I20260709 12:21:53.165357 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:53.165552 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:53.165863 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:53.167160 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/master-0-root/instance:
uuid: "2efc56154eae451eb395adff577a7ab5"
format_stamp: "Formatted at 2026-07-09 12:21:53 on dist-test-slave-9q7n"
I20260709 12:21:53.172099 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:21:53.176312  6834 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.177151 27116 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:21:53.177460 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/master-0-root
uuid: "2efc56154eae451eb395adff577a7ab5"
format_stamp: "Formatted at 2026-07-09 12:21:53 on dist-test-slave-9q7n"
I20260709 12:21:53.177789 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:53.195061 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:53.196448 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:53.239114 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.62:45429
I20260709 12:21:53.239243  6885 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.62:45429 every 8 connection(s)
I20260709 12:21:53.243155  6886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.257514  6886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5: Bootstrap starting.
I20260709 12:21:53.263240  6886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.268187  6886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5: No bootstrap required, opened a new log
I20260709 12:21:53.270844  6886 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2efc56154eae451eb395adff577a7ab5" member_type: VOTER }
I20260709 12:21:53.271301  6886 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.271543  6886 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2efc56154eae451eb395adff577a7ab5, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.272167  6886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2efc56154eae451eb395adff577a7ab5" member_type: VOTER }
I20260709 12:21:53.272679  6886 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:53.272902  6886 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:53.273170  6886 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.279011  6886 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2efc56154eae451eb395adff577a7ab5" member_type: VOTER }
I20260709 12:21:53.279572  6886 leader_election.cc:304] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2efc56154eae451eb395adff577a7ab5; no voters: 
I20260709 12:21:53.280830  6886 leader_election.cc:290] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:53.281206  6889 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.282682  6889 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 1 LEADER]: Becoming Leader. State: Replica: 2efc56154eae451eb395adff577a7ab5, State: Running, Role: LEADER
I20260709 12:21:53.283460  6889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2efc56154eae451eb395adff577a7ab5" member_type: VOTER }
I20260709 12:21:53.283926  6886 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:53.288759  6891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efc56154eae451eb395adff577a7ab5. Latest consensus state: current_term: 1 leader_uuid: "2efc56154eae451eb395adff577a7ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2efc56154eae451eb395adff577a7ab5" member_type: VOTER } }
I20260709 12:21:53.289724  6891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:53.291718  6890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2efc56154eae451eb395adff577a7ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2efc56154eae451eb395adff577a7ab5" member_type: VOTER } }
I20260709 12:21:53.292405  6890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:53.296927  6898 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:53.302560  6898 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:53.303699 27116 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:53.312520  6898 catalog_manager.cc:1383] Generated new cluster ID: 37d7179fbd16454abb9333725ba4b7b2
I20260709 12:21:53.312783  6898 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:53.340857  6898 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:53.342413  6898 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:53.354693  6898 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5: Generated new TSK 0
I20260709 12:21:53.355314  6898 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:53.370573 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:53.377236  6907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:53.378165  6908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:53.380024  6910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:53.380904 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:53.381927 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:53.382120 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:53.382248 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599713382236 us; error 0 us; skew 500 ppm
I20260709 12:21:53.382735 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:53.385342 27116 webserver.cc:533] Webserver started at http://127.26.123.1:43231/ using document root <none> and password file <none>
I20260709 12:21:53.385887 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:53.386059 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:53.386277 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:53.387392 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-0-root/instance:
uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
format_stamp: "Formatted at 2026-07-09 12:21:53 on dist-test-slave-9q7n"
I20260709 12:21:53.392427 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:21:53.395933  6915 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.396751 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:21:53.397011 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-0-root
uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
format_stamp: "Formatted at 2026-07-09 12:21:53 on dist-test-slave-9q7n"
I20260709 12:21:53.397265 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:53.427170 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:53.428459 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:53.430073 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:53.432581 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:53.432776 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.433030 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:53.433209 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.478806 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.1:39251
I20260709 12:21:53.478935  6977 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.1:39251 every 8 connection(s)
I20260709 12:21:53.484119 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:53.491343  6982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:53.492312  6983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:53.495186 27116 server_base.cc:1061] running on GCE node
W20260709 12:21:53.495339  6985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:53.496882 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:53.497100 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:53.497159  6978 heartbeater.cc:344] Connected to a master server at 127.26.123.62:45429
I20260709 12:21:53.497334 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599713497309 us; error 0 us; skew 500 ppm
I20260709 12:21:53.497645  6978 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:53.498188 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:53.498652  6978 heartbeater.cc:507] Master 127.26.123.62:45429 requested a full tablet report, sending...
I20260709 12:21:53.501001 27116 webserver.cc:533] Webserver started at http://127.26.123.2:34737/ using document root <none> and password file <none>
I20260709 12:21:53.501210  6851 ts_manager.cc:194] Registered new tserver with Master: 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251)
I20260709 12:21:53.501585 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:53.501881 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:53.502139 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:53.503304  6851 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48152
I20260709 12:21:53.503314 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-1-root/instance:
uuid: "378f6aabcd114831bf096f436d7ca046"
format_stamp: "Formatted at 2026-07-09 12:21:53 on dist-test-slave-9q7n"
I20260709 12:21:53.508733 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:21:53.512236  6990 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.512979 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:21:53.513254 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-1-root
uuid: "378f6aabcd114831bf096f436d7ca046"
format_stamp: "Formatted at 2026-07-09 12:21:53 on dist-test-slave-9q7n"
I20260709 12:21:53.513526 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:53.542685 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:53.543995 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:53.545521 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:53.547935 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:53.548120 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.548368 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:53.548514 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.592305 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.2:46857
I20260709 12:21:53.592577  7052 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.2:46857 every 8 connection(s)
I20260709 12:21:53.597456 27116 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:53.604849  7056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:53.605705  7057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:53.608808  7059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:53.609063 27116 server_base.cc:1061] running on GCE node
I20260709 12:21:53.610426 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:53.610658 27116 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:53.610751  7053 heartbeater.cc:344] Connected to a master server at 127.26.123.62:45429
I20260709 12:21:53.610896 27116 hybrid_clock.cc:648] HybridClock initialized: now 1783599713610873 us; error 0 us; skew 500 ppm
I20260709 12:21:53.611315  7053 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:53.611810 27116 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:53.612268  7053 heartbeater.cc:507] Master 127.26.123.62:45429 requested a full tablet report, sending...
I20260709 12:21:53.614564 27116 webserver.cc:533] Webserver started at http://127.26.123.3:39359/ using document root <none> and password file <none>
I20260709 12:21:53.614807  6851 ts_manager.cc:194] Registered new tserver with Master: 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:53.615288 27116 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:53.615610 27116 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:53.615944 27116 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:53.616743  6851 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48154
I20260709 12:21:53.617609 27116 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-2-root/instance:
uuid: "615cf24a2e7843d88c0371f123d9101d"
format_stamp: "Formatted at 2026-07-09 12:21:53 on dist-test-slave-9q7n"
I20260709 12:21:53.623292 27116 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:21:53.626906  7064 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.627671 27116 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:53.627974 27116 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-2-root
uuid: "615cf24a2e7843d88c0371f123d9101d"
format_stamp: "Formatted at 2026-07-09 12:21:53 on dist-test-slave-9q7n"
I20260709 12:21:53.628265 27116 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskNzQYC5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599329322122-27116-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:53.640925 27116 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:53.642257 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:53.643848 27116 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:53.646205 27116 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:53.646418 27116 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.646660 27116 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:53.646997 27116 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:53.691540 27116 rpc_server.cc:307] RPC server started. Bound to: 127.26.123.3:40969
I20260709 12:21:53.691669  7126 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.26.123.3:40969 every 8 connection(s)
I20260709 12:21:53.706453  7127 heartbeater.cc:344] Connected to a master server at 127.26.123.62:45429
I20260709 12:21:53.706926  7127 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:53.707679  7127 heartbeater.cc:507] Master 127.26.123.62:45429 requested a full tablet report, sending...
I20260709 12:21:53.709610  6851 ts_manager.cc:194] Registered new tserver with Master: 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:53.710589 27116 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015789374s
I20260709 12:21:53.711359  6851 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48160
I20260709 12:21:53.737947  6851 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48172:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:53.740793  6851 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:53.810107  6943 tablet_service.cc:1511] Processing CreateTablet for tablet 20b9ed72120b44d39a1260902d5371a9 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:53.811728  6943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20b9ed72120b44d39a1260902d5371a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.813652  6942 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1d3b5a577d46029b259dc75f6dd66c (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:53.815284  6942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1d3b5a577d46029b259dc75f6dd66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.822137  6941 tablet_service.cc:1511] Processing CreateTablet for tablet 789ada7109834532b815f3b0bcb5e69f (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:53.823870  6941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 789ada7109834532b815f3b0bcb5e69f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.830204  6940 tablet_service.cc:1511] Processing CreateTablet for tablet 7daed065ac5140b2988d4a834806283a (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:53.831794  6940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7daed065ac5140b2988d4a834806283a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.840279  7018 tablet_service.cc:1511] Processing CreateTablet for tablet 20b9ed72120b44d39a1260902d5371a9 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:53.843979  7016 tablet_service.cc:1511] Processing CreateTablet for tablet 789ada7109834532b815f3b0bcb5e69f (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:53.846587  7015 tablet_service.cc:1511] Processing CreateTablet for tablet 7daed065ac5140b2988d4a834806283a (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:53.848239  7017 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1d3b5a577d46029b259dc75f6dd66c (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:53.849957  7018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20b9ed72120b44d39a1260902d5371a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.854127  7015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7daed065ac5140b2988d4a834806283a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.855099  7016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 789ada7109834532b815f3b0bcb5e69f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.856343  7017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1d3b5a577d46029b259dc75f6dd66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.857878  7092 tablet_service.cc:1511] Processing CreateTablet for tablet 20b9ed72120b44d39a1260902d5371a9 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:53.859397  7092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20b9ed72120b44d39a1260902d5371a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.859880  6943 tablet_service.cc:1511] Processing CreateTablet for tablet 35cb8325a0024767ad2aeabad1666a60 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:53.861497  6943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35cb8325a0024767ad2aeabad1666a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.861929  7090 tablet_service.cc:1511] Processing CreateTablet for tablet 789ada7109834532b815f3b0bcb5e69f (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:53.863312  7090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 789ada7109834532b815f3b0bcb5e69f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.859396  7088 tablet_service.cc:1511] Processing CreateTablet for tablet 35cb8325a0024767ad2aeabad1666a60 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:53.864974  7088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35cb8325a0024767ad2aeabad1666a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.867002  7089 tablet_service.cc:1511] Processing CreateTablet for tablet 7daed065ac5140b2988d4a834806283a (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:53.868397  7089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7daed065ac5140b2988d4a834806283a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.860546  7091 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1d3b5a577d46029b259dc75f6dd66c (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:53.874341  7091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1d3b5a577d46029b259dc75f6dd66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.877467  7014 tablet_service.cc:1511] Processing CreateTablet for tablet 35cb8325a0024767ad2aeabad1666a60 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:53.879027  7014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35cb8325a0024767ad2aeabad1666a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.929803  6943 tablet_service.cc:1511] Processing CreateTablet for tablet c07a49706e1d448fb18cb100ecc08e72 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:53.931159  7013 tablet_service.cc:1511] Processing CreateTablet for tablet c07a49706e1d448fb18cb100ecc08e72 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:53.931649  6943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07a49706e1d448fb18cb100ecc08e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.932791  7013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07a49706e1d448fb18cb100ecc08e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.954608  7142 tablet_bootstrap.cc:492] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:53.966423  7144 tablet_bootstrap.cc:492] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:53.976401  7013 tablet_service.cc:1511] Processing CreateTablet for tablet bcdfaa3c489a4693a721d0417519e71b (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:53.980211  6940 tablet_service.cc:1511] Processing CreateTablet for tablet f812b8f6840f46bcb4c4d2a551fc24d1 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:53.985289  6940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f812b8f6840f46bcb4c4d2a551fc24d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.982167  7092 tablet_service.cc:1511] Processing CreateTablet for tablet c07a49706e1d448fb18cb100ecc08e72 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:53.990798  7092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07a49706e1d448fb18cb100ecc08e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:53.984026  7142 tablet_bootstrap.cc:654] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.991761  7144 tablet_bootstrap.cc:654] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.990916  6941 tablet_service.cc:1511] Processing CreateTablet for tablet bcdfaa3c489a4693a721d0417519e71b (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:53.998658  7013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcdfaa3c489a4693a721d0417519e71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:54.001770  7090 tablet_service.cc:1511] Processing CreateTablet for tablet bcdfaa3c489a4693a721d0417519e71b (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:54.003171  7090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcdfaa3c489a4693a721d0417519e71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:54.008877  7015 tablet_service.cc:1511] Processing CreateTablet for tablet f812b8f6840f46bcb4c4d2a551fc24d1 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:54.010363  7015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f812b8f6840f46bcb4c4d2a551fc24d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:54.019582  7091 tablet_service.cc:1511] Processing CreateTablet for tablet f812b8f6840f46bcb4c4d2a551fc24d1 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:54.021059  7091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f812b8f6840f46bcb4c4d2a551fc24d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:54.022455  7014 tablet_service.cc:1511] Processing CreateTablet for tablet b7ba6f81b56943a88a91bdc549a91976 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:54.023696  7014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ba6f81b56943a88a91bdc549a91976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:54.026495  6941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcdfaa3c489a4693a721d0417519e71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:54.026937  6943 tablet_service.cc:1511] Processing CreateTablet for tablet b7ba6f81b56943a88a91bdc549a91976 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:54.027249  7089 tablet_service.cc:1511] Processing CreateTablet for tablet b7ba6f81b56943a88a91bdc549a91976 (DEFAULT_TABLE table=test-workload [id=96bd3225d0b540788943c3aa24879460]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:54.028229  6943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ba6f81b56943a88a91bdc549a91976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:54.028556  7089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ba6f81b56943a88a91bdc549a91976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:54.040289  7142 tablet_bootstrap.cc:492] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:54.041082  7142 ts_tablet_manager.cc:1403] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.087s	user 0.008s	sys 0.014s
I20260709 12:21:54.044126  7143 tablet_bootstrap.cc:492] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:54.045293  7142 raft_consensus.cc:359] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.046213  7142 raft_consensus.cc:385] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.050469  7142 raft_consensus.cc:740] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.051606  7142 consensus_queue.cc:260] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.067662  7142 ts_tablet_manager.cc:1434] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.005s
I20260709 12:21:54.068578  6978 heartbeater.cc:499] Master 127.26.123.62:45429 was elected leader, sending a full tablet report...
I20260709 12:21:54.071523  7143 tablet_bootstrap.cc:654] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.073035  7142 tablet_bootstrap.cc:492] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:54.081660  7142 tablet_bootstrap.cc:654] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.082832  7146 raft_consensus.cc:493] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.083415  7146 raft_consensus.cc:515] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.086478  7144 tablet_bootstrap.cc:492] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:54.087447  7144 ts_tablet_manager.cc:1403] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.121s	user 0.017s	sys 0.000s
I20260709 12:21:54.090153  7143 tablet_bootstrap.cc:492] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:54.090785  7143 ts_tablet_manager.cc:1403] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.000s
I20260709 12:21:54.091228  7144 raft_consensus.cc:359] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.092141  7144 raft_consensus.cc:385] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.092442  7144 raft_consensus.cc:740] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.093333  7144 consensus_queue.cc:260] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.094251  7143 raft_consensus.cc:359] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.095170  7143 raft_consensus.cc:385] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.095468  7143 raft_consensus.cc:740] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.096457  7143 consensus_queue.cc:260] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.100844  7146 leader_election.cc:290] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.105293  7053 heartbeater.cc:499] Master 127.26.123.62:45429 was elected leader, sending a full tablet report...
I20260709 12:21:54.106431  7143 ts_tablet_manager.cc:1434] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.004s
I20260709 12:21:54.107628  7143 tablet_bootstrap.cc:492] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:54.120705  7143 tablet_bootstrap.cc:654] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.130249  7142 tablet_bootstrap.cc:492] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:54.129707  7144 ts_tablet_manager.cc:1434] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.040s	user 0.019s	sys 0.019s
I20260709 12:21:54.132424  7142 ts_tablet_manager.cc:1403] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.060s	user 0.014s	sys 0.003s
I20260709 12:21:54.133620  7127 heartbeater.cc:499] Master 127.26.123.62:45429 was elected leader, sending a full tablet report...
I20260709 12:21:54.137424  7142 raft_consensus.cc:359] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:54.138957  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "789ada7109834532b815f3b0bcb5e69f" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:54.141747  7028 raft_consensus.cc:2468] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:54.144598  6918 leader_election.cc:304] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.145116  7143 tablet_bootstrap.cc:492] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:54.145538  7146 raft_consensus.cc:2804] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.145826  7143 ts_tablet_manager.cc:1403] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.014s
I20260709 12:21:54.146023  7146 raft_consensus.cc:493] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.146503  7146 raft_consensus.cc:3060] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.149300  7143 raft_consensus.cc:359] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.150179  7143 raft_consensus.cc:385] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.150486  7143 raft_consensus.cc:740] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.151521  7142 raft_consensus.cc:385] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.151335  7143 consensus_queue.cc:260] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.151940  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "789ada7109834532b815f3b0bcb5e69f" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.152388  7142 raft_consensus.cc:740] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.153436  7142 consensus_queue.cc:260] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
W20260709 12:21:54.153838  6916 leader_election.cc:343] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.155997  7143 ts_tablet_manager.cc:1434] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:21:54.157141  7143 tablet_bootstrap.cc:492] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:54.160116  7146 raft_consensus.cc:515] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.163669  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "789ada7109834532b815f3b0bcb5e69f" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:54.145990  7144 tablet_bootstrap.cc:492] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:54.164471  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "789ada7109834532b815f3b0bcb5e69f" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:54.164373  7028 raft_consensus.cc:3060] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:54.169613  6916 leader_election.cc:343] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.171334  7146 leader_election.cc:290] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.171746  7144 tablet_bootstrap.cc:654] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.175448  7028 raft_consensus.cc:2468] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:54.177044  6918 leader_election.cc:304] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 615cf24a2e7843d88c0371f123d9101d
I20260709 12:21:54.180917  7155 raft_consensus.cc:2804] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.163913  7143 tablet_bootstrap.cc:654] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.182026  7142 ts_tablet_manager.cc:1434] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.048s	user 0.022s	sys 0.009s
I20260709 12:21:54.183399  7142 tablet_bootstrap.cc:492] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:54.183983  7144 tablet_bootstrap.cc:492] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:54.184592  7144 ts_tablet_manager.cc:1403] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.039s	user 0.008s	sys 0.010s
I20260709 12:21:54.187781  7144 raft_consensus.cc:359] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:54.188536  7144 raft_consensus.cc:385] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.188833  7144 raft_consensus.cc:740] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.189868  7144 consensus_queue.cc:260] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:54.191814  7142 tablet_bootstrap.cc:654] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:54.195907  7128 tablet.cc:2406] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:21:54.199743  7144 ts_tablet_manager.cc:1434] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:21:54.200796  7144 tablet_bootstrap.cc:492] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:54.202133  7143 tablet_bootstrap.cc:492] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:54.202838  7143 ts_tablet_manager.cc:1403] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.046s	user 0.005s	sys 0.011s
I20260709 12:21:54.205847  7155 raft_consensus.cc:697] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:54.207036  7155 consensus_queue.cc:237] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.208698  7153 raft_consensus.cc:493] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.209177  7153 raft_consensus.cc:515] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.210768  7143 raft_consensus.cc:359] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.211686  7143 raft_consensus.cc:385] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.212055  7143 raft_consensus.cc:740] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.212997  7143 consensus_queue.cc:260] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.211457  7153 leader_election.cc:290] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.217774  7143 ts_tablet_manager.cc:1434] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:21:54.217939  7142 tablet_bootstrap.cc:492] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:54.218693  7142 ts_tablet_manager.cc:1403] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.011s
I20260709 12:21:54.217797  6848 catalog_manager.cc:5697] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.218953  7143 tablet_bootstrap.cc:492] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:54.222520  7142 raft_consensus.cc:359] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.223431  7142 raft_consensus.cc:385] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.223801  7142 raft_consensus.cc:740] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.224792  7142 consensus_queue.cc:260] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.224080  7144 tablet_bootstrap.cc:654] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.226570  7143 tablet_bootstrap.cc:654] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.227531  7142 ts_tablet_manager.cc:1434] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:21:54.228765  7142 tablet_bootstrap.cc:492] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:54.235790  7142 tablet_bootstrap.cc:654] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.244709  7144 tablet_bootstrap.cc:492] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:54.245362  7144 ts_tablet_manager.cc:1403] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.002s
I20260709 12:21:54.250326  7153 raft_consensus.cc:493] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.251082  7153 raft_consensus.cc:515] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.253656  7153 leader_election.cc:290] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.262475  7144 raft_consensus.cc:359] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.263502  7144 raft_consensus.cc:385] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.263819  7144 raft_consensus.cc:740] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.264744  7144 consensus_queue.cc:260] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.266074  7143 tablet_bootstrap.cc:492] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:54.266686  7143 ts_tablet_manager.cc:1403] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.048s	user 0.026s	sys 0.020s
I20260709 12:21:54.268116  7155 raft_consensus.cc:493] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.268708  7155 raft_consensus.cc:515] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.269580  7143 raft_consensus.cc:359] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.269927  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:54.270349  7143 raft_consensus.cc:385] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.270699  7143 raft_consensus.cc:740] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.271770  7142 tablet_bootstrap.cc:492] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
W20260709 12:21:54.272090  6991 leader_election.cc:343] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.272276  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.272434  7142 ts_tablet_manager.cc:1403] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.044s	user 0.024s	sys 0.016s
I20260709 12:21:54.272380  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:54.273506  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
W20260709 12:21:54.274130  6916 leader_election.cc:343] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.272806  7155 leader_election.cc:290] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
W20260709 12:21:54.274645  6918 leader_election.cc:343] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.275249  6918 leader_election.cc:304] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d
I20260709 12:21:54.276034  7155 raft_consensus.cc:2749] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.276402  7142 raft_consensus.cc:359] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
W20260709 12:21:54.276813  6991 leader_election.cc:343] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.277223  7142 raft_consensus.cc:385] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.277619  7142 raft_consensus.cc:740] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.274461  7143 consensus_queue.cc:260] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.278478  7142 consensus_queue.cc:260] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.279532  7152 raft_consensus.cc:493] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.280076  7152 raft_consensus.cc:515] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.281157  7142 ts_tablet_manager.cc:1434] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:54.283032  7152 leader_election.cc:290] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:54.282840  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.284289  7142 tablet_bootstrap.cc:492] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
W20260709 12:21:54.284668  6991 leader_election.cc:343] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.285194  6991 leader_election.cc:304] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046; no voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:54.287721  7153 raft_consensus.cc:2749] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.288023  7143 ts_tablet_manager.cc:1434] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:21:54.292691  7142 tablet_bootstrap.cc:654] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.294442  7144 ts_tablet_manager.cc:1434] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.001s
I20260709 12:21:54.295657  7144 tablet_bootstrap.cc:492] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:54.297001  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
W20260709 12:21:54.297443  6899 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:54.297389  7143 tablet_bootstrap.cc:492] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:54.307024  7153 raft_consensus.cc:493] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.307667  7143 tablet_bootstrap.cc:654] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:54.307562  6991 leader_election.cc:343] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.307484  7142 tablet_bootstrap.cc:492] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:54.307741  7153 raft_consensus.cc:515] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.309226  7142 ts_tablet_manager.cc:1403] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.006s
I20260709 12:21:54.308566  6991 leader_election.cc:304] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046; no voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:54.311096  7153 leader_election.cc:290] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.311743  7153 raft_consensus.cc:2749] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.313035  7142 raft_consensus.cc:359] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.314004  7142 raft_consensus.cc:385] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.314330  7142 raft_consensus.cc:740] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.315378  7142 consensus_queue.cc:260] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.318001  7142 ts_tablet_manager.cc:1434] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:21:54.319162  7142 tablet_bootstrap.cc:492] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:54.326486  7144 tablet_bootstrap.cc:654] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.326735  7142 tablet_bootstrap.cc:654] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.331789  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7daed065ac5140b2988d4a834806283a" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:54.332608  6953 raft_consensus.cc:2468] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:54.335569  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7daed065ac5140b2988d4a834806283a" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.335657  6991 leader_election.cc:304] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.336382  7102 raft_consensus.cc:2468] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:54.336915  7153 raft_consensus.cc:2804] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.337453  7153 raft_consensus.cc:493] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.337817  7153 raft_consensus.cc:3060] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.349252  7143 tablet_bootstrap.cc:492] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:54.349992  7143 ts_tablet_manager.cc:1403] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.004s
I20260709 12:21:54.352779  7153 raft_consensus.cc:515] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.354300  7143 raft_consensus.cc:359] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.355135  7143 raft_consensus.cc:385] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.355443  7143 raft_consensus.cc:740] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.356225  7143 consensus_queue.cc:260] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.357244  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7daed065ac5140b2988d4a834806283a" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:54.357939  6953 raft_consensus.cc:3060] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.359045  7143 ts_tablet_manager.cc:1434] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:21:54.360191  7143 tablet_bootstrap.cc:492] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:54.366806  7143 tablet_bootstrap.cc:654] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.369171  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7daed065ac5140b2988d4a834806283a" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:54.369997  7102 raft_consensus.cc:3060] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.374032  7144 tablet_bootstrap.cc:492] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:54.374683  7144 ts_tablet_manager.cc:1403] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.079s	user 0.014s	sys 0.004s
I20260709 12:21:54.377646  6952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7daed065ac5140b2988d4a834806283a" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:54.378925  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7daed065ac5140b2988d4a834806283a" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:54.379796  7028 raft_consensus.cc:2393] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 615cf24a2e7843d88c0371f123d9101d in current term 1: Already voted for candidate 378f6aabcd114831bf096f436d7ca046 in this term.
I20260709 12:21:54.380111  7166 raft_consensus.cc:493] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.380633  7166 raft_consensus.cc:515] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.382639  7102 raft_consensus.cc:2468] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:54.383908  6991 leader_election.cc:304] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:54.381529  7065 leader_election.cc:304] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d; no voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:54.385975  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "789ada7109834532b815f3b0bcb5e69f" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:54.386072  7152 raft_consensus.cc:2749] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.386883  7028 raft_consensus.cc:2393] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 615cf24a2e7843d88c0371f123d9101d in current term 1: Already voted for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in this term.
I20260709 12:21:54.387063  7166 leader_election.cc:290] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:54.384907  7170 raft_consensus.cc:2804] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.387617  6952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "789ada7109834532b815f3b0bcb5e69f" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:54.385069  7143 tablet_bootstrap.cc:492] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:54.388628  7143 ts_tablet_manager.cc:1403] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.000s
I20260709 12:21:54.381542  7144 raft_consensus.cc:359] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.389493  7065 leader_election.cc:304] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d; no voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:54.389643  7170 raft_consensus.cc:697] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:54.390165  7144 raft_consensus.cc:385] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.390561  7166 raft_consensus.cc:3060] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.391302  7144 raft_consensus.cc:740] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.391760  7143 raft_consensus.cc:359] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:54.392407  7143 raft_consensus.cc:385] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.392143  7144 consensus_queue.cc:260] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.388060  7142 tablet_bootstrap.cc:492] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:54.393369  7142 ts_tablet_manager.cc:1403] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.075s	user 0.018s	sys 0.037s
I20260709 12:21:54.390799  7170 consensus_queue.cc:237] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.396253  7142 raft_consensus.cc:359] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.380604  6953 raft_consensus.cc:2468] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:54.397126  7142 raft_consensus.cc:385] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.397569  7142 raft_consensus.cc:740] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.398448  7142 consensus_queue.cc:260] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.401364  7144 ts_tablet_manager.cc:1434] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.002s
I20260709 12:21:54.403116  7144 tablet_bootstrap.cc:492] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:54.408439  6850 catalog_manager.cc:5697] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.381680  7153 leader_election.cc:290] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.392659  7143 raft_consensus.cc:740] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.402089  7166 raft_consensus.cc:2749] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.411145  7143 consensus_queue.cc:260] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:54.413694  7143 ts_tablet_manager.cc:1434] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.001s
I20260709 12:21:54.405433  7142 ts_tablet_manager.cc:1434] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:21:54.414633  7144 tablet_bootstrap.cc:654] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.415112  7143 tablet_bootstrap.cc:492] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:54.417199  7142 tablet_bootstrap.cc:492] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:54.426431  7153 raft_consensus.cc:493] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.426954  7142 tablet_bootstrap.cc:654] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.427155  7153 raft_consensus.cc:515] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.427675  7143 tablet_bootstrap.cc:654] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.430713  7144 tablet_bootstrap.cc:492] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:54.430940  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:54.431478  7144 ts_tablet_manager.cc:1403] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.004s
I20260709 12:21:54.431825  6953 raft_consensus.cc:2468] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:54.433475  6991 leader_election.cc:304] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.434451  7170 raft_consensus.cc:2804] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.434830  7170 raft_consensus.cc:493] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.435168  7170 raft_consensus.cc:3060] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.435088  7144 raft_consensus.cc:359] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.435921  7144 raft_consensus.cc:385] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.436224  7144 raft_consensus.cc:740] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.436429  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.437242  7102 raft_consensus.cc:2468] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:54.437201  7144 consensus_queue.cc:260] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.439092  7153 leader_election.cc:290] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.442052  7144 ts_tablet_manager.cc:1434] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:21:54.443225  7144 tablet_bootstrap.cc:492] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:54.444909  7170 raft_consensus.cc:515] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.445874  7143 tablet_bootstrap.cc:492] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:54.448853  7143 ts_tablet_manager.cc:1403] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.009s
I20260709 12:21:54.449985  7144 tablet_bootstrap.cc:654] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.450755  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:54.451464  6953 raft_consensus.cc:3060] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.452215  7143 raft_consensus.cc:359] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.453043  7143 raft_consensus.cc:385] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.453416  7143 raft_consensus.cc:740] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.454325  7143 consensus_queue.cc:260] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.458212  7143 ts_tablet_manager.cc:1434] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:54.459412  7143 tablet_bootstrap.cc:492] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:54.460470  6953 raft_consensus.cc:2468] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:54.462029  6991 leader_election.cc:304] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.462370  7170 leader_election.cc:290] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.463001  7170 raft_consensus.cc:2804] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.463667  7170 raft_consensus.cc:697] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:54.464126  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:54.464922  7102 raft_consensus.cc:3060] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.464546  7170 consensus_queue.cc:237] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.469165  7144 tablet_bootstrap.cc:492] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:54.469866  7144 ts_tablet_manager.cc:1403] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.007s
I20260709 12:21:54.472462  7144 raft_consensus.cc:359] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.472999  7144 raft_consensus.cc:385] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.473156  7143 tablet_bootstrap.cc:654] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.473382  7144 raft_consensus.cc:740] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.448020  7142 tablet_bootstrap.cc:492] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:54.474131  7102 raft_consensus.cc:2468] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:54.474781  7142 ts_tablet_manager.cc:1403] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.004s
I20260709 12:21:54.477262  7144 consensus_queue.cc:260] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.478938  7152 raft_consensus.cc:493] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.478443  7142 raft_consensus.cc:359] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.480952  7142 raft_consensus.cc:385] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.480850  7152 raft_consensus.cc:515] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:54.481367  7142 raft_consensus.cc:740] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.482345  7142 consensus_queue.cc:260] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.483485  6849 catalog_manager.cc:5697] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.480022  7144 ts_tablet_manager.cc:1434] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:21:54.479260  7143 tablet_bootstrap.cc:492] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:54.488795  7144 tablet_bootstrap.cc:492] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:54.484505  7152 leader_election.cc:290] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:54.485273  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b9ed72120b44d39a1260902d5371a9" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:54.486653  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b9ed72120b44d39a1260902d5371a9" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:54.489873  7028 raft_consensus.cc:2468] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 0.
I20260709 12:21:54.491307  6953 raft_consensus.cc:2468] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 0.
I20260709 12:21:54.488982  7143 ts_tablet_manager.cc:1403] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.000s
I20260709 12:21:54.492683  7065 leader_election.cc:304] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.493680  7152 raft_consensus.cc:2804] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.494151  7152 raft_consensus.cc:493] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.494539  7152 raft_consensus.cc:3060] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.495507  7144 tablet_bootstrap.cc:654] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.496027  7143 raft_consensus.cc:359] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.484887  7142 ts_tablet_manager.cc:1434] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:21:54.497901  7142 tablet_bootstrap.cc:492] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:54.503851  7143 raft_consensus.cc:385] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.504319  7143 raft_consensus.cc:740] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.504484  7142 tablet_bootstrap.cc:654] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.505334  7143 consensus_queue.cc:260] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.507694  7143 ts_tablet_manager.cc:1434] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:21:54.508888  7143 tablet_bootstrap.cc:492] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:54.503779  7152 raft_consensus.cc:515] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:54.515358  7170 raft_consensus.cc:493] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.515933  7170 raft_consensus.cc:515] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.516333  7143 tablet_bootstrap.cc:654] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.517230  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b9ed72120b44d39a1260902d5371a9" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:54.517921  6953 raft_consensus.cc:3060] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.518314  7142 tablet_bootstrap.cc:492] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:54.518812  7142 ts_tablet_manager.cc:1403] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 12:21:54.521821  7142 raft_consensus.cc:359] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.522611  7142 raft_consensus.cc:385] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.522886  7142 raft_consensus.cc:740] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.518190  7170 leader_election.cc:290] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.523607  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcdfaa3c489a4693a721d0417519e71b" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.524467  7102 raft_consensus.cc:2468] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:54.526057  6991 leader_election.cc:304] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:54.527506  7170 raft_consensus.cc:2804] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.527648  6952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcdfaa3c489a4693a721d0417519e71b" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:54.527974  7170 raft_consensus.cc:493] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.528350  7170 raft_consensus.cc:3060] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:54.529248  6991 leader_election.cc:343] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.532286  7152 leader_election.cc:290] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:54.532605  6953 raft_consensus.cc:2468] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 1.
I20260709 12:21:54.533602  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b9ed72120b44d39a1260902d5371a9" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:54.533929  7065 leader_election.cc:304] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.534348  7028 raft_consensus.cc:3060] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.534924  7152 raft_consensus.cc:2804] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.536347  7152 raft_consensus.cc:697] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Becoming Leader. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:21:54.536679  7170 raft_consensus.cc:515] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.537282  7152 consensus_queue.cc:237] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:54.523684  7142 consensus_queue.cc:260] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.542405  7142 ts_tablet_manager.cc:1434] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:21:54.543898  7142 tablet_bootstrap.cc:492] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:54.546536  7155 raft_consensus.cc:493] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.547228  7155 raft_consensus.cc:515] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.550814  7027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:54.551179  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
W20260709 12:21:54.553174  6916 leader_election.cc:343] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.551582  7155 leader_election.cc:290] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.555892  7170 leader_election.cc:290] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.557204  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcdfaa3c489a4693a721d0417519e71b" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:54.558089  7102 raft_consensus.cc:3060] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.557693  6849 catalog_manager.cc:5697] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d reported cstate change: term changed from 0 to 1, leader changed from <none> to 615cf24a2e7843d88c0371f123d9101d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "615cf24a2e7843d88c0371f123d9101d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.561115  7142 tablet_bootstrap.cc:654] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.562788  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcdfaa3c489a4693a721d0417519e71b" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
W20260709 12:21:54.564613  6991 leader_election.cc:343] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.551668  7027 raft_consensus.cc:2468] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:54.567108  6918 leader_election.cc:304] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 615cf24a2e7843d88c0371f123d9101d
I20260709 12:21:54.568037  7155 raft_consensus.cc:2804] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.568414  7155 raft_consensus.cc:493] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.568733  7155 raft_consensus.cc:3060] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.576234  7028 raft_consensus.cc:2468] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 1.
I20260709 12:21:54.578822  7155 raft_consensus.cc:515] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.579236  7144 tablet_bootstrap.cc:492] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:54.580586  7144 ts_tablet_manager.cc:1403] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.092s	user 0.019s	sys 0.018s
I20260709 12:21:54.581821  7143 tablet_bootstrap.cc:492] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:54.582583  7143 ts_tablet_manager.cc:1403] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.074s	user 0.029s	sys 0.020s
I20260709 12:21:54.583983  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:54.583966  7144 raft_consensus.cc:359] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.585086  7144 raft_consensus.cc:385] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.585466  7144 raft_consensus.cc:740] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.585289  7155 leader_election.cc:290] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.585894  7143 raft_consensus.cc:359] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.585990  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:54.586403  7144 consensus_queue.cc:260] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.586923  7143 raft_consensus.cc:385] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.587309  7028 raft_consensus.cc:3060] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.587458  7143 raft_consensus.cc:740] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.588670  7143 consensus_queue.cc:260] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.589165  7101 raft_consensus.cc:3060] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.591683  7143 ts_tablet_manager.cc:1434] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:21:54.594340  7102 raft_consensus.cc:2468] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:54.595774  7142 tablet_bootstrap.cc:492] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:54.595911  6991 leader_election.cc:304] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:54.596539  7142 ts_tablet_manager.cc:1403] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.053s	user 0.034s	sys 0.006s
I20260709 12:21:54.597246  7170 raft_consensus.cc:2804] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.597836  7170 raft_consensus.cc:697] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:54.598731  7170 consensus_queue.cc:237] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.599736  7101 raft_consensus.cc:2468] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:54.599867  7142 raft_consensus.cc:359] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.600773  7142 raft_consensus.cc:385] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.601163  7142 raft_consensus.cc:740] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.602043  7142 consensus_queue.cc:260] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.604328  7142 ts_tablet_manager.cc:1434] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:21:54.604431  7144 ts_tablet_manager.cc:1434] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:21:54.607707  7144 tablet_bootstrap.cc:492] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:54.612732  7028 raft_consensus.cc:2468] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:54.614558  6918 leader_election.cc:304] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.615940  7157 raft_consensus.cc:2804] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.616461  7157 raft_consensus.cc:697] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:54.617380  7157 consensus_queue.cc:237] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.623826  7144 tablet_bootstrap.cc:654] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.628033  6850 catalog_manager.cc:5697] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.630908  7144 tablet_bootstrap.cc:492] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:54.631556  7144 ts_tablet_manager.cc:1403] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.004s
I20260709 12:21:54.634601  7144 raft_consensus.cc:359] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.635272  7144 raft_consensus.cc:385] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.635555  7144 raft_consensus.cc:740] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.636286  7144 consensus_queue.cc:260] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.638365  7144 ts_tablet_manager.cc:1434] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:54.638054  6849 catalog_manager.cc:5697] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.639480  7144 tablet_bootstrap.cc:492] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:54.645532  7144 tablet_bootstrap.cc:654] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.651657  7157 consensus_queue.cc:1048] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:54.652539  7144 tablet_bootstrap.cc:492] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:54.653064  7144 ts_tablet_manager.cc:1403] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 12:21:54.655710  7144 raft_consensus.cc:359] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.656292  7144 raft_consensus.cc:385] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.656631  7144 raft_consensus.cc:740] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.657359  7144 consensus_queue.cc:260] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.659514  7144 ts_tablet_manager.cc:1434] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:21:54.667227  7157 consensus_queue.cc:1048] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:54.702987  7170 raft_consensus.cc:493] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.703611  7170 raft_consensus.cc:515] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.705646  7170 leader_election.cc:290] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.706537  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ba6f81b56943a88a91bdc549a91976" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:54.706734  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ba6f81b56943a88a91bdc549a91976" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.707209  6953 raft_consensus.cc:2468] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:54.707405  7101 raft_consensus.cc:2468] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:54.708225  6991 leader_election.cc:304] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.708928  7170 raft_consensus.cc:2804] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.709242  7170 raft_consensus.cc:493] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.709578  7170 raft_consensus.cc:3060] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.715984  7170 raft_consensus.cc:515] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.717213  7157 raft_consensus.cc:493] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.717818  7170 leader_election.cc:290] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.717789  7157 raft_consensus.cc:515] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.718920  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ba6f81b56943a88a91bdc549a91976" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:54.718843  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ba6f81b56943a88a91bdc549a91976" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:54.719684  6953 raft_consensus.cc:3060] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.719904  7157 leader_election.cc:290] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.720750  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:54.720932  7101 raft_consensus.cc:3060] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.721040  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.721443  7028 raft_consensus.cc:2468] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:54.721896  7102 raft_consensus.cc:2468] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:54.722514  6918 leader_election.cc:304] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.723359  7157 raft_consensus.cc:2804] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.723757  7157 raft_consensus.cc:493] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.724097  7157 raft_consensus.cc:3060] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.727914  6953 raft_consensus.cc:2468] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:54.728168  7101 raft_consensus.cc:2468] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:54.729310  6991 leader_election.cc:304] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:54.730172  7170 raft_consensus.cc:2804] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.730655  7170 raft_consensus.cc:697] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:54.731393  7170 consensus_queue.cc:237] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.731825  7157 raft_consensus.cc:515] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.733677  7157 leader_election.cc:290] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.734623  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:54.734968  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:54.735339  7028 raft_consensus.cc:3060] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.737617  7101 raft_consensus.cc:3060] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.739948  6850 catalog_manager.cc:5697] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.745645  7101 raft_consensus.cc:2468] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:54.745645  7028 raft_consensus.cc:2468] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:54.747177  6918 leader_election.cc:304] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.748231  7157 raft_consensus.cc:2804] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.748642  7157 raft_consensus.cc:697] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:54.749473  7157 consensus_queue.cc:237] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.756556  6850 catalog_manager.cc:5697] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.767050  7157 raft_consensus.cc:493] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.767567  7157 raft_consensus.cc:515] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.769443  7157 leader_election.cc:290] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.770216  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcdfaa3c489a4693a721d0417519e71b" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:54.770469  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcdfaa3c489a4693a721d0417519e71b" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.771163  7101 raft_consensus.cc:2393] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90de1cd7a5db4da3bf6d6eb2faade974 in current term 1: Already voted for candidate 378f6aabcd114831bf096f436d7ca046 in this term.
I20260709 12:21:54.771977  6916 leader_election.cc:304] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d
I20260709 12:21:54.772644  7157 raft_consensus.cc:3060] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.778010  7155 raft_consensus.cc:493] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.778666  7157 raft_consensus.cc:2749] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.778579  7155 raft_consensus.cc:515] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.780691  7155 leader_election.cc:290] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.781515  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f812b8f6840f46bcb4c4d2a551fc24d1" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:54.781761  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f812b8f6840f46bcb4c4d2a551fc24d1" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:54.782155  7028 raft_consensus.cc:2468] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:54.782398  7101 raft_consensus.cc:2468] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:54.783111  6918 leader_election.cc:304] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.783831  7155 raft_consensus.cc:2804] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.784214  7155 raft_consensus.cc:493] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.784464  7155 raft_consensus.cc:3060] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.790237  7155 raft_consensus.cc:515] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.791981  7155 leader_election.cc:290] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:54.792793  7028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f812b8f6840f46bcb4c4d2a551fc24d1" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:54.793041  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f812b8f6840f46bcb4c4d2a551fc24d1" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:54.793352  7028 raft_consensus.cc:3060] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.793622  7101 raft_consensus.cc:3060] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.799345  7028 raft_consensus.cc:2468] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:54.799726  7101 raft_consensus.cc:2468] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:54.800472  6918 leader_election.cc:304] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:54.801296  7155 raft_consensus.cc:2804] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.801760  7155 raft_consensus.cc:697] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:54.802412  7155 consensus_queue.cc:237] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:54.809926  6850 catalog_manager.cc:5697] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.931214  7170 consensus_queue.cc:1048] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:54.948231  7153 consensus_queue.cc:1048] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:54.948247  7102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20b9ed72120b44d39a1260902d5371a9"
dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
mode: GRACEFUL
new_leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:35272
I20260709 12:21:54.950263  7102 raft_consensus.cc:606] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Received request to transfer leadership to TS 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:54.971236  7028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
mode: GRACEFUL
new_leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:48318
I20260709 12:21:54.971997  7028 raft_consensus.cc:606] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Received request to transfer leadership to TS 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:54.981643  7028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7daed065ac5140b2988d4a834806283a"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
mode: GRACEFUL
new_leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:48318
I20260709 12:21:54.982277  7028 raft_consensus.cc:606] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Received request to transfer leadership to TS 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:54.993894  7153 consensus_queue.cc:1048] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:54.998580  6952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
mode: GRACEFUL
new_leader_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:48212
I20260709 12:21:54.999393  6952 raft_consensus.cc:606] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Received request to transfer leadership to TS 378f6aabcd114831bf096f436d7ca046
I20260709 12:21:55.008477  7170 consensus_queue.cc:1048] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.009683  6952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
mode: GRACEFUL
new_leader_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:48212
I20260709 12:21:55.010533  6952 raft_consensus.cc:606] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Received request to transfer leadership to TS 378f6aabcd114831bf096f436d7ca046
I20260709 12:21:55.024773  6952 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f812b8f6840f46bcb4c4d2a551fc24d1"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
mode: GRACEFUL
new_leader_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:48212
I20260709 12:21:55.025411  6952 raft_consensus.cc:606] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Received request to transfer leadership to TS 378f6aabcd114831bf096f436d7ca046
I20260709 12:21:55.029217  7170 consensus_queue.cc:1048] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.033195  7028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7ba6f81b56943a88a91bdc549a91976"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
mode: GRACEFUL
new_leader_uuid: "615cf24a2e7843d88c0371f123d9101d"
 from {username='slave'} at 127.0.0.1:48318
I20260709 12:21:55.033567  7155 consensus_queue.cc:1048] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.034097  7028 raft_consensus.cc:606] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Received request to transfer leadership to TS 615cf24a2e7843d88c0371f123d9101d
I20260709 12:21:55.038897  7155 consensus_queue.cc:1048] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.046515  7153 consensus_queue.cc:1048] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.048981  7152 consensus_queue.cc:1048] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.055408  7157 raft_consensus.cc:993] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974: : Instructing follower 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:55.056039  7157 raft_consensus.cc:1081] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Signalling peer 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:55.062204  7026 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:48292
I20260709 12:21:55.063019  7026 raft_consensus.cc:493] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:55.063470  7026 raft_consensus.cc:3060] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.071667  7153 raft_consensus.cc:993] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046: : Instructing follower 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:55.072340  7153 raft_consensus.cc:1081] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Signalling peer 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:55.076531  6952 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:48202
I20260709 12:21:55.077294  6952 raft_consensus.cc:493] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:55.077811  6952 raft_consensus.cc:3060] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.079439  7026 raft_consensus.cc:515] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.080849  7025 raft_consensus.cc:1240] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Rejecting Update request from peer 90de1cd7a5db4da3bf6d6eb2faade974 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:55.083060  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:55.083891  6953 raft_consensus.cc:3055] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.084343  6953 raft_consensus.cc:740] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:55.085417  7026 leader_election.cc:290] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:55.085301  6953 consensus_queue.cc:260] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.087040  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:55.087706  7102 raft_consensus.cc:3060] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.088008  6953 raft_consensus.cc:3060] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.098110  7102 raft_consensus.cc:2468] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:55.100960  6952 raft_consensus.cc:515] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.104367  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:55.105068  7025 raft_consensus.cc:3055] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.105403  7025 raft_consensus.cc:740] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:55.105610  6991 leader_election.cc:304] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:55.106976  7170 raft_consensus.cc:2804] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:55.106526  7025 consensus_queue.cc:260] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.107518  7170 raft_consensus.cc:697] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:55.108119  7025 raft_consensus.cc:3060] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.108428  7170 consensus_queue.cc:237] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.116109  7215 consensus_queue.cc:1048] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.118887  6953 raft_consensus.cc:2468] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:55.123154  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:55.123998  7101 raft_consensus.cc:3060] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.116619  6952 leader_election.cc:290] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:55.129135  6848 catalog_manager.cc:5697] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 1 to 2, leader changed from 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1) to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.134492  7101 raft_consensus.cc:2468] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:55.136543  7025 raft_consensus.cc:2468] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:55.138981  7215 raft_consensus.cc:993] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d: : Instructing follower 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:55.140587  6918 leader_election.cc:304] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:55.141084  7215 raft_consensus.cc:1081] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Signalling peer 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:55.141820  7146 raft_consensus.cc:2804] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:55.142349  7146 raft_consensus.cc:697] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:55.143146  6953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20b9ed72120b44d39a1260902d5371a9"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:48208
I20260709 12:21:55.144021  6953 raft_consensus.cc:493] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:55.143294  7146 consensus_queue.cc:237] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.144469  6953 raft_consensus.cc:3060] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.154781  6953 raft_consensus.cc:515] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:55.156664  6953 leader_election.cc:290] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Requested vote from peers 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:55.157850  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b9ed72120b44d39a1260902d5371a9" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:55.158015  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20b9ed72120b44d39a1260902d5371a9" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:55.158759  7101 raft_consensus.cc:3055] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.158859  7025 raft_consensus.cc:3060] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.159132  7101 raft_consensus.cc:740] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:21:55.160157  7101 consensus_queue.cc:260] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:55.161791  7101 raft_consensus.cc:3060] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.162248  6849 catalog_manager.cc:5697] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 1 to 2, leader changed from 378f6aabcd114831bf096f436d7ca046 (127.26.123.2) to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.166335  7025 raft_consensus.cc:2468] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:55.168120  6918 leader_election.cc:304] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:55.169020  7146 raft_consensus.cc:2804] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:55.169469  7146 raft_consensus.cc:697] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:55.170341  7101 raft_consensus.cc:2468] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:55.170485  7146 consensus_queue.cc:237] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } }
I20260709 12:21:55.180071  6849 catalog_manager.cc:5697] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 1 to 2, leader changed from 615cf24a2e7843d88c0371f123d9101d (127.26.123.3) to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.191015  7146 consensus_queue.cc:1048] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.204138  7146 consensus_queue.cc:1048] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.214376  7157 raft_consensus.cc:993] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974: : Instructing follower 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:55.214874  7146 raft_consensus.cc:1081] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Signalling peer 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:55.217397  7025 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:48292
I20260709 12:21:55.218226  7025 raft_consensus.cc:493] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:55.218627  7025 raft_consensus.cc:3060] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.219300  7170 consensus_queue.cc:1048] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.227615  7153 raft_consensus.cc:993] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046: : Instructing follower 615cf24a2e7843d88c0371f123d9101d to start an election
I20260709 12:21:55.227586  7025 raft_consensus.cc:515] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.228467  7153 raft_consensus.cc:1081] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Signalling peer 615cf24a2e7843d88c0371f123d9101d to start an election
I20260709 12:21:55.230463  7025 leader_election.cc:290] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:55.232677  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:55.233130  7100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35cb8325a0024767ad2aeabad1666a60" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:55.233191  7102 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7ba6f81b56943a88a91bdc549a91976"
dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
 from {username='slave'} at 127.0.0.1:35270
I20260709 12:21:55.233541  6953 raft_consensus.cc:3055] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.234056  6953 raft_consensus.cc:740] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:55.234196  7100 raft_consensus.cc:3060] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.234066  7102 raft_consensus.cc:493] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:55.235152  7102 raft_consensus.cc:3060] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.235102  6953 consensus_queue.cc:260] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.236743  6953 raft_consensus.cc:3060] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.243263  7157 consensus_queue.cc:1048] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.244987  7100 raft_consensus.cc:2468] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:55.251569  6991 leader_election.cc:304] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:55.252730  7153 raft_consensus.cc:2804] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:55.253304  7153 raft_consensus.cc:697] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:55.254411  7153 consensus_queue.cc:237] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.255775  7102 raft_consensus.cc:515] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.256860  6953 raft_consensus.cc:2468] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:55.253193  7170 consensus_queue.cc:1048] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:21:55.267710  6849 catalog_manager.cc:5697] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 1 to 2, leader changed from 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1) to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.268951  7102 leader_election.cc:290] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 2 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:55.273145  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ba6f81b56943a88a91bdc549a91976" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:55.274183  7025 raft_consensus.cc:3055] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.274567  7025 raft_consensus.cc:740] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:55.275635  7025 consensus_queue.cc:260] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.276824  7025 raft_consensus.cc:3060] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.291527  6952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ba6f81b56943a88a91bdc549a91976" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:55.292454  6952 raft_consensus.cc:3060] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.295794  7146 consensus_queue.cc:1048] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:55.298689  7025 raft_consensus.cc:2468] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 2.
I20260709 12:21:55.300662  7067 leader_election.cc:304] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:55.301838  7152 raft_consensus.cc:2804] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:55.302408  7152 raft_consensus.cc:697] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 2 LEADER]: Becoming Leader. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:21:55.303120  6952 raft_consensus.cc:2468] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 2.
I20260709 12:21:55.303375  7152 consensus_queue.cc:237] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.304111  6953 raft_consensus.cc:1240] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Rejecting Update request from peer 378f6aabcd114831bf096f436d7ca046 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:21:55.318800  7146 raft_consensus.cc:993] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974: : Instructing follower 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:55.318393  6848 catalog_manager.cc:5697] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d reported cstate change: term changed from 1 to 2, leader changed from 378f6aabcd114831bf096f436d7ca046 (127.26.123.2) to 615cf24a2e7843d88c0371f123d9101d (127.26.123.3). New cstate: current_term: 2 leader_uuid: "615cf24a2e7843d88c0371f123d9101d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:55.319496  7155 raft_consensus.cc:1081] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Signalling peer 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:55.321663  7025 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f812b8f6840f46bcb4c4d2a551fc24d1"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:48292
I20260709 12:21:55.322352  7025 raft_consensus.cc:493] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:55.322774  7025 raft_consensus.cc:3060] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.330799  7025 raft_consensus.cc:515] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.332962  7025 leader_election.cc:290] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:55.333912  6952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f812b8f6840f46bcb4c4d2a551fc24d1" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:55.334336  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f812b8f6840f46bcb4c4d2a551fc24d1" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:55.334749  6952 raft_consensus.cc:3055] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.335038  7099 raft_consensus.cc:3060] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.335160  6952 raft_consensus.cc:740] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:55.336134  6952 consensus_queue.cc:260] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.337322  6952 raft_consensus.cc:3060] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.343247  7099 raft_consensus.cc:2468] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:55.343917  6952 raft_consensus.cc:2468] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:55.344679  6991 leader_election.cc:304] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:55.345508  7153 raft_consensus.cc:2804] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:55.345979  7153 raft_consensus.cc:697] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:55.346863  7153 consensus_queue.cc:237] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.354179  6848 catalog_manager.cc:5697] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 1 to 2, leader changed from 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1) to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.538772  7170 raft_consensus.cc:993] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046: : Instructing follower 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:55.539342  7153 raft_consensus.cc:1081] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Signalling peer 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:55.540848  6952 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7daed065ac5140b2988d4a834806283a"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:48202
I20260709 12:21:55.541445  6952 raft_consensus.cc:493] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:55.541867  6952 raft_consensus.cc:3060] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.548378  6952 raft_consensus.cc:515] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.550367  6952 leader_election.cc:290] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:55.551378  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7daed065ac5140b2988d4a834806283a" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:55.551697  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7daed065ac5140b2988d4a834806283a" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:55.552158  7025 raft_consensus.cc:3055] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:55.552286  7099 raft_consensus.cc:3060] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.552476  7025 raft_consensus.cc:740] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:55.553372  7025 consensus_queue.cc:260] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.554522  7025 raft_consensus.cc:3060] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:55.560221  7099 raft_consensus.cc:2468] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:55.561700  6916 leader_election.cc:304] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:55.562218  7025 raft_consensus.cc:2468] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:55.562628  7155 raft_consensus.cc:2804] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:55.563138  7155 raft_consensus.cc:697] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:55.563911  7155 consensus_queue.cc:237] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:55.571463  7099 raft_consensus.cc:1275] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.571413  6848 catalog_manager.cc:5697] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 1 to 2, leader changed from 378f6aabcd114831bf096f436d7ca046 (127.26.123.2) to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.573201  7155 consensus_queue.cc:1048] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.576835  7102 raft_consensus.cc:1275] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.578826  7153 consensus_queue.cc:1048] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.586473  7025 raft_consensus.cc:1275] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.590310  7155 consensus_queue.cc:1048] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.596578  6952 raft_consensus.cc:1275] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.599645  7153 consensus_queue.cc:1048] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.625993  7025 raft_consensus.cc:1275] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.627771  7155 consensus_queue.cc:1048] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.637655  7102 raft_consensus.cc:1275] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.639990  7146 consensus_queue.cc:1048] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.684202  6952 raft_consensus.cc:1275] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Refusing update from remote peer 615cf24a2e7843d88c0371f123d9101d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.685643  7152 consensus_queue.cc:1048] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.696781  7025 raft_consensus.cc:1275] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Refusing update from remote peer 615cf24a2e7843d88c0371f123d9101d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.698889  7152 consensus_queue.cc:1048] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:55.777895  6952 raft_consensus.cc:1275] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.779484  7153 consensus_queue.cc:1048] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:55.782735  7102 raft_consensus.cc:1275] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.784608  7153 consensus_queue.cc:1048] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.880105  7102 raft_consensus.cc:1275] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.881544  7153 consensus_queue.cc:1048] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:55.889820  6952 raft_consensus.cc:1275] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:55.892467  7153 consensus_queue.cc:1048] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.140935  7025 raft_consensus.cc:1275] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:56.142403  7146 consensus_queue.cc:1048] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:56.154472  7102 raft_consensus.cc:1275] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:56.156190  7227 consensus_queue.cc:1048] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:58.072260  6850 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48188:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:21:58.074662  6850 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:21:58.128798  6941 tablet_service.cc:1511] Processing CreateTablet for tablet 8ef03d76aee04b3ca1ed0203616c5510 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:58.130424  6941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef03d76aee04b3ca1ed0203616c5510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.130057  7014 tablet_service.cc:1511] Processing CreateTablet for tablet 8ef03d76aee04b3ca1ed0203616c5510 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:58.131134  7089 tablet_service.cc:1511] Processing CreateTablet for tablet 8ef03d76aee04b3ca1ed0203616c5510 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:21:58.132578  7089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef03d76aee04b3ca1ed0203616c5510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.135812  7013 tablet_service.cc:1511] Processing CreateTablet for tablet 7897cdd045b54a56b67292d58c9a1528 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:58.136585  7091 tablet_service.cc:1511] Processing CreateTablet for tablet 7897cdd045b54a56b67292d58c9a1528 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:58.138027  7091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7897cdd045b54a56b67292d58c9a1528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.139087  7018 tablet_service.cc:1511] Processing CreateTablet for tablet cc4a77ec797341359459f9036e8984ed (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:58.139887  6943 tablet_service.cc:1511] Processing CreateTablet for tablet 7897cdd045b54a56b67292d58c9a1528 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:21:58.140318  6940 tablet_service.cc:1511] Processing CreateTablet for tablet cc4a77ec797341359459f9036e8984ed (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:58.131567  7014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef03d76aee04b3ca1ed0203616c5510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.141436  6943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7897cdd045b54a56b67292d58c9a1528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.147192  7092 tablet_service.cc:1511] Processing CreateTablet for tablet cc4a77ec797341359459f9036e8984ed (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:21:58.153059  6940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4a77ec797341359459f9036e8984ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.153148  7092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4a77ec797341359459f9036e8984ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.153441  7018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4a77ec797341359459f9036e8984ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.155692  7013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7897cdd045b54a56b67292d58c9a1528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.159001  6942 tablet_service.cc:1511] Processing CreateTablet for tablet 1e875b8b9fb640daa6d3db4b5f421dec (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:58.160717  6942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e875b8b9fb640daa6d3db4b5f421dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.181562  7015 tablet_service.cc:1511] Processing CreateTablet for tablet 1e875b8b9fb640daa6d3db4b5f421dec (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:58.186964  7015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e875b8b9fb640daa6d3db4b5f421dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.184199  7090 tablet_service.cc:1511] Processing CreateTablet for tablet 1e875b8b9fb640daa6d3db4b5f421dec (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:21:58.191444  7090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e875b8b9fb640daa6d3db4b5f421dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.200098  6939 tablet_service.cc:1511] Processing CreateTablet for tablet 9654410a8c924c168ded45f76f55c7b9 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:58.202831  7017 tablet_service.cc:1511] Processing CreateTablet for tablet 9654410a8c924c168ded45f76f55c7b9 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:58.204419  7017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9654410a8c924c168ded45f76f55c7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.204434  7088 tablet_service.cc:1511] Processing CreateTablet for tablet 9654410a8c924c168ded45f76f55c7b9 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:21:58.208112  6938 tablet_service.cc:1511] Processing CreateTablet for tablet 6fa7e02e304f48bead9b2711a3de29f0 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:58.209668  7088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9654410a8c924c168ded45f76f55c7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.212752  6939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9654410a8c924c168ded45f76f55c7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.213696  6938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa7e02e304f48bead9b2711a3de29f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.244540  7016 tablet_service.cc:1511] Processing CreateTablet for tablet 6fa7e02e304f48bead9b2711a3de29f0 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:58.245725  7087 tablet_service.cc:1511] Processing CreateTablet for tablet 6fa7e02e304f48bead9b2711a3de29f0 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:21:58.247352  7087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa7e02e304f48bead9b2711a3de29f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.246289  7016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa7e02e304f48bead9b2711a3de29f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.255038  7244 tablet_bootstrap.cc:492] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:58.260785  7012 tablet_service.cc:1511] Processing CreateTablet for tablet 3bf2e5ad820b4050a741334066504e8f (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:58.262442  7012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bf2e5ad820b4050a741334066504e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.276072  7245 tablet_bootstrap.cc:492] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:58.270098  7089 tablet_service.cc:1511] Processing CreateTablet for tablet 3bf2e5ad820b4050a741334066504e8f (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:58.280526  7089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bf2e5ad820b4050a741334066504e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.269341  6940 tablet_service.cc:1511] Processing CreateTablet for tablet 3bf2e5ad820b4050a741334066504e8f (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:21:58.285017  7244 tablet_bootstrap.cc:654] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.266894  7246 tablet_bootstrap.cc:492] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:58.286635  7013 tablet_service.cc:1511] Processing CreateTablet for tablet 839e0ad151884d3587e3686965c0eda4 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:58.288259  7013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839e0ad151884d3587e3686965c0eda4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.290170  6940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bf2e5ad820b4050a741334066504e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.290891  6938 tablet_service.cc:1511] Processing CreateTablet for tablet 839e0ad151884d3587e3686965c0eda4 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:58.292325  6938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839e0ad151884d3587e3686965c0eda4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.297988  7246 tablet_bootstrap.cc:654] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.310014  7245 tablet_bootstrap.cc:654] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.311697  7018 tablet_service.cc:1511] Processing CreateTablet for tablet 9b75463bdf994b82934830b5ef7daef5 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:58.312166  6942 tablet_service.cc:1511] Processing CreateTablet for tablet 9b75463bdf994b82934830b5ef7daef5 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:58.316519  6942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b75463bdf994b82934830b5ef7daef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.318573  7090 tablet_service.cc:1511] Processing CreateTablet for tablet 839e0ad151884d3587e3686965c0eda4 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:21:58.320461  7091 tablet_service.cc:1511] Processing CreateTablet for tablet 9b75463bdf994b82934830b5ef7daef5 (DEFAULT_TABLE table=soft_deleted_table [id=324168d38c1146fd919194e9d85585e1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:21:58.327943  7018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b75463bdf994b82934830b5ef7daef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.320842  7090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839e0ad151884d3587e3686965c0eda4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:58.330041  7091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b75463bdf994b82934830b5ef7daef5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:58.347577  6899 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:58.376930  7245 tablet_bootstrap.cc:492] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:58.377665  7245 ts_tablet_manager.cc:1403] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.102s	user 0.005s	sys 0.009s
I20260709 12:21:58.379379  7244 tablet_bootstrap.cc:492] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:58.380177  7244 ts_tablet_manager.cc:1403] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.125s	user 0.020s	sys 0.027s
I20260709 12:21:58.381873  7246 tablet_bootstrap.cc:492] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:58.381381  7245 raft_consensus.cc:359] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.386600  7245 raft_consensus.cc:385] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.387096  7245 raft_consensus.cc:740] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.388100  7245 consensus_queue.cc:260] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.390833  7245 ts_tablet_manager.cc:1434] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:21:58.394757  7245 tablet_bootstrap.cc:492] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:58.386691  7246 ts_tablet_manager.cc:1403] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.012s
I20260709 12:21:58.390434  7244 raft_consensus.cc:359] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.401986  7244 raft_consensus.cc:385] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.402345  7244 raft_consensus.cc:740] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.401365  7246 raft_consensus.cc:359] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.403644  7246 raft_consensus.cc:385] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.404012  7246 raft_consensus.cc:740] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.405100  7244 consensus_queue.cc:260] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.408560  7246 consensus_queue.cc:260] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.414652  7233 raft_consensus.cc:493] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.415246  7233 raft_consensus.cc:515] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.417635  7244 ts_tablet_manager.cc:1434] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.006s
I20260709 12:21:58.418972  7244 tablet_bootstrap.cc:492] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:58.424595  7245 tablet_bootstrap.cc:654] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.425729  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:58.425855  7233 leader_election.cc:290] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:58.431248  7244 tablet_bootstrap.cc:654] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:58.433245  7065 leader_election.cc:343] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.442022  7244 tablet_bootstrap.cc:492] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:58.441655  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:58.442840  7244 ts_tablet_manager.cc:1403] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.003s
I20260709 12:21:58.442883  7246 ts_tablet_manager.cc:1434] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.004s
I20260709 12:21:58.442760  7025 raft_consensus.cc:2468] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 0.
I20260709 12:21:58.444559  7246 tablet_bootstrap.cc:492] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:58.445255  7245 tablet_bootstrap.cc:492] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:58.445968  7245 ts_tablet_manager.cc:1403] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.006s
I20260709 12:21:58.447067  7244 raft_consensus.cc:359] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.448071  7244 raft_consensus.cc:385] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.448474  7244 raft_consensus.cc:740] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.447417  7067 leader_election.cc:304] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:58.449348  7244 consensus_queue.cc:260] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.450151  7251 raft_consensus.cc:2804] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.449343  7245 raft_consensus.cc:359] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.452370  7244 ts_tablet_manager.cc:1434] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:21:58.452445  7245 raft_consensus.cc:385] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.452585  7246 tablet_bootstrap.cc:654] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.453047  7245 raft_consensus.cc:740] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.453655  7244 tablet_bootstrap.cc:492] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:58.450798  7251 raft_consensus.cc:493] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.454260  7245 consensus_queue.cc:260] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.455080  7251 raft_consensus.cc:3060] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.457103  7245 ts_tablet_manager.cc:1434] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:21:58.458455  7245 tablet_bootstrap.cc:492] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:58.461588  7244 tablet_bootstrap.cc:654] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.469211  7245 tablet_bootstrap.cc:654] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.475786  7251 raft_consensus.cc:515] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.480602  7251 leader_election.cc:290] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:58.482829  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:58.483657  7025 raft_consensus.cc:3060] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.483623  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
W20260709 12:21:58.487125  7065 leader_election.cc:343] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.493460  7025 raft_consensus.cc:2468] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 1.
I20260709 12:21:58.495226  7067 leader_election.cc:304] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:58.495630  7244 tablet_bootstrap.cc:492] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:58.496085  7251 raft_consensus.cc:2804] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.497637  7251 raft_consensus.cc:697] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Becoming Leader. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:21:58.498078  7246 tablet_bootstrap.cc:492] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:58.498673  7246 ts_tablet_manager.cc:1403] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.015s
I20260709 12:21:58.496311  7244 ts_tablet_manager.cc:1403] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.018s
I20260709 12:21:58.499051  7251 consensus_queue.cc:237] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.498754  7245 tablet_bootstrap.cc:492] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:58.501686  7245 ts_tablet_manager.cc:1403] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.008s
I20260709 12:21:58.502292  7246 raft_consensus.cc:359] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.503156  7246 raft_consensus.cc:385] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.503518  7246 raft_consensus.cc:740] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.504724  7246 consensus_queue.cc:260] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.507117  7246 ts_tablet_manager.cc:1434] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:21:58.506964  7244 raft_consensus.cc:359] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.508139  7244 raft_consensus.cc:385] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.508481  7244 raft_consensus.cc:740] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.508536  7246 tablet_bootstrap.cc:492] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:58.505193  7245 raft_consensus.cc:359] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.509366  7245 raft_consensus.cc:385] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.509701  7245 raft_consensus.cc:740] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.510737  7245 consensus_queue.cc:260] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.512066  7244 consensus_queue.cc:260] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.514765  7244 ts_tablet_manager.cc:1434] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 12:21:58.515980  7244 tablet_bootstrap.cc:492] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:58.517553  7246 tablet_bootstrap.cc:654] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.523433  7245 ts_tablet_manager.cc:1434] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:21:58.523763  7246 tablet_bootstrap.cc:492] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:58.524410  7246 ts_tablet_manager.cc:1403] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:21:58.524874  7245 tablet_bootstrap.cc:492] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:58.524044  6850 catalog_manager.cc:5697] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d reported cstate change: term changed from 0 to 1, leader changed from <none> to 615cf24a2e7843d88c0371f123d9101d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "615cf24a2e7843d88c0371f123d9101d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:58.532230  7246 raft_consensus.cc:359] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.533234  7246 raft_consensus.cc:385] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.533536  7246 raft_consensus.cc:740] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.533763  7245 tablet_bootstrap.cc:654] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.534533  7246 consensus_queue.cc:260] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.535413  7244 tablet_bootstrap.cc:654] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.537775  7246 ts_tablet_manager.cc:1434] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.013s	user 0.000s	sys 0.007s
I20260709 12:21:58.539191  7246 tablet_bootstrap.cc:492] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:58.551244  7246 tablet_bootstrap.cc:654] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.556494  7231 raft_consensus.cc:493] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.557308  7231 raft_consensus.cc:515] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.562330  7231 leader_election.cc:290] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.562754  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a77ec797341359459f9036e8984ed" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:58.563755  7102 raft_consensus.cc:2468] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:58.566004  6991 leader_election.cc:304] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:58.570614  7263 raft_consensus.cc:2804] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.571092  7263 raft_consensus.cc:493] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.571439  7263 raft_consensus.cc:3060] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.585695  7146 raft_consensus.cc:493] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.586472  7146 raft_consensus.cc:515] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.590517  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef03d76aee04b3ca1ed0203616c5510" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:58.591486  7102 raft_consensus.cc:2468] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:58.593180  6916 leader_election.cc:304] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:58.594704  7146 leader_election.cc:290] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.595575  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef03d76aee04b3ca1ed0203616c5510" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:58.596487  7146 raft_consensus.cc:2804] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.596966  7146 raft_consensus.cc:493] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.597321  7146 raft_consensus.cc:3060] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:58.590718  7257 log.cc:927] Time spent T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:21:58.602224  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a77ec797341359459f9036e8984ed" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:58.603278  6953 raft_consensus.cc:2468] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
W20260709 12:21:58.606235  6918 leader_election.cc:343] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.614971  7244 tablet_bootstrap.cc:492] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:58.615799  7244 ts_tablet_manager.cc:1403] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.100s	user 0.032s	sys 0.046s
I20260709 12:21:58.616904  7246 tablet_bootstrap.cc:492] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:58.617601  7246 ts_tablet_manager.cc:1403] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.079s	user 0.029s	sys 0.025s
I20260709 12:21:58.618341  7263 raft_consensus.cc:515] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.619611  7244 raft_consensus.cc:359] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.620396  7245 tablet_bootstrap.cc:492] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:58.620529  7244 raft_consensus.cc:385] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.621914  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a77ec797341359459f9036e8984ed" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:58.622707  6953 raft_consensus.cc:3060] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.623560  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a77ec797341359459f9036e8984ed" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:58.624270  7102 raft_consensus.cc:3060] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.621044  7245 ts_tablet_manager.cc:1403] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.096s	user 0.031s	sys 0.017s
I20260709 12:21:58.620988  7246 raft_consensus.cc:359] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.629047  7246 raft_consensus.cc:385] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.629403  7246 raft_consensus.cc:740] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.630453  7246 consensus_queue.cc:260] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.621155  7244 raft_consensus.cc:740] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.632086  7263 leader_election.cc:290] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.633613  7246 ts_tablet_manager.cc:1434] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.005s
I20260709 12:21:58.632109  7244 consensus_queue.cc:260] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.634948  7246 tablet_bootstrap.cc:492] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:58.636869  7244 ts_tablet_manager.cc:1434] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 12:21:58.638968  7244 tablet_bootstrap.cc:492] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:58.639144  7245 raft_consensus.cc:359] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.637629  6953 raft_consensus.cc:2468] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:58.640029  7245 raft_consensus.cc:385] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.640410  7245 raft_consensus.cc:740] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.641461  7245 consensus_queue.cc:260] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.642112  6991 leader_election.cc:304] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:58.643712  7231 raft_consensus.cc:2804] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.644116  7245 ts_tablet_manager.cc:1434] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:21:58.644207  7231 raft_consensus.cc:697] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:58.645363  7245 tablet_bootstrap.cc:492] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:58.630591  7146 raft_consensus.cc:515] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.645571  7231 consensus_queue.cc:237] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.648115  7146 leader_election.cc:290] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.650861  7244 tablet_bootstrap.cc:654] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.651141  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef03d76aee04b3ca1ed0203616c5510" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:58.651855  7099 raft_consensus.cc:3060] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.654783  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef03d76aee04b3ca1ed0203616c5510" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
W20260709 12:21:58.656610  6918 leader_election.cc:343] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:58.659579  7245 tablet_bootstrap.cc:654] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.669013  7099 raft_consensus.cc:2468] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:58.673105  6916 leader_election.cc:304] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 378f6aabcd114831bf096f436d7ca046
I20260709 12:21:58.673410  7244 tablet_bootstrap.cc:492] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:58.674196  7244 ts_tablet_manager.cc:1403] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.010s
I20260709 12:21:58.676577  7102 raft_consensus.cc:2468] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:58.675328  7246 tablet_bootstrap.cc:654] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.675544  7146 raft_consensus.cc:2804] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.679450  7146 raft_consensus.cc:697] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:58.680580  7146 consensus_queue.cc:237] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.686875  7244 raft_consensus.cc:359] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.687834  7244 raft_consensus.cc:385] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.688128  7244 raft_consensus.cc:740] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.688948  7244 consensus_queue.cc:260] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.691275  7245 tablet_bootstrap.cc:492] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:58.691967  7245 ts_tablet_manager.cc:1403] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.010s
I20260709 12:21:58.700477  7244 ts_tablet_manager.cc:1434] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.004s
I20260709 12:21:58.702790  7246 tablet_bootstrap.cc:492] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:58.702905  7244 tablet_bootstrap.cc:492] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:58.703130  7245 raft_consensus.cc:359] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.704053  7245 raft_consensus.cc:385] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.704373  7245 raft_consensus.cc:740] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.705346  7245 consensus_queue.cc:260] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.707988  7245 ts_tablet_manager.cc:1434] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.004s
I20260709 12:21:58.709178  7245 tablet_bootstrap.cc:492] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:58.703560  7246 ts_tablet_manager.cc:1403] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.069s	user 0.015s	sys 0.000s
I20260709 12:21:58.710320  6850 catalog_manager.cc:5697] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:58.711687  7244 tablet_bootstrap.cc:654] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.713305  7246 raft_consensus.cc:359] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.714203  7246 raft_consensus.cc:385] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.714547  7246 raft_consensus.cc:740] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.715569  7246 consensus_queue.cc:260] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.716506  7245 tablet_bootstrap.cc:654] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.725090  7246 ts_tablet_manager.cc:1434] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:21:58.725960  7244 tablet_bootstrap.cc:492] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:58.726584  7244 ts_tablet_manager.cc:1403] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.007s
I20260709 12:21:58.726760  7246 tablet_bootstrap.cc:492] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:58.728005  6848 catalog_manager.cc:5697] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:58.730188  7244 raft_consensus.cc:359] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.735028  7246 tablet_bootstrap.cc:654] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.735193  7244 raft_consensus.cc:385] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.737339  7244 raft_consensus.cc:740] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.738332  7244 consensus_queue.cc:260] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.742002  7244 ts_tablet_manager.cc:1434] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.003s
I20260709 12:21:58.743216  7244 tablet_bootstrap.cc:492] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:58.747361  7246 tablet_bootstrap.cc:492] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:58.748126  7246 ts_tablet_manager.cc:1403] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 12:21:58.750447  7261 raft_consensus.cc:493] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.751405  7244 tablet_bootstrap.cc:654] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.751111  7261 raft_consensus.cc:515] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.753556  7261 leader_election.cc:290] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.755532  7246 raft_consensus.cc:359] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.756502  7246 raft_consensus.cc:385] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.757117  7246 raft_consensus.cc:740] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.758002  7246 consensus_queue.cc:260] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.757124  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:58.759486  7025 raft_consensus.cc:2393] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90de1cd7a5db4da3bf6d6eb2faade974 in current term 1: Already voted for candidate 615cf24a2e7843d88c0371f123d9101d in this term.
I20260709 12:21:58.761058  7246 ts_tablet_manager.cc:1434] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.003s
I20260709 12:21:58.762331  7246 tablet_bootstrap.cc:492] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:58.769583  7246 tablet_bootstrap.cc:654] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.779445  7231 raft_consensus.cc:493] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.780256  7231 raft_consensus.cc:515] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.755688  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:58.785600  7246 tablet_bootstrap.cc:492] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:58.785769  6916 leader_election.cc:304] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d
I20260709 12:21:58.786422  7246 ts_tablet_manager.cc:1403] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.024s	user 0.003s	sys 0.016s
I20260709 12:21:58.787214  7146 raft_consensus.cc:3060] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.790113  7261 raft_consensus.cc:493] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.789937  7246 raft_consensus.cc:359] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.790803  7261 raft_consensus.cc:515] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.791968  7246 raft_consensus.cc:385] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.792356  7246 raft_consensus.cc:740] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.793151  7246 consensus_queue.cc:260] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.793879  7261 leader_election.cc:290] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.795425  7246 ts_tablet_manager.cc:1434] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:21:58.795199  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e875b8b9fb640daa6d3db4b5f421dec" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:58.796134  7102 raft_consensus.cc:2468] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:58.797529  6916 leader_election.cc:304] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:58.798314  7245 tablet_bootstrap.cc:492] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:58.798465  7261 raft_consensus.cc:2804] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.799264  7261 raft_consensus.cc:493] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.798833  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e875b8b9fb640daa6d3db4b5f421dec" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:58.799736  7261 raft_consensus.cc:3060] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.800294  7025 raft_consensus.cc:2468] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:58.804441  7146 raft_consensus.cc:2749] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:58.805769  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9654410a8c924c168ded45f76f55c7b9" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:58.798952  7245 ts_tablet_manager.cc:1403] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.090s	user 0.007s	sys 0.011s
I20260709 12:21:58.799460  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9654410a8c924c168ded45f76f55c7b9" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:58.807330  6953 raft_consensus.cc:2468] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:58.808899  6991 leader_election.cc:304] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:58.810334  7245 raft_consensus.cc:359] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.811152  7263 raft_consensus.cc:2804] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.811523  7245 raft_consensus.cc:385] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.811616  7263 raft_consensus.cc:493] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.811887  7245 raft_consensus.cc:740] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.812079  7263 raft_consensus.cc:3060] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.815764  7261 raft_consensus.cc:515] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.816174  7245 consensus_queue.cc:260] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.815660  7246 tablet_bootstrap.cc:492] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:58.806660  7102 raft_consensus.cc:2468] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:58.822301  7261 leader_election.cc:290] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.825924  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e875b8b9fb640daa6d3db4b5f421dec" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:58.826877  7102 raft_consensus.cc:3060] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.830377  7246 tablet_bootstrap.cc:654] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.830796  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e875b8b9fb640daa6d3db4b5f421dec" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:58.831607  7025 raft_consensus.cc:3060] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.836196  7244 tablet_bootstrap.cc:492] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:58.837092  7244 ts_tablet_manager.cc:1403] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.094s	user 0.036s	sys 0.038s
I20260709 12:21:58.822880  7231 leader_election.cc:290] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.846212  7245 ts_tablet_manager.cc:1434] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.039s	user 0.019s	sys 0.013s
I20260709 12:21:58.848510  7244 raft_consensus.cc:359] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.850244  7244 raft_consensus.cc:385] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.850605  7244 raft_consensus.cc:740] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.851470  7244 consensus_queue.cc:260] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.853098  7263 raft_consensus.cc:515] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.847641  7231 raft_consensus.cc:493] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.847888  7245 tablet_bootstrap.cc:492] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:58.857409  7244 ts_tablet_manager.cc:1434] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.010s
I20260709 12:21:58.857713  7231 raft_consensus.cc:515] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.858950  7244 tablet_bootstrap.cc:492] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:58.861014  7231 leader_election.cc:290] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.861459  7263 leader_election.cc:290] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.864105  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef03d76aee04b3ca1ed0203616c5510" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:58.865187  7099 raft_consensus.cc:2393] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 378f6aabcd114831bf096f436d7ca046 in current term 1: Already voted for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in this term.
I20260709 12:21:58.866475  7025 raft_consensus.cc:2468] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:58.867802  7245 tablet_bootstrap.cc:654] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.867913  6918 leader_election.cc:304] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:58.868997  7261 raft_consensus.cc:2804] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.869506  7261 raft_consensus.cc:697] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:58.867733  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef03d76aee04b3ca1ed0203616c5510" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:58.870501  7261 consensus_queue.cc:237] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.869603  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9654410a8c924c168ded45f76f55c7b9" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:58.872445  7099 raft_consensus.cc:3060] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.888325  7102 raft_consensus.cc:2468] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:58.889762  7244 tablet_bootstrap.cc:654] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.891016  7246 tablet_bootstrap.cc:492] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:58.891682  7246 ts_tablet_manager.cc:1403] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.076s	user 0.017s	sys 0.001s
I20260709 12:21:58.894169  7263 raft_consensus.cc:493] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.894855  7263 raft_consensus.cc:515] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.897320  7263 leader_election.cc:290] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.899281  7246 raft_consensus.cc:359] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.900234  7246 raft_consensus.cc:385] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.900452  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9654410a8c924c168ded45f76f55c7b9" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:58.901212  6953 raft_consensus.cc:3060] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.901391  7099 raft_consensus.cc:2468] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:58.900530  7246 raft_consensus.cc:740] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.903754  7246 consensus_queue.cc:260] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.902760  6991 leader_election.cc:304] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046; no voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:58.908226  7263 raft_consensus.cc:3060] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.909464  6991 leader_election.cc:304] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:58.910339  6951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa7e02e304f48bead9b2711a3de29f0" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
I20260709 12:21:58.910624  7231 raft_consensus.cc:2804] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:58.911237  7231 raft_consensus.cc:697] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:58.911185  6951 raft_consensus.cc:2468] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:58.912269  7231 consensus_queue.cc:237] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.914073  6991 leader_election.cc:304] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:58.915207  7276 raft_consensus.cc:2804] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:58.915633  7276 raft_consensus.cc:493] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.915980  7276 raft_consensus.cc:3060] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.912148  7250 consensus_queue.cc:1048] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:21:58.924294  7245 tablet_bootstrap.cc:492] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:58.924445  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa7e02e304f48bead9b2711a3de29f0" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:58.925031  7245 ts_tablet_manager.cc:1403] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.078s	user 0.024s	sys 0.015s
I20260709 12:21:58.922766  7246 ts_tablet_manager.cc:1434] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 12:21:58.920837  6953 raft_consensus.cc:2468] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:58.933228  7246 tablet_bootstrap.cc:492] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d: Bootstrap starting.
I20260709 12:21:58.925256  7099 raft_consensus.cc:2468] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 0.
I20260709 12:21:58.954089  7263 raft_consensus.cc:2749] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:58.956604  7245 raft_consensus.cc:359] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.957542  7245 raft_consensus.cc:385] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:58.957873  7245 raft_consensus.cc:740] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:58.958187  6848 catalog_manager.cc:5697] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:58.958938  6850 catalog_manager.cc:5697] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:58.960944  7276 raft_consensus.cc:515] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.963471  7276 leader_election.cc:290] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:58.965345  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa7e02e304f48bead9b2711a3de29f0" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:58.958775  7245 consensus_queue.cc:260] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:58.966889  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa7e02e304f48bead9b2711a3de29f0" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:58.966295  6953 raft_consensus.cc:3060] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.968708  7245 ts_tablet_manager.cc:1434] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.003s
I20260709 12:21:58.970054  7245 tablet_bootstrap.cc:492] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:58.972687  7246 tablet_bootstrap.cc:654] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.973225  7099 raft_consensus.cc:3060] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:58.980248  7245 tablet_bootstrap.cc:654] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:58.992370  7244 tablet_bootstrap.cc:492] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:58.995555  7246 tablet_bootstrap.cc:492] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d: No bootstrap required, opened a new log
I20260709 12:21:58.996348  7246 ts_tablet_manager.cc:1403] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d: Time spent bootstrapping tablet: real 0.066s	user 0.011s	sys 0.006s
I20260709 12:21:58.992064  6953 raft_consensus.cc:2468] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:59.000564  7251 consensus_queue.cc:1048] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:58.993053  7244 ts_tablet_manager.cc:1403] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.134s	user 0.031s	sys 0.004s
I20260709 12:21:59.000713  7246 raft_consensus.cc:359] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.001603  7246 raft_consensus.cc:385] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.001971  7246 raft_consensus.cc:740] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.002753  7246 consensus_queue.cc:260] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.004081  7244 raft_consensus.cc:359] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.004907  7244 raft_consensus.cc:385] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.005194  7244 raft_consensus.cc:740] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.005421  7246 ts_tablet_manager.cc:1434] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:21:59.011075  7244 consensus_queue.cc:260] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.014071  7244 ts_tablet_manager.cc:1434] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 12:21:59.015177  7244 tablet_bootstrap.cc:492] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974: Bootstrap starting.
I20260709 12:21:59.018968  6991 leader_election.cc:304] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:59.020248  7276 raft_consensus.cc:2804] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:59.020910  7276 raft_consensus.cc:697] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:59.021869  7276 consensus_queue.cc:237] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.022897  7250 raft_consensus.cc:493] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.023525  7250 raft_consensus.cc:515] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.025622  7245 tablet_bootstrap.cc:492] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:59.026263  7245 ts_tablet_manager.cc:1403] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.028s
I20260709 12:21:59.027083  6952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" is_pre_election: true
W20260709 12:21:59.029232  7065 leader_election.cc:343] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:59.031236  7250 leader_election.cc:290] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:59.033419  7245 raft_consensus.cc:359] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.034375  7245 raft_consensus.cc:385] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.034672  7245 raft_consensus.cc:740] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.035493  7026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:59.035449  7245 consensus_queue.cc:260] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.036286  7026 raft_consensus.cc:2468] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 0.
I20260709 12:21:59.038427  7263 consensus_queue.cc:1048] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.038944  7244 tablet_bootstrap.cc:654] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:59.041778  7245 ts_tablet_manager.cc:1434] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:21:59.042945  7245 tablet_bootstrap.cc:492] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046: Bootstrap starting.
I20260709 12:21:59.040932  7067 leader_election.cc:304] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:59.045277  7251 raft_consensus.cc:2804] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:59.045919  7251 raft_consensus.cc:493] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.046268  7251 raft_consensus.cc:3060] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.053565  7099 raft_consensus.cc:2468] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 1.
I20260709 12:21:59.054976  7245 tablet_bootstrap.cc:654] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:59.063215  7251 raft_consensus.cc:515] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.062682  6849 catalog_manager.cc:5697] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 0 to 1, leader changed from <none> to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 1 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.066963  6951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:59.067296  7251 leader_election.cc:290] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
W20260709 12:21:59.068701  7065 leader_election.cc:343] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:59.071241  7231 consensus_queue.cc:1048] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.086568  7026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:59.087466  7026 raft_consensus.cc:3060] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.095952  7244 tablet_bootstrap.cc:492] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974: No bootstrap required, opened a new log
I20260709 12:21:59.096577  7244 ts_tablet_manager.cc:1403] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent bootstrapping tablet: real 0.082s	user 0.020s	sys 0.002s
I20260709 12:21:59.098929  7244 raft_consensus.cc:359] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.099510  7244 raft_consensus.cc:385] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.099716  7026 raft_consensus.cc:2468] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 1.
I20260709 12:21:59.099741  7244 raft_consensus.cc:740] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.101079  7244 consensus_queue.cc:260] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.101522  7067 leader_election.cc:304] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:59.102526  7245 tablet_bootstrap.cc:492] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046: No bootstrap required, opened a new log
I20260709 12:21:59.102538  7251 raft_consensus.cc:2804] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:59.103217  7251 raft_consensus.cc:697] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Becoming Leader. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:21:59.103861  7244 ts_tablet_manager.cc:1434] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:21:59.104892  7245 ts_tablet_manager.cc:1403] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.026s
I20260709 12:21:59.104108  7251 consensus_queue.cc:237] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.108395  7261 raft_consensus.cc:493] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.108968  7261 raft_consensus.cc:515] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.110987  7261 leader_election.cc:290] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:59.107997  7245 raft_consensus.cc:359] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.112365  7245 raft_consensus.cc:385] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:59.112709  7245 raft_consensus.cc:740] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.113073  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839e0ad151884d3587e3686965c0eda4" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:59.113967  7099 raft_consensus.cc:2468] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:59.114260  7026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839e0ad151884d3587e3686965c0eda4" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:59.115505  6916 leader_election.cc:304] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:59.116292  7261 raft_consensus.cc:2804] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:59.116622  7261 raft_consensus.cc:493] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.116899  7261 raft_consensus.cc:3060] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.117117  7245 consensus_queue.cc:260] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.120318  7245 ts_tablet_manager.cc:1434] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:21:59.123958  7026 raft_consensus.cc:2468] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:59.123651  6850 catalog_manager.cc:5697] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d reported cstate change: term changed from 0 to 1, leader changed from <none> to 615cf24a2e7843d88c0371f123d9101d (127.26.123.3). New cstate: current_term: 1 leader_uuid: "615cf24a2e7843d88c0371f123d9101d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.127240  7261 raft_consensus.cc:515] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.129864  7261 leader_election.cc:290] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:59.131224  7026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839e0ad151884d3587e3686965c0eda4" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:59.131381  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839e0ad151884d3587e3686965c0eda4" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:59.131968  7026 raft_consensus.cc:3060] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.132066  7099 raft_consensus.cc:3060] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.140010  7099 raft_consensus.cc:2468] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:59.140704  7261 raft_consensus.cc:493] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.140479  7026 raft_consensus.cc:2468] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:59.141386  6916 leader_election.cc:304] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:59.141353  7261 raft_consensus.cc:515] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.142462  7264 raft_consensus.cc:2804] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:59.142988  7264 raft_consensus.cc:697] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:59.144155  7261 leader_election.cc:290] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:59.143834  7264 consensus_queue.cc:237] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.144450  7026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75463bdf994b82934830b5ef7daef5" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:59.145200  7026 raft_consensus.cc:2468] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:59.145407  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75463bdf994b82934830b5ef7daef5" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:59.146186  7099 raft_consensus.cc:2468] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 0.
I20260709 12:21:59.146456  6918 leader_election.cc:304] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:59.147364  7261 raft_consensus.cc:2804] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:59.147822  7261 raft_consensus.cc:493] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.148157  7261 raft_consensus.cc:3060] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.154527  6850 catalog_manager.cc:5697] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.156922  7261 raft_consensus.cc:515] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.160197  7026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75463bdf994b82934830b5ef7daef5" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:59.160320  7261 leader_election.cc:290] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:59.160526  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75463bdf994b82934830b5ef7daef5" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:59.161203  7099 raft_consensus.cc:3060] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.161041  7026 raft_consensus.cc:3060] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.170375  7099 raft_consensus.cc:2468] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:59.171365  7026 raft_consensus.cc:2468] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 1.
I20260709 12:21:59.171748  6916 leader_election.cc:304] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:59.172690  7261 raft_consensus.cc:2804] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:59.173110  7261 raft_consensus.cc:697] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:59.173919  7261 consensus_queue.cc:237] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.183347  6850 catalog_manager.cc:5697] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 1 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.249171  7261 consensus_queue.cc:1048] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.262130  7261 consensus_queue.cc:1048] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.304915  7099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528"
dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
mode: GRACEFUL
new_leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:35272
I20260709 12:21:59.305505  7099 raft_consensus.cc:606] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Received request to transfer leadership to TS 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:59.309533  7261 consensus_queue.cc:1048] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.311197  7026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc4a77ec797341359459f9036e8984ed"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
mode: GRACEFUL
new_leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:48318
I20260709 12:21:59.312034  7026 raft_consensus.cc:606] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Received request to transfer leadership to TS 90de1cd7a5db4da3bf6d6eb2faade974
I20260709 12:21:59.324219  7146 consensus_queue.cc:1048] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.337402  7099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f"
dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
mode: GRACEFUL
new_leader_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:35272
I20260709 12:21:59.338145  7099 raft_consensus.cc:606] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Received request to transfer leadership to TS 378f6aabcd114831bf096f436d7ca046
I20260709 12:21:59.344686  6951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "839e0ad151884d3587e3686965c0eda4"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
mode: GRACEFUL
new_leader_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:48212
I20260709 12:21:59.345528  6951 raft_consensus.cc:606] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Received request to transfer leadership to TS 378f6aabcd114831bf096f436d7ca046
I20260709 12:21:59.351361  6951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b75463bdf994b82934830b5ef7daef5"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
mode: GRACEFUL
new_leader_uuid: "615cf24a2e7843d88c0371f123d9101d"
 from {username='slave'} at 127.0.0.1:48212
I20260709 12:21:59.351949  6951 raft_consensus.cc:606] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Received request to transfer leadership to TS 615cf24a2e7843d88c0371f123d9101d
I20260709 12:21:59.445783  7276 consensus_queue.cc:1048] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.457690  7231 consensus_queue.cc:1048] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.459236  7275 consensus_queue.cc:1048] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.480672  7263 consensus_queue.cc:1048] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.513940  7146 raft_consensus.cc:493] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.514446  7146 raft_consensus.cc:515] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.516484  7146 leader_election.cc:290] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:59.517236  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378f6aabcd114831bf096f436d7ca046" is_pre_election: true
I20260709 12:21:59.517647  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615cf24a2e7843d88c0371f123d9101d" is_pre_election: true
I20260709 12:21:59.518261  7025 raft_consensus.cc:2393] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90de1cd7a5db4da3bf6d6eb2faade974 in current term 1: Already voted for candidate 615cf24a2e7843d88c0371f123d9101d in this term.
I20260709 12:21:59.519356  6918 leader_election.cc:304] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d
I20260709 12:21:59.520036  7146 raft_consensus.cc:3060] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:59.526566  7146 raft_consensus.cc:2749] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:59.537611  7146 consensus_queue.cc:1048] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.544526  7264 raft_consensus.cc:993] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974: : Instructing follower 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:59.545042  7264 raft_consensus.cc:1081] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Signalling peer 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:59.547945  7026 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "839e0ad151884d3587e3686965c0eda4"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:48292
I20260709 12:21:59.548612  7026 raft_consensus.cc:493] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:59.549011  7026 raft_consensus.cc:3060] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.551930  7261 consensus_queue.cc:1048] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.559168  7251 consensus_queue.cc:1048] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.559515  7026 raft_consensus.cc:515] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.560710  7025 raft_consensus.cc:1240] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Rejecting Update request from peer 90de1cd7a5db4da3bf6d6eb2faade974 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:59.564385  7261 consensus_queue.cc:1059] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.565335  7261 raft_consensus.cc:3055] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:59.565672  7261 raft_consensus.cc:740] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:59.566599  7261 consensus_queue.cc:260] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.567576  6951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839e0ad151884d3587e3686965c0eda4" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:59.568611  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839e0ad151884d3587e3686965c0eda4" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:59.569279  7099 raft_consensus.cc:3060] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.569725  7261 raft_consensus.cc:3060] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.572245  7305 consensus_queue.cc:1048] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.576277  7026 leader_election.cc:290] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:59.581115  7099 raft_consensus.cc:2468] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:59.583346  6991 leader_election.cc:304] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:59.584353  7263 raft_consensus.cc:2804] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:59.584857  7263 raft_consensus.cc:697] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:59.588315  6951 raft_consensus.cc:2468] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:59.585840  7263 consensus_queue.cc:237] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.592063  7305 raft_consensus.cc:993] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d: : Instructing follower 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:59.596172  7250 raft_consensus.cc:1081] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Signalling peer 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:59.599138  6951 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:48208
I20260709 12:21:59.600033  7305 raft_consensus.cc:993] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d: : Instructing follower 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:59.600930  7250 raft_consensus.cc:1081] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Signalling peer 378f6aabcd114831bf096f436d7ca046 to start an election
I20260709 12:21:59.599782  6951 raft_consensus.cc:493] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:59.602450  6951 raft_consensus.cc:3060] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.603063  7027 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
 from {username='slave'} at 127.0.0.1:48304
I20260709 12:21:59.603653  7027 raft_consensus.cc:493] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:59.603972  7027 raft_consensus.cc:3060] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.608450  6848 catalog_manager.cc:5697] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 1 to 2, leader changed from 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1) to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.610430  7027 raft_consensus.cc:515] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.611025  6951 raft_consensus.cc:515] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.614090  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:59.614346  7100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:59.615054  7100 raft_consensus.cc:3055] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:59.614724  7099 raft_consensus.cc:3055] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:59.615417  7100 raft_consensus.cc:740] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:21:59.615852  7099 raft_consensus.cc:740] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:21:59.616653  7100 consensus_queue.cc:260] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.616756  7099 consensus_queue.cc:260] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.617563  7100 raft_consensus.cc:3060] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.615231  7025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7897cdd045b54a56b67292d58c9a1528" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:59.618357  7025 raft_consensus.cc:3060] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.615231  6953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bf2e5ad820b4050a741334066504e8f" candidate_uuid: "378f6aabcd114831bf096f436d7ca046" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:59.618918  6953 raft_consensus.cc:3060] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.621086  7261 consensus_queue.cc:1048] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:59.625837  6951 leader_election.cc:290] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:59.628721  7025 raft_consensus.cc:2468] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:59.630096  7099 raft_consensus.cc:3060] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.630236  6918 leader_election.cc:304] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:59.631050  7100 raft_consensus.cc:2468] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:59.631126  7261 raft_consensus.cc:2804] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:59.631755  7261 raft_consensus.cc:697] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:59.632616  7261 consensus_queue.cc:237] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.637471  7027 leader_election.cc:290] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:59.638417  7099 raft_consensus.cc:2468] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:59.636607  6953 raft_consensus.cc:2468] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378f6aabcd114831bf096f436d7ca046 in term 2.
I20260709 12:21:59.643925  7311 consensus_queue.cc:1048] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:59.646267  6991 leader_election.cc:304] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:59.647817  7275 raft_consensus.cc:993] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046: : Instructing follower 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:59.647181  7263 raft_consensus.cc:2804] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:59.648377  7231 raft_consensus.cc:1081] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Signalling peer 90de1cd7a5db4da3bf6d6eb2faade974 to start an election
I20260709 12:21:59.648758  7263 raft_consensus.cc:697] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Becoming Leader. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:59.649665  7263 consensus_queue.cc:237] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.650074  6849 catalog_manager.cc:5697] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 1 to 2, leader changed from 615cf24a2e7843d88c0371f123d9101d (127.26.123.3) to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.654695  6951 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc4a77ec797341359459f9036e8984ed"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
 from {username='slave'} at 127.0.0.1:48202
I20260709 12:21:59.655462  6951 raft_consensus.cc:493] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:59.655846  6951 raft_consensus.cc:3060] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.661475  6848 catalog_manager.cc:5697] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 reported cstate change: term changed from 1 to 2, leader changed from 615cf24a2e7843d88c0371f123d9101d (127.26.123.3) to 378f6aabcd114831bf096f436d7ca046 (127.26.123.2). New cstate: current_term: 2 leader_uuid: "378f6aabcd114831bf096f436d7ca046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.663106  6951 raft_consensus.cc:515] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.666934  7026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a77ec797341359459f9036e8984ed" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:59.667570  7026 raft_consensus.cc:3055] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:59.667883  7026 raft_consensus.cc:740] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:21:59.668447  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc4a77ec797341359459f9036e8984ed" candidate_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
I20260709 12:21:59.668901  7026 consensus_queue.cc:260] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.669076  7102 raft_consensus.cc:3060] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.670295  7026 raft_consensus.cc:3060] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.674521  6951 leader_election.cc:290] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Requested vote from peers 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857), 615cf24a2e7843d88c0371f123d9101d (127.26.123.3:40969)
I20260709 12:21:59.678785  7102 raft_consensus.cc:2468] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:59.680200  6916 leader_election.cc:304] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:21:59.681134  7311 raft_consensus.cc:2804] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:59.683207  7026 raft_consensus.cc:2468] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90de1cd7a5db4da3bf6d6eb2faade974 in term 2.
I20260709 12:21:59.683916  7311 raft_consensus.cc:697] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Becoming Leader. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:59.685336  7146 raft_consensus.cc:993] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974: : Instructing follower 615cf24a2e7843d88c0371f123d9101d to start an election
I20260709 12:21:59.685828  7146 raft_consensus.cc:1081] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Signalling peer 615cf24a2e7843d88c0371f123d9101d to start an election
I20260709 12:21:59.685883  7311 consensus_queue.cc:237] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.687467  7100 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b75463bdf994b82934830b5ef7daef5"
dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
 from {username='slave'} at 127.0.0.1:35264
I20260709 12:21:59.687979  7100 raft_consensus.cc:493] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:59.688274  7100 raft_consensus.cc:3060] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.694351  6848 catalog_manager.cc:5697] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 reported cstate change: term changed from 1 to 2, leader changed from 378f6aabcd114831bf096f436d7ca046 (127.26.123.2) to 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1). New cstate: current_term: 2 leader_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.696372  7100 raft_consensus.cc:515] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.698553  7100 leader_election.cc:290] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 2 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:21:59.700274  7027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75463bdf994b82934830b5ef7daef5" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:21:59.700708  6951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b75463bdf994b82934830b5ef7daef5" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:21:59.700906  7027 raft_consensus.cc:3060] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.701371  6951 raft_consensus.cc:3055] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:59.701720  6951 raft_consensus.cc:740] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:21:59.702579  6951 consensus_queue.cc:260] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.703902  6951 raft_consensus.cc:3060] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.711589  7027 raft_consensus.cc:2468] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 2.
I20260709 12:21:59.711589  6951 raft_consensus.cc:2468] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 2.
I20260709 12:21:59.713006  7067 leader_election.cc:304] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378f6aabcd114831bf096f436d7ca046, 615cf24a2e7843d88c0371f123d9101d; no voters: 
I20260709 12:21:59.713744  7305 raft_consensus.cc:2804] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:59.714181  7305 raft_consensus.cc:697] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 2 LEADER]: Becoming Leader. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:21:59.714841  7305 consensus_queue.cc:237] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:21:59.721877  6848 catalog_manager.cc:5697] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d reported cstate change: term changed from 1 to 2, leader changed from 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1) to 615cf24a2e7843d88c0371f123d9101d (127.26.123.3). New cstate: current_term: 2 leader_uuid: "615cf24a2e7843d88c0371f123d9101d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.979401  7102 raft_consensus.cc:1275] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:59.980700  7263 consensus_queue.cc:1048] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:59.990840  6951 raft_consensus.cc:1275] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:59.992555  7231 consensus_queue.cc:1048] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.043663  6951 raft_consensus.cc:1275] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.044906  7263 consensus_queue.cc:1048] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.052654  7102 raft_consensus.cc:1275] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 378f6aabcd114831bf096f436d7ca046: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.056573  7231 consensus_queue.cc:1048] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.072726  7102 raft_consensus.cc:1275] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.073889  7264 consensus_queue.cc:1048] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.082546  7026 raft_consensus.cc:1275] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.084578  7311 consensus_queue.cc:1048] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.114365  7102 raft_consensus.cc:1275] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.115849  7311 consensus_queue.cc:1048] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.124279  7026 raft_consensus.cc:1275] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Refusing update from remote peer 90de1cd7a5db4da3bf6d6eb2faade974: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.125929  7146 consensus_queue.cc:1048] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.185922  7026 raft_consensus.cc:1275] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Refusing update from remote peer 615cf24a2e7843d88c0371f123d9101d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.187335  7305 consensus_queue.cc:1048] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.195835  6951 raft_consensus.cc:1275] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Refusing update from remote peer 615cf24a2e7843d88c0371f123d9101d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.197925  7305 consensus_queue.cc:1048] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:02.356325  6848 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48188:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:22:02.387030  6848 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48188:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:22:02.420434 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:02.549139  7358 tablet.cc:1724] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.550093  7360 tablet.cc:1724] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:22:02.626438  6991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:39251, user_credentials={real_user=slave}} blocked reactor thread for 62199.8us
I20260709 12:22:02.627745  7371 tablet.cc:1724] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:22:02.631451  6918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.2:46857, user_credentials={real_user=slave}} blocked reactor thread for 65092.8us
I20260709 12:22:02.612303  7362 tablet.cc:1724] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.669507  7358 tablet.cc:1724] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.735911  7380 tablet.cc:1724] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.760342  7370 tablet.cc:1724] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.765233  6953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c"
dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
mode: GRACEFUL
new_leader_uuid: "615cf24a2e7843d88c0371f123d9101d"
 from {username='slave'} at 127.0.0.1:50694
I20260709 12:22:02.766252  6953 raft_consensus.cc:606] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Received request to transfer leadership to TS 615cf24a2e7843d88c0371f123d9101d
I20260709 12:22:02.770839  7360 tablet.cc:1724] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:22:02.797681  6991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.123.1:39251, user_credentials={real_user=slave}} blocked reactor thread for 137187us
I20260709 12:22:02.809357  7387 tablet.cc:1724] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.846694  7380 tablet.cc:1724] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.853144  7362 tablet.cc:1724] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.858987  7392 tablet.cc:1724] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.867889  7358 tablet.cc:1724] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.904188  7344 raft_consensus.cc:993] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974: : Instructing follower 615cf24a2e7843d88c0371f123d9101d to start an election
I20260709 12:22:02.904824  7344 raft_consensus.cc:1081] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Signalling peer 615cf24a2e7843d88c0371f123d9101d to start an election
I20260709 12:22:02.911072  7371 tablet.cc:1724] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.914104  7360 tablet.cc:1724] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.915302  7095 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c"
dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
 from {username='slave'} at 127.0.0.1:35264
I20260709 12:22:02.916182  7095 raft_consensus.cc:493] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:02.916704  7095 raft_consensus.cc:3060] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:02.924178  7385 tablet.cc:1724] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.854099  7389 tablet.cc:1724] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.945830  7390 tablet.cc:1724] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.945864  7391 tablet.cc:1724] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.949304  7380 tablet.cc:1724] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.903139  7370 tablet.cc:1724] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.929329  7358 tablet.cc:1724] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.945250  7392 tablet.cc:1724] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.988111  7387 tablet.cc:1724] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.992354  7027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72"
dest_uuid: "378f6aabcd114831bf096f436d7ca046"
mode: GRACEFUL
new_leader_uuid: "615cf24a2e7843d88c0371f123d9101d"
 from {username='slave'} at 127.0.0.1:36836
I20260709 12:22:02.993280  7027 raft_consensus.cc:606] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Received request to transfer leadership to TS 615cf24a2e7843d88c0371f123d9101d
I20260709 12:22:02.995544 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:02.996395 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:02.968714  7389 tablet.cc:1724] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:02.998662  7095 raft_consensus.cc:515] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:22:03.002555  6949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:22:03.002954  7027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1d3b5a577d46029b259dc75f6dd66c" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:22:03.003506  6949 raft_consensus.cc:3055] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:03.003716  7027 raft_consensus.cc:3060] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.004415  6949 raft_consensus.cc:740] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90de1cd7a5db4da3bf6d6eb2faade974, State: Running, Role: LEADER
I20260709 12:22:03.005718  6949 consensus_queue.cc:260] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:22:03.007364  6949 raft_consensus.cc:3060] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.019670  7387 tablet.cc:1724] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:03.024979  6949 raft_consensus.cc:2468] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 3.
I20260709 12:22:03.026913  7065 leader_election.cc:304] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:22:03.028107  7382 raft_consensus.cc:2804] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:03.028658  7382 raft_consensus.cc:697] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 3 LEADER]: Becoming Leader. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:22:03.029706  7382 consensus_queue.cc:237] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:22:03.038831  7387 tablet.cc:1724] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:22:03.023821  7027 raft_consensus.cc:2468] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 3.
I20260709 12:22:03.042266  7095 leader_election.cc:290] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 3 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:22:03.077342  6849 catalog_manager.cc:5697] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d reported cstate change: term changed from 2 to 3, leader changed from 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1) to 615cf24a2e7843d88c0371f123d9101d (127.26.123.3). New cstate: current_term: 3 leader_uuid: "615cf24a2e7843d88c0371f123d9101d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:03.097970  6849 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5: soft_deleted_table [id=324168d38c1146fd919194e9d85585e1] alter complete (version 1)
I20260709 12:22:03.373392  7373 raft_consensus.cc:993] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046: : Instructing follower 615cf24a2e7843d88c0371f123d9101d to start an election
I20260709 12:22:03.374106  7275 raft_consensus.cc:1081] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Signalling peer 615cf24a2e7843d88c0371f123d9101d to start an election
I20260709 12:22:03.375813  7095 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72"
dest_uuid: "615cf24a2e7843d88c0371f123d9101d"
 from {username='slave'} at 127.0.0.1:35270
I20260709 12:22:03.376335  7095 raft_consensus.cc:493] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:03.376636  7095 raft_consensus.cc:3060] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.383291  7095 raft_consensus.cc:515] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:22:03.385643  7095 leader_election.cc:290] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 3 election: Requested vote from peers 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251), 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857)
I20260709 12:22:03.386119  6949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "90de1cd7a5db4da3bf6d6eb2faade974"
I20260709 12:22:03.386267  7027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07a49706e1d448fb18cb100ecc08e72" candidate_uuid: "615cf24a2e7843d88c0371f123d9101d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "378f6aabcd114831bf096f436d7ca046"
I20260709 12:22:03.387147  6949 raft_consensus.cc:3060] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.387687  7027 raft_consensus.cc:3055] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:03.388095  7027 raft_consensus.cc:740] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 378f6aabcd114831bf096f436d7ca046, State: Running, Role: LEADER
I20260709 12:22:03.388996  7027 consensus_queue.cc:260] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:22:03.390254  7027 raft_consensus.cc:3060] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:03.395637  6949 raft_consensus.cc:2468] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 3.
I20260709 12:22:03.397032  7065 leader_election.cc:304] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 615cf24a2e7843d88c0371f123d9101d, 90de1cd7a5db4da3bf6d6eb2faade974; no voters: 
I20260709 12:22:03.397867  7027 raft_consensus.cc:2468] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 615cf24a2e7843d88c0371f123d9101d in term 3.
I20260709 12:22:03.398054  7319 raft_consensus.cc:2804] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:03.398586  7319 raft_consensus.cc:697] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 3 LEADER]: Becoming Leader. State: Replica: 615cf24a2e7843d88c0371f123d9101d, State: Running, Role: LEADER
I20260709 12:22:03.399588  7319 consensus_queue.cc:237] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } }
I20260709 12:22:03.409130  6849 catalog_manager.cc:5697] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d reported cstate change: term changed from 2 to 3, leader changed from 378f6aabcd114831bf096f436d7ca046 (127.26.123.2) to 615cf24a2e7843d88c0371f123d9101d (127.26.123.3). New cstate: current_term: 3 leader_uuid: "615cf24a2e7843d88c0371f123d9101d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "615cf24a2e7843d88c0371f123d9101d" member_type: VOTER last_known_addr { host: "127.26.123.3" port: 40969 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:03.533344  7027 raft_consensus.cc:1275] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 3 FOLLOWER]: Refusing update from remote peer 615cf24a2e7843d88c0371f123d9101d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:03.535130  7319 consensus_queue.cc:1048] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:03.546845  6949 raft_consensus.cc:1275] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 3 FOLLOWER]: Refusing update from remote peer 615cf24a2e7843d88c0371f123d9101d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:03.548450  7382 consensus_queue.cc:1048] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:03.808701  6949 raft_consensus.cc:1275] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 3 FOLLOWER]: Refusing update from remote peer 615cf24a2e7843d88c0371f123d9101d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:03.809934  7319 consensus_queue.cc:1048] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "90de1cd7a5db4da3bf6d6eb2faade974" member_type: VOTER last_known_addr { host: "127.26.123.1" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:03.819339  7027 raft_consensus.cc:1275] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 3 FOLLOWER]: Refusing update from remote peer 615cf24a2e7843d88c0371f123d9101d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:03.821694  7382 consensus_queue.cc:1048] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [LEADER]: Connected to new peer: Peer: permanent_uuid: "378f6aabcd114831bf096f436d7ca046" member_type: VOTER last_known_addr { host: "127.26.123.2" port: 46857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:03.997622 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:04.003530 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:04.003993 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:05.005311 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:05.012089 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:05.012630 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:06.013823 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:06.019424 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:06.019959 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:07.021085 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:07.026716 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:07.027251 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:08.028506 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:08.034053 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:08.034523 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:09.035686 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:09.042447 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:09.042913 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:10.044123 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:10.050729 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:10.051330 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:11.052417 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:11.057996 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:11.058632 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:12.060235 27116 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:12.066939 27116 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:12.067516 27116 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:13.144428 27116 tablet_server.cc:179] TabletServer@127.26.123.1:0 shutting down...
W20260709 12:22:13.147897  7065 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 13 similar messages]
W20260709 12:22:13.156939  7065 consensus_peers.cc:597] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d -> Peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Couldn't send request to peer 90de1cd7a5db4da3bf6d6eb2faade974. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:13.185238  6991 consensus_peers.cc:597] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 -> Peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Couldn't send request to peer 90de1cd7a5db4da3bf6d6eb2faade974. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.195953 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:13.197243 27116 tablet_replica.cc:333] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.198621 27116 raft_consensus.cc:2243] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.199527 27116 raft_consensus.cc:2272] T 9654410a8c924c168ded45f76f55c7b9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.202829 27116 tablet_replica.cc:333] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.203569 27116 raft_consensus.cc:2243] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.204666 27116 raft_consensus.cc:2272] T 1e875b8b9fb640daa6d3db4b5f421dec P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.207540 27116 tablet_replica.cc:333] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.208292 27116 raft_consensus.cc:2243] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.208884 27116 raft_consensus.cc:2272] T 6fa7e02e304f48bead9b2711a3de29f0 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.213207 27116 tablet_replica.cc:333] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.214118 27116 raft_consensus.cc:2243] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.215279 27116 raft_consensus.cc:2272] T 7897cdd045b54a56b67292d58c9a1528 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.218302 27116 tablet_replica.cc:333] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.218950 27116 raft_consensus.cc:2243] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.220126 27116 raft_consensus.cc:2272] T 789ada7109834532b815f3b0bcb5e69f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.224090 27116 tablet_replica.cc:333] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.225018 27116 raft_consensus.cc:2243] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.225785 27116 raft_consensus.cc:2272] T bcdfaa3c489a4693a721d0417519e71b P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.229931 27116 tablet_replica.cc:333] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.230865 27116 raft_consensus.cc:2243] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.231647 27116 raft_consensus.cc:2272] T b7ba6f81b56943a88a91bdc549a91976 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.234754 27116 tablet_replica.cc:333] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.235528 27116 raft_consensus.cc:2243] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.236080 27116 raft_consensus.cc:2272] T f812b8f6840f46bcb4c4d2a551fc24d1 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.238858 27116 tablet_replica.cc:333] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.239460 27116 raft_consensus.cc:2243] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.240609 27116 raft_consensus.cc:2272] T 8ef03d76aee04b3ca1ed0203616c5510 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.243603 27116 tablet_replica.cc:333] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.244253 27116 raft_consensus.cc:2243] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.244848 27116 raft_consensus.cc:2272] T 839e0ad151884d3587e3686965c0eda4 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.247588 27116 tablet_replica.cc:333] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.248216 27116 raft_consensus.cc:2243] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.248797 27116 raft_consensus.cc:2272] T c07a49706e1d448fb18cb100ecc08e72 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.251453 27116 tablet_replica.cc:333] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.252036 27116 raft_consensus.cc:2243] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.253031 27116 raft_consensus.cc:2272] T cc4a77ec797341359459f9036e8984ed P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.256314 27116 tablet_replica.cc:333] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.257072 27116 raft_consensus.cc:2243] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.258145 27116 raft_consensus.cc:2272] T 20b9ed72120b44d39a1260902d5371a9 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.260993 27116 tablet_replica.cc:333] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.261628 27116 raft_consensus.cc:2243] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.262207 27116 raft_consensus.cc:2272] T 35cb8325a0024767ad2aeabad1666a60 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.264942 27116 tablet_replica.cc:333] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.265556 27116 raft_consensus.cc:2243] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.266225 27116 raft_consensus.cc:2272] T 9b75463bdf994b82934830b5ef7daef5 P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.268870 27116 tablet_replica.cc:333] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.269407 27116 raft_consensus.cc:2243] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.270494 27116 raft_consensus.cc:2272] T 7daed065ac5140b2988d4a834806283a P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.273988 27116 tablet_replica.cc:333] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.274767 27116 raft_consensus.cc:2243] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.275581 27116 raft_consensus.cc:2272] T 3bf2e5ad820b4050a741334066504e8f P 90de1cd7a5db4da3bf6d6eb2faade974 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:13.275439  6991 consensus_peers.cc:597] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 -> Peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Couldn't send request to peer 90de1cd7a5db4da3bf6d6eb2faade974. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.278854 27116 tablet_replica.cc:333] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974: stopping tablet replica
I20260709 12:22:13.279536 27116 raft_consensus.cc:2243] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.280339 27116 raft_consensus.cc:2272] T 8b1d3b5a577d46029b259dc75f6dd66c P 90de1cd7a5db4da3bf6d6eb2faade974 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:13.298202  6991 consensus_peers.cc:597] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 -> Peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Couldn't send request to peer 90de1cd7a5db4da3bf6d6eb2faade974. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:13.319700  6991 consensus_peers.cc:597] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 -> Peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Couldn't send request to peer 90de1cd7a5db4da3bf6d6eb2faade974. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.336884 27116 tablet_server.cc:196] TabletServer@127.26.123.1:0 shutdown complete.
I20260709 12:22:13.392012 27116 tablet_server.cc:179] TabletServer@127.26.123.2:0 shutting down...
W20260709 12:22:13.398480  7067 consensus_peers.cc:597] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d -> Peer 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857): Couldn't send request to peer 378f6aabcd114831bf096f436d7ca046. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.422991 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:13.424088 27116 tablet_replica.cc:333] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
W20260709 12:22:13.424877  7065 consensus_peers.cc:597] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d -> Peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Couldn't send request to peer 90de1cd7a5db4da3bf6d6eb2faade974. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:39251: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.425279 27116 raft_consensus.cc:2243] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.426074 27116 raft_consensus.cc:2272] T 8ef03d76aee04b3ca1ed0203616c5510 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.428788 27116 tablet_replica.cc:333] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.429819 27116 raft_consensus.cc:2243] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.431010 27116 raft_consensus.cc:2272] T 6fa7e02e304f48bead9b2711a3de29f0 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.433584 27116 tablet_replica.cc:333] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.434338 27116 raft_consensus.cc:2243] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.434907 27116 raft_consensus.cc:2272] T 1e875b8b9fb640daa6d3db4b5f421dec P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.437552 27116 tablet_replica.cc:333] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.438198 27116 raft_consensus.cc:2243] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.439167 27116 raft_consensus.cc:2272] T 9654410a8c924c168ded45f76f55c7b9 P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.441668 27116 tablet_replica.cc:333] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.442379 27116 raft_consensus.cc:2243] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.442884 27116 raft_consensus.cc:2272] T 7897cdd045b54a56b67292d58c9a1528 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.445706 27116 tablet_replica.cc:333] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.446587 27116 raft_consensus.cc:2243] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.447234 27116 raft_consensus.cc:2272] T 789ada7109834532b815f3b0bcb5e69f P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:13.448407  7067 consensus_peers.cc:597] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d -> Peer 378f6aabcd114831bf096f436d7ca046 (127.26.123.2:46857): Couldn't send request to peer 378f6aabcd114831bf096f436d7ca046. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.450131 27116 tablet_replica.cc:333] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.450889 27116 raft_consensus.cc:2243] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.451922 27116 raft_consensus.cc:2272] T bcdfaa3c489a4693a721d0417519e71b P 378f6aabcd114831bf096f436d7ca046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.454638 27116 tablet_replica.cc:333] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.455281 27116 raft_consensus.cc:2243] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.455854 27116 raft_consensus.cc:2272] T b7ba6f81b56943a88a91bdc549a91976 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.458407 27116 tablet_replica.cc:333] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.459044 27116 raft_consensus.cc:2243] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.460111 27116 raft_consensus.cc:2272] T f812b8f6840f46bcb4c4d2a551fc24d1 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.462584 27116 tablet_replica.cc:333] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.463192 27116 raft_consensus.cc:2243] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.463691 27116 raft_consensus.cc:2272] T 9b75463bdf994b82934830b5ef7daef5 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.466094 27116 tablet_replica.cc:333] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.466688 27116 raft_consensus.cc:2243] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.467190 27116 raft_consensus.cc:2272] T 7daed065ac5140b2988d4a834806283a P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.469640 27116 tablet_replica.cc:333] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.470302 27116 raft_consensus.cc:2243] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.470837 27116 raft_consensus.cc:2272] T cc4a77ec797341359459f9036e8984ed P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.473225 27116 tablet_replica.cc:333] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.473790 27116 raft_consensus.cc:2243] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.474301 27116 raft_consensus.cc:2272] T 20b9ed72120b44d39a1260902d5371a9 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.476692 27116 tablet_replica.cc:333] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.477234 27116 raft_consensus.cc:2243] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.478317 27116 raft_consensus.cc:2272] T 35cb8325a0024767ad2aeabad1666a60 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.481242 27116 tablet_replica.cc:333] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.481938 27116 raft_consensus.cc:2243] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.482978 27116 raft_consensus.cc:2272] T 839e0ad151884d3587e3686965c0eda4 P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.486471 27116 tablet_replica.cc:333] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.487285 27116 raft_consensus.cc:2243] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.487983 27116 raft_consensus.cc:2272] T c07a49706e1d448fb18cb100ecc08e72 P 378f6aabcd114831bf096f436d7ca046 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.491520 27116 tablet_replica.cc:333] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.492208 27116 raft_consensus.cc:2243] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.493342 27116 raft_consensus.cc:2272] T 3bf2e5ad820b4050a741334066504e8f P 378f6aabcd114831bf096f436d7ca046 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.496304 27116 tablet_replica.cc:333] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046: stopping tablet replica
I20260709 12:22:13.496914 27116 raft_consensus.cc:2243] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.497588 27116 raft_consensus.cc:2272] T 8b1d3b5a577d46029b259dc75f6dd66c P 378f6aabcd114831bf096f436d7ca046 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.545253 27116 tablet_server.cc:196] TabletServer@127.26.123.2:0 shutdown complete.
W20260709 12:22:13.551712  7065 consensus_peers.cc:597] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d -> Peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Couldn't send request to peer 90de1cd7a5db4da3bf6d6eb2faade974. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:39251: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.597671 27116 tablet_server.cc:179] TabletServer@127.26.123.3:0 shutting down...
I20260709 12:22:13.632076 27116 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:13.632866 27116 tablet_replica.cc:333] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.633679 27116 raft_consensus.cc:2243] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.634310 27116 raft_consensus.cc:2272] T 6fa7e02e304f48bead9b2711a3de29f0 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.636946 27116 tablet_replica.cc:333] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.637609 27116 raft_consensus.cc:2243] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.638177 27116 raft_consensus.cc:2272] T 9654410a8c924c168ded45f76f55c7b9 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.641216 27116 tablet_replica.cc:333] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.642138 27116 raft_consensus.cc:2243] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.642933 27116 raft_consensus.cc:2272] T 1e875b8b9fb640daa6d3db4b5f421dec P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:13.643709  7065 consensus_peers.cc:597] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d -> Peer 90de1cd7a5db4da3bf6d6eb2faade974 (127.26.123.1:39251): Couldn't send request to peer 90de1cd7a5db4da3bf6d6eb2faade974. Status: Network error: Client connection negotiation failed: client connection to 127.26.123.1:39251: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:13.645757 27116 tablet_replica.cc:333] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.646477 27116 raft_consensus.cc:2243] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.646946 27116 raft_consensus.cc:2272] T 8ef03d76aee04b3ca1ed0203616c5510 P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.649168 27116 tablet_replica.cc:333] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.649814 27116 raft_consensus.cc:2243] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.650331 27116 raft_consensus.cc:2272] T 7897cdd045b54a56b67292d58c9a1528 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.652788 27116 tablet_replica.cc:333] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.653400 27116 raft_consensus.cc:2243] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.654358 27116 raft_consensus.cc:2272] T b7ba6f81b56943a88a91bdc549a91976 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.656752 27116 tablet_replica.cc:333] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.657294 27116 raft_consensus.cc:2243] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.657779 27116 raft_consensus.cc:2272] T f812b8f6840f46bcb4c4d2a551fc24d1 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.660070 27116 tablet_replica.cc:333] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.660651 27116 raft_consensus.cc:2243] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.661167 27116 raft_consensus.cc:2272] T 839e0ad151884d3587e3686965c0eda4 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.663548 27116 tablet_replica.cc:333] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.664146 27116 raft_consensus.cc:2243] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.665014 27116 raft_consensus.cc:2272] T c07a49706e1d448fb18cb100ecc08e72 P 615cf24a2e7843d88c0371f123d9101d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.667462 27116 tablet_replica.cc:333] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.668010 27116 raft_consensus.cc:2243] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.668422 27116 raft_consensus.cc:2272] T 789ada7109834532b815f3b0bcb5e69f P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.670647 27116 tablet_replica.cc:333] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.671177 27116 raft_consensus.cc:2243] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.671583 27116 raft_consensus.cc:2272] T bcdfaa3c489a4693a721d0417519e71b P 615cf24a2e7843d88c0371f123d9101d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.673700 27116 tablet_replica.cc:333] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.674257 27116 raft_consensus.cc:2243] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.674734 27116 raft_consensus.cc:2272] T cc4a77ec797341359459f9036e8984ed P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.676828 27116 tablet_replica.cc:333] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.677335 27116 raft_consensus.cc:2243] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.677821 27116 raft_consensus.cc:2272] T 20b9ed72120b44d39a1260902d5371a9 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.680085 27116 tablet_replica.cc:333] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.680562 27116 raft_consensus.cc:2243] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.681005 27116 raft_consensus.cc:2272] T 35cb8325a0024767ad2aeabad1666a60 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.683202 27116 tablet_replica.cc:333] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.683681 27116 raft_consensus.cc:2243] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.684136 27116 raft_consensus.cc:2272] T 3bf2e5ad820b4050a741334066504e8f P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.686386 27116 tablet_replica.cc:333] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.686831 27116 raft_consensus.cc:2243] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.687754 27116 raft_consensus.cc:2272] T 8b1d3b5a577d46029b259dc75f6dd66c P 615cf24a2e7843d88c0371f123d9101d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.690390 27116 tablet_replica.cc:333] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.691093 27116 raft_consensus.cc:2243] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.692230 27116 raft_consensus.cc:2272] T 9b75463bdf994b82934830b5ef7daef5 P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.694559 27116 tablet_replica.cc:333] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d: stopping tablet replica
I20260709 12:22:13.695047 27116 raft_consensus.cc:2243] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:13.695535 27116 raft_consensus.cc:2272] T 7daed065ac5140b2988d4a834806283a P 615cf24a2e7843d88c0371f123d9101d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.739569 27116 tablet_server.cc:196] TabletServer@127.26.123.3:0 shutdown complete.
I20260709 12:22:13.782217 27116 master.cc:562] Master@127.26.123.62:45429 shutting down...
I20260709 12:22:13.803390 27116 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:13.804124 27116 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:13.804471 27116 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2efc56154eae451eb395adff577a7ab5: stopping tablet replica
I20260709 12:22:13.825393 27116 master.cc:584] Master@127.26.123.62:45429 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (20731 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (42113 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (404381 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 12:22:13.884604 27116 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2257 messages since previous log ~32 seconds ago
I20260709 12:22:13.884848 27116 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2257 messages since previous log ~32 seconds ago
I20260709 12:22:13.885038 27116 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2254 messages since previous log ~32 seconds ago
I20260709 12:22:13.885239 27116 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:22:13.885437 27116 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:22:13.885632 27116 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 5 messages since previous log ~187 seconds ago
I20260709 12:22:13.885927 27116 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 5 messages since previous log ~187 seconds ago
I20260709 12:22:13.886165 27116 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 9 messages since previous log ~207 seconds ago
I20260709 12:22:13.886430 27116 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 9 messages since previous log ~0 seconds ago
I20260709 12:22:13.886734 27116 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~19 seconds ago
